# Telemetry Implementation Following are the 3 ways to generate and send telemetry events. However note that this is applicable only when operating on DSEP 1.0 ### 1. Using the Protocol Server Protocol Server is enhanced to emit telemetry and send to ONEST Obsrv (the data platform where telemetry events are captured and stored). Use this[ GitHub Repo](https://github.com/ONEST-Network/protocol-server) to get the updated protocol server code and use it. \ \ Backup the logs generated by protocol server till now before doing a deployment. Extract the API data from the backed up logs and ingest [#using-the-protocol-server](#using-the-protocol-server "mention")\ \ Add the following configuration in `default-bap-client.yml` , `default-bap-network.yml` , `default-bpp-client.yml` , `default-bpp-network.yml` files and deploy the protocol server. ```yaml telemetry: network: url: "TELEMETRY_SERVER_URL" # raw events are optional raw: url: "" batchSize: 100 # In minutes syncInterval: 5 # Either can be LOCAL or REDIS storageType: "LOCAL" backupFilePath: "backups" redis: db: 4 messageProperties: [] service: name: "network_service" version: "1.0.0" ``` The protocol server will generate telemetry events and send them in batches of 100 or every 5 minutes to ONEST Obsrv. **Note:** Configure the telemetry URL to "" for lower environments such as dev, staging, and preprod during testing. For the production telemetry URL, contact the ONEST team when the application is ready for go-live ### 2. Using the Open Network Telemetry SDK Open Network Telemetry SDK is a javascript SDK which takes API request message and context object and generates the telemetry events. Please refer to the [readme](https://github.com/Sunbird-Obsrv/open-network-telemetry) for more details. ### 3. Using the API Use the following API to send telemetry events to ONEST Obsrv. **API curl command** ```url curl --location '{TELEMETRY_SERVER_URL}' \ --header 'Content-Type: application/json' \ --data '{ "data": { "id": "onest-batch-1", "events": [{}] } }' ``` **Note:** Send the telemetry events in a batch size of **50, 100** or send **once in every 30 min**. **Sample API Payloads** ```json curl --location '{TELEMETRY_SERVER_URL}' \ --header 'Content-Type: application/json' \ --data '{ "data": { "id": "onest-batch-1-1", "events": [ { "resourceSpans": [ { "resource": { "attributes": [ { "key": "eid", "value": { "stringValue": "API" } }, { "key": "producer", "value": { "stringValue": "fs-ps-bap-network.onest.network" } }, { "key": "domain", "value": { "stringValue": "ONDC:ONEST12" } } ] }, "scopeSpans": [ { "scope": { "name": "network_service", "version": "1.0.0", "attributes": [ { "key": "scope_uuid", "value": { "stringValue": "640a7f6438e722b31a8b44e23762767f" } }, { "key": "checksum", "value": { "stringValue": "" } }, { "key": "count", "value": { "intValue": 1 } } ] }, "spans": [ { "name": "search", "traceId": "15e956e5-2a8d-4a3a-8556-e33108e70226", "spanId": "9509845c-5a32-4a8c-b5bd-78a1de071bd6", "span_uuid": "b63cc6bcc452a73fd479f2e888a5b044", "startTimeUnixNano": "1708691608401", "endTimeUnixNano": "1708691609440", "status": "Ok", "attributes": [ { "key": "sender.id", "value": { "stringValue": "fs-ps-bap-network.onest.network" } }, { "key": "recipient.id", "value": { "stringValue": "fs-ps-bpp-network.onest.network" } }, { "key": "sender.uri", "value": { "stringValue": "https://fs-ps-bap-network.onest.network/" } }, { "key": "recipient.uri", "value": { "stringValue": "https://fs-ps-bpp-network.onest.network" } }, { "key": "sender.type", "value": { "stringValue": "seeker" } }, { "key": "recipient.type", "value": { "stringValue": "provider" } }, { "key": "observedTimeUnixNano", "value": { "intValue": 1708691601754 } }, { "key": "http.request.method", "value": { "stringValue": "POST" } }, { "key": "http.request.route", "value": { "stringValue": "/search" } }, { "key": "http.request.host", "value": { "stringValue": "8da0-49-207-215-34.ngrok-free.app" } }, { "key": "http.request.scheme", "value": { "stringValue": "http" } }, { "key": "http.status.code", "value": { "intValue": 200 } } ], "events": [] } ] } ] } ] } ] } }' ``` ### Telemetry Event ```json { "resourceSpans": [{ "resource": { // Required. Contextual information as per envelope structure Attribute("eid", String), // Required. Event ID - must be API for all api events Attribute("producer", String), // Required. Identifier of the system that produced this event Attribute("domain", String), // Required. Domain where the event has occurred. }, "scopeSpans": [{ "scope": { // Optional. Contextual information as per envelope structure "name": String, "version": String, "attributes": [ Attribute("scope_uuid", String), // Optional. unique id for the batch for idempotency Attribute("checksum", String), // Optional. checksum to enable tampering checks Attribute("count", String), // Optional. Total count of spans in this batch ] }, "spans": [{ // Required. One or more API events in detail "name": String, // Required. API Name "traceId": String, // Required. Unique ID of the entire transaction "spanId": String, // Required. Unique ID of the API event call "span_uuid": String, //Required. Unique identifier for this span record "startTimeUnixNano": String, // Required. Start time of the API call in nano-seconds "endTimeUnixNano": String, // Required. End time of the API call in nano-seconds "status": String, // Required. one of Unset, Error, Ok "attributes": [ // Required. List of attributes providing additional details about the span Attribute("observedTimeUnixNano", String), // Required. Event generated time as ISO datetime Attribute("sender.id", String), // Required. Identifier of the system initiated the API call Attribute("recipient.id", String), // Required. Identifier of the system that is expected to be the recipient of the API call Attribute("http.method", String), // Required. Http method one of GET/POST/PATCH/DELETE etc Attribute("http.server_name", String), // Optional. Server name if any Attribute("http.route", String), // Required. URL of the request Attribute("http.scheme", String), // Optional. Is the call http or https Attribute("http.host", String), // Required. Host ip or domain name Attribute("http.status.code", Int), // Required. Status code of the API call ], "events": [{ // Optional. Capture additional API specific data like errors, req & res data, or multiple events that occurred during the API processing "name": String, // Required. Name of the additional data or event "time": String, // Required. Event time as ISO datetime "attributes": [Attribute] // Optional. List of attributes }] }] }] }] } ``` #### Example Sample Event ```json [ { "resourceSpans": [ { "resource": { "attributes": [ { "key": "eid", "value": { "stringValue": "API" } }, { "key": "producer", "value": { "stringValue": "fs-ps-bap-network.onest.network" } }, { "key": "domain", "value": { "stringValue": "ONDC:ONEST12" } } ] }, "scopeSpans": [ { "scope": { "name": "network_service", "version": "1.0.0", "attributes": [ { "key": "scope_uuid", "value": { "stringValue": "640a7f6438e722b31a8b44e23762767f" } }, { "key": "checksum", "value": { "stringValue": "" } }, { "key": "count", "value": { "intValue": 1 } } ] }, "spans": [ { "name": "search", "traceId": "15e956e5-2a8d-4a3a-8556-e33108e70226", "spanId": "9509845c-5a32-4a8c-b5bd-78a1de071bd6", "span_uuid": "b63cc6bcc452a73fd479f2e888a5b044", "startTimeUnixNano": "1708691608401", "endTimeUnixNano": "1708691609440", "status": "Ok", "attributes": [ { "key": "sender.id", "value": { "stringValue": "fs-ps-bap-network.onest.network" } }, { "key": "recipient.id", "value": { "stringValue": "fs-ps-bpp-network.onest.network" } }, { "key": "sender.uri", "value": { "stringValue": "https://fs-ps-bap-network.onest.network/" } }, { "key": "recipient.uri", "value": { "stringValue": "https://fs-ps-bpp-network.onest.network" } }, { "key": "sender.type", "value": { "stringValue": "seeker" } }, { "key": "recipient.type", "value": { "stringValue": "provider" } }, { "key": "observedTimeUnixNano", "value": { "intValue": 1708691601754 } }, { "key": "http.request.method", "value": { "stringValue": "POST" } }, { "key": "http.request.route", "value": { "stringValue": "/search" } }, { "key": "http.request.host", "value": { "stringValue": "8da0-49-207-215-34.ngrok-free.app" } }, { "key": "http.request.scheme", "value": { "stringValue": "http" } }, { "key": "http.status.code", "value": { "intValue": 200 } } ], "events": [] } ] } ] } ] }, { "resourceSpans": [ { "resource": { "attributes": [ { "key": "eid", "value": { "stringValue": "API" } }, { "key": "producer", "value": { "stringValue": "fs-ps-bpp-network.onest.network" } }, { "key": "domain", "value": { "stringValue": "ONDC:ONEST12" } } ] }, "scopeSpans": [ { "scope": { "name": "network_service", "version": "1.0.0", "attributes": [ { "key": "scope_uuid", "value": { "stringValue": "640a7f6438e722b31a8b44e23762767f" } }, { "key": "checksum", "value": { "stringValue": "" } }, { "key": "count", "value": { "intValue": 1 } } ] }, "spans": [ { "name": "on_search", "traceId": "15e956e5-2a8d-4a3a-8556-e33108e70226", "spanId": "9509845c-5a32-4a8c-b5bd-78a1de071bd6", "span_uuid": "b63cc6bcc452a73fd479f2e888a5b044", "startTimeUnixNano": "1708691608401", "endTimeUnixNano": "1708691609440", "status": "Ok", "attributes": [ { "key": "sender.id", "value": { "stringValue": "fs-ps-bpp-network.onest.network" } }, { "key": "recipient.id", "value": { "stringValue": "fs-ps-bap-network.onest.network" } }, { "key": "sender.uri", "value": { "stringValue": "https://fs-ps-bpp-network.onest.network/" } }, { "key": "recipient.uri", "value": { "stringValue": "https://fs-ps-bap-network.onest.network" } }, { "key": "sender.type", "value": { "stringValue": "provider" } }, { "key": "recipient.type", "value": { "stringValue": "seeker" } }, { "key": "observedTimeUnixNano", "value": { "intValue": 1708691601754 } }, { "key": "http.request.method", "value": { "stringValue": "POST" } }, { "key": "http.request.route", "value": { "stringValue": "/on_search" } }, { "key": "http.request.host", "value": { "stringValue": "8da0-49-207-215-34.ngrok-free.app" } }, { "key": "http.request.scheme", "value": { "stringValue": "http" } }, { "key": "http.status.code", "value": { "intValue": 200 } } ], "events": [ { "name": "error", "time": "2023-04-29T18:52:58.114561Z", "attributes": [ { "key": "type", "value": { "stringValue": "CORE-ERROR" } }, { "key": "code", "value": { "stringValue": "ERR_BPP_1010" } }, { "key": "msg", "value": { "stringValue": "Unable to fetch the catalog. Please try later" } } ] } ] } ] } ] } ] } ] ``` Refer to this [repo](https://github.com/Sunbird-Obsrv/network-telemetry-spec), for more details about the open network telemetry specifications ### Test Telemetry Generation 1. Use the to generate an endpoint URL. 2. Set the web hook URL as Telemetry URL. 3. Ensure telemetry events are getting generated for all the relevant APIs in this [format](#telemetry-event).