ONEST Starter Pack
  • Introduction
    • ONEST- Focus Areas
    • Core Principles
    • Roles You Can Play
  • Join the network
  • Network Observability
    • Introduction
    • Telemetry Implementation
    • Transaction Dashboard
  • Learn
    • Beckn Protocol - Video Library
      • Chapter 1: Open Networks as a Digital Marketplace
      • Chapter 2: Beckn – An open protocol for decentralized commerce
      • Chapter 3: Beckn Ecosystem Architecture
      • Chapter 4: Specification and Support Layer
      • Chapter 5: Transaction Layer
      • Chapter 6: Implementing Beckn Protocol
      • Chapter 7: Network Topologies
      • Chapter 8: Beckn API Overview
      • Chapter 9: Example of a Retail Transaction
      • Chapter 10: Example of a Logistics Transaction
      • Chapter 11: Example of a Transaction using Interactive UI
    • Getting started on DSEP - Video Library
      • Chapter 1: Understanding Challenges in the Ecosystem
      • Chapter 2: Open Networks for Skilling and Education
      • Chapter 3: DSEP Overview
      • Chapter 4: DSEP Specification and Network Architecture
      • Chapter 5: DSEP APIs
      • Chapter 6: DSEP Schema
      • Courses and Training Transaction Workflow
      • Mentorships and Coaching Transaction Workflow
      • Scholarships and Grants Transaction Workflow
      • Internships and Jobs Transaction Workflow
    • Reference Implementation Guides
      • Financial Support (DSEP 1.0)
      • Skilling Experience (DSEP 1.0)
        • Course with scholarship
      • Work Opportunities (DSEP 2.0)
      • Suggested Form Structure for xInput
    • Understanding the APIs
    • Developing the APIs
      • Work Opportunities (DSEP 2.0)
      • Financial support, Skilling (DSEP 1.0)
    • Integration of Adaptors
      • Protocol Server
        • Introduction & Features
        • Architecture
        • Installation
        • Testing
        • Generate Keys
        • Local Tunnelling and Nginx Setup
        • FAQs
      • Job Manager
      • Profile Manager
      • AI Matching Engine
  • Community
    • Contribution Guide
    • Discussion Forum
    • Cohort calls
    • Issues Board
  • HELP
    • FAQs
Powered by GitBook
On this page
  • 1. Using the Protocol Server
  • 2. Using the Open Network Telemetry SDK
  • 3. Using the API
  • Telemetry Event
  • Test Telemetry Generation

Was this helpful?

Edit on GitHub
  1. Network Observability

Telemetry Implementation

Last updated 5 months ago

Was this helpful?

Following are the 3 ways to generate and send telemetry events. However note that this is applicable only when operating on DSEP 1.0

1. Using the Protocol Server

Protocol Server is enhanced to emit telemetry and send to ONEST Obsrv (the data platform where telemetry events are captured and stored). Use this to get the updated protocol server code and use it. Backup the logs generated by protocol server till now before doing a deployment. Extract the API data from the backed up logs and ingest #using-the-protocol-server Add the following configuration in default-bap-client.yml , default-bap-network.yml , default-bpp-client.yml , default-bpp-network.yml files and deploy the protocol server.

telemetry:
    network:
      url: "TELEMETRY_SERVER_URL"
    # raw events are optional
    raw:
      url: ""  
    batchSize: 100
    # In minutes
    syncInterval: 5
    # Either can be LOCAL or REDIS
    storageType: "LOCAL"
    backupFilePath: "backups"
    redis:
      db: 4
    messageProperties: []
 service: 
   name: "network_service"
   version: "1.0.0"

The protocol server will generate telemetry events and send them in batches of 100 or every 5 minutes to ONEST Obsrv.

2. Using the Open Network Telemetry SDK

Open Network Telemetry SDK is a javascript SDK which takes API request message and context object and generates the telemetry events.

3. Using the API

Use the following API to send telemetry events to ONEST Obsrv.

API curl command

curl --location '{TELEMETRY_SERVER_URL}' \
--header 'Content-Type: application/json' \
--data '{
    "data": {
        "id": "onest-batch-1",
        "events": [{}]
    }
}'

Note: Send the telemetry events in a batch size of 50, 100 or send once in every 30 min.

Sample API Payloads

curl --location '{TELEMETRY_SERVER_URL}' \
--header 'Content-Type: application/json' \
--data '{
    "data": {
        "id": "onest-batch-1-1",
        "events": [
            {
                "resourceSpans": [
                    {
                        "resource": {
                            "attributes": [
                                {
                                    "key": "eid",
                                    "value": {
                                        "stringValue": "API"
                                    }
                                },
                                {
                                    "key": "producer",
                                    "value": {
                                        "stringValue": "fs-ps-bap-network.onest.network"
                                    }
                                },
                                {
                                    "key": "domain",
                                    "value": {
                                        "stringValue": "ONDC:ONEST12"
                                    }
                                }
                            ]
                        },
                        "scopeSpans": [
                            {
                                "scope": {
                                    "name": "network_service",
                                    "version": "1.0.0",
                                    "attributes": [
                                        {
                                            "key": "scope_uuid",
                                            "value": {
                                                "stringValue": "640a7f6438e722b31a8b44e23762767f"
                                            }
                                        },
                                        {
                                            "key": "checksum",
                                            "value": {
                                                "stringValue": ""
                                            }
                                        },
                                        {
                                            "key": "count",
                                            "value": {
                                                "intValue": 1
                                            }
                                        }
                                    ]
                                },
                                "spans": [
                                    {
                                        "name": "search",
                                        "traceId": "15e956e5-2a8d-4a3a-8556-e33108e70226",
                                        "spanId": "9509845c-5a32-4a8c-b5bd-78a1de071bd6",
                                        "span_uuid": "b63cc6bcc452a73fd479f2e888a5b044",
                                        "startTimeUnixNano": "1708691608401",
                                        "endTimeUnixNano": "1708691609440",
                                        "status": "Ok",
                                        "attributes": [
                                            {
                                                "key": "sender.id",
                                                "value": {
                                                    "stringValue": "fs-ps-bap-network.onest.network"
                                                }
                                            },
                                            {
                                                "key": "recipient.id",
                                                "value": {
                                                    "stringValue": "fs-ps-bpp-network.onest.network"
                                                }
                                            },
                                            {
                                                "key": "sender.uri",
                                                "value": {
                                                    "stringValue": "https://fs-ps-bap-network.onest.network/"
                                                }
                                            },
                                            {
                                                "key": "recipient.uri",
                                                "value": {
                                                    "stringValue": "https://fs-ps-bpp-network.onest.network"
                                                }
                                            },
                                            {
                                                "key": "sender.type",
                                                "value": {
                                                    "stringValue": "seeker"
                                                }
                                            },
                                            {
                                                "key": "recipient.type",
                                                "value": {
                                                    "stringValue": "provider"
                                                }
                                            },
                                            {
                                                "key": "observedTimeUnixNano",
                                                "value": {
                                                    "intValue": 1708691601754
                                                }
                                            },
                                            {
                                                "key": "http.request.method",
                                                "value": {
                                                    "stringValue": "POST"
                                                }
                                            },
                                            {
                                                "key": "http.request.route",
                                                "value": {
                                                    "stringValue": "/search"
                                                }
                                            },
                                            {
                                                "key": "http.request.host",
                                                "value": {
                                                    "stringValue": "8da0-49-207-215-34.ngrok-free.app"
                                                }
                                            },
                                            {
                                                "key": "http.request.scheme",
                                                "value": {
                                                    "stringValue": "http"
                                                }
                                            },
                                            {
                                                "key": "http.status.code",
                                                "value": {
                                                    "intValue": 200
                                                }
                                            }
                                        ],
                                        "events": []
                                    }
                                ]
                            }
                        ]
                    }
                ]
            }
        ]
    }
}'

Telemetry Event

{
 "resourceSpans": [{
   "resource": { // Required. Contextual information as per envelope structure
     Attribute("eid", String), // Required. Event ID - must be API for all api events
     Attribute("producer", String), // Required. Identifier of the system that produced this event
     Attribute("domain", String), // Required. Domain where the event has occurred.
   }, 
   "scopeSpans": [{
     "scope": { // Optional. Contextual information as per envelope structure
       "name": String, 
       "version": String, 
       "attributes": [
         Attribute("scope_uuid", String), // Optional. unique id for the batch for idempotency
         Attribute("checksum", String), // Optional. checksum to enable tampering checks
         Attribute("count", String), // Optional. Total count of spans in this batch 
       ]
     },
     "spans": [{ // Required. One or more API events in detail
       "name": String, // Required. API Name
       "traceId": String, // Required. Unique ID of the entire transaction
       "spanId": String, // Required. Unique ID of the API event call
       "span_uuid": String, //Required. Unique identifier for this span record
       "startTimeUnixNano": String, // Required. Start time of the API call in nano-seconds
       "endTimeUnixNano": String, // Required. End time of the API call in nano-seconds
       "status": String, // Required. one of Unset, Error, Ok
       "attributes": [ // Required. List of attributes providing additional details about the span
         Attribute("observedTimeUnixNano", String), // Required. Event generated time as ISO datetime
         Attribute("sender.id", String), // Required. Identifier of the system initiated the API call
         Attribute("recipient.id", String), // Required. Identifier of the system that is expected to be the recipient of the API call
         Attribute("http.method", String), // Required. Http method one of GET/POST/PATCH/DELETE etc
         Attribute("http.server_name", String), // Optional. Server name if any
         Attribute("http.route", String), // Required. URL of the request
         Attribute("http.scheme", String), // Optional. Is the call http or https
         Attribute("http.host", String), // Required. Host ip or domain name
         Attribute("http.status.code", Int), // Required. Status code of the API call
       ],
       "events": [{ // Optional. Capture additional API specific data like errors, req & res data, or multiple events that occurred during the API processing
         "name": String, // Required. Name of the additional data or event
         "time": String, // Required. Event time as ISO datetime
         "attributes": [Attribute] // Optional. List of attributes 
       }]
     }]
   }]
 }]
}

Example Sample Event

[
    {
        "resourceSpans": [
            {
                "resource": {
                    "attributes": [
                        {
                            "key": "eid",
                            "value": {
                                "stringValue": "API"
                            }
                        },
                        {
                            "key": "producer",
                            "value": {
                                "stringValue": "fs-ps-bap-network.onest.network"
                            }
                        },
                        {
                            "key": "domain",
                            "value": {
                                "stringValue": "ONDC:ONEST12"
                            }
                        }
                    ]
                },
                "scopeSpans": [
                    {
                        "scope": {
                            "name": "network_service",
                            "version": "1.0.0",
                            "attributes": [
                                {
                                    "key": "scope_uuid",
                                    "value": {
                                        "stringValue": "640a7f6438e722b31a8b44e23762767f"
                                    }
                                },
                                {
                                    "key": "checksum",
                                    "value": {
                                        "stringValue": ""
                                    }
                                },
                                {
                                    "key": "count",
                                    "value": {
                                        "intValue": 1
                                    }
                                }
                            ]
                        },
                        "spans": [
                            {
                                "name": "search",
                                "traceId": "15e956e5-2a8d-4a3a-8556-e33108e70226",
                                "spanId": "9509845c-5a32-4a8c-b5bd-78a1de071bd6",
                                "span_uuid": "b63cc6bcc452a73fd479f2e888a5b044",
                                "startTimeUnixNano": "1708691608401",
                                "endTimeUnixNano": "1708691609440",
                                "status": "Ok",
                                "attributes": [
                                    {
                                        "key": "sender.id",
                                        "value": {
                                            "stringValue": "fs-ps-bap-network.onest.network"
                                        }
                                    },
                                    {
                                        "key": "recipient.id",
                                        "value": {
                                            "stringValue": "fs-ps-bpp-network.onest.network"
                                        }
                                    },
                                    {
                                        "key": "sender.uri",
                                        "value": {
                                            "stringValue": "https://fs-ps-bap-network.onest.network/"
                                        }
                                    },
                                    {
                                        "key": "recipient.uri",
                                        "value": {
                                            "stringValue": "https://fs-ps-bpp-network.onest.network"
                                        }
                                    },
                                    {
                                        "key": "sender.type",
                                        "value": {
                                            "stringValue": "seeker"
                                        }
                                    },
                                    {
                                        "key": "recipient.type",
                                        "value": {
                                            "stringValue": "provider"
                                        }
                                    },
                                    {
                                        "key": "observedTimeUnixNano",
                                        "value": {
                                            "intValue": 1708691601754
                                        }
                                    },
                                    {
                                        "key": "http.request.method",
                                        "value": {
                                            "stringValue": "POST"
                                        }
                                    },
                                    {
                                        "key": "http.request.route",
                                        "value": {
                                            "stringValue": "/search"
                                        }
                                    },
                                    {
                                        "key": "http.request.host",
                                        "value": {
                                            "stringValue": "8da0-49-207-215-34.ngrok-free.app"
                                        }
                                    },
                                    {
                                        "key": "http.request.scheme",
                                        "value": {
                                            "stringValue": "http"
                                        }
                                    },
                                    {
                                        "key": "http.status.code",
                                        "value": {
                                            "intValue": 200
                                        }
                                    }
                                ],
                                "events": []
                            }
                        ]
                    }
                ]
            }
        ]
    },
    {
        "resourceSpans": [
            {
                "resource": {
                    "attributes": [
                        {
                            "key": "eid",
                            "value": {
                                "stringValue": "API"
                            }
                        },
                        {
                            "key": "producer",
                            "value": {
                                "stringValue": "fs-ps-bpp-network.onest.network"
                            }
                        },
                        {
                            "key": "domain",
                            "value": {
                                "stringValue": "ONDC:ONEST12"
                            }
                        }
                    ]
                },
                "scopeSpans": [
                    {
                        "scope": {
                            "name": "network_service",
                            "version": "1.0.0",
                            "attributes": [
                                {
                                    "key": "scope_uuid",
                                    "value": {
                                        "stringValue": "640a7f6438e722b31a8b44e23762767f"
                                    }
                                },
                                {
                                    "key": "checksum",
                                    "value": {
                                        "stringValue": ""
                                    }
                                },
                                {
                                    "key": "count",
                                    "value": {
                                        "intValue": 1
                                    }
                                }
                            ]
                        },
                        "spans": [
                            {
                                "name": "on_search",
                                "traceId": "15e956e5-2a8d-4a3a-8556-e33108e70226",
                                "spanId": "9509845c-5a32-4a8c-b5bd-78a1de071bd6",
                                "span_uuid": "b63cc6bcc452a73fd479f2e888a5b044",
                                "startTimeUnixNano": "1708691608401",
                                "endTimeUnixNano": "1708691609440",
                                "status": "Ok",
                                "attributes": [
                                    {
                                        "key": "sender.id",
                                        "value": {
                                            "stringValue": "fs-ps-bpp-network.onest.network"
                                        }
                                    },
                                    {
                                        "key": "recipient.id",
                                        "value": {
                                            "stringValue": "fs-ps-bap-network.onest.network"
                                        }
                                    },
                                    {
                                        "key": "sender.uri",
                                        "value": {
                                            "stringValue": "https://fs-ps-bpp-network.onest.network/"
                                        }
                                    },
                                    {
                                        "key": "recipient.uri",
                                        "value": {
                                            "stringValue": "https://fs-ps-bap-network.onest.network"
                                        }
                                    },
                                    {
                                        "key": "sender.type",
                                        "value": {
                                            "stringValue": "provider"
                                        }
                                    },
                                    {
                                        "key": "recipient.type",
                                        "value": {
                                            "stringValue": "seeker"
                                        }
                                    },
                                    {
                                        "key": "observedTimeUnixNano",
                                        "value": {
                                            "intValue": 1708691601754
                                        }
                                    },
                                    {
                                        "key": "http.request.method",
                                        "value": {
                                            "stringValue": "POST"
                                        }
                                    },
                                    {
                                        "key": "http.request.route",
                                        "value": {
                                            "stringValue": "/on_search"
                                        }
                                    },
                                    {
                                        "key": "http.request.host",
                                        "value": {
                                            "stringValue": "8da0-49-207-215-34.ngrok-free.app"
                                        }
                                    },
                                    {
                                        "key": "http.request.scheme",
                                        "value": {
                                            "stringValue": "http"
                                        }
                                    },
                                    {
                                        "key": "http.status.code",
                                        "value": {
                                            "intValue": 200
                                        }
                                    }
                                ],
                                "events": [
                                    {
                                        "name": "error",
                                        "time": "2023-04-29T18:52:58.114561Z",
                                        "attributes": [
                                            {
                                                "key": "type",
                                                "value": {
                                                    "stringValue": "CORE-ERROR"
                                                }
                                            },
                                            {
                                                "key": "code",
                                                "value": {
                                                    "stringValue": "ERR_BPP_1010"
                                                }
                                            },
                                            {
                                                "key": "msg",
                                                "value": {
                                                    "stringValue": "Unable to fetch the catalog. Please try later"
                                                }
                                            }
                                        ]
                                    }
                                ]
                            }
                        ]
                    }
                ]
            }
        ]
    }
]

Test Telemetry Generation

  1. Set the web hook URL as Telemetry URL.

Note: Configure the telemetry URL to "" for lower environments such as dev, staging, and preprod during testing. For the production telemetry URL, contact the ONEST team when the application is ready for go-live

Please refer to the for more details.

Refer to this , for more details about the open network telemetry specifications

Use the to generate an endpoint URL.

Ensure telemetry events are getting generated for all the relevant APIs in this .

GitHub Repo
https://data.onest.network/data/v1/in/onest-telemetry-testing
readme
repo
https://webhook.site/
format