Telemetry Implementation
Following are the 3 ways to generate and send the telemetry events
1. Using the Protocol Server
Protocol Server is enhanced to emit telemetry and send to ONEST Obsrv (the data platform where telemetry events are captured and stored). Use this GitHub Repo to get the updated protocol server code and use it.
Backup the logs generated by protocol server till now before doing a deployment. Extract the API data from the backed up logs and ingest #using-the-protocol-server
Add the following configuration in default-bap-client.yml , default-bap-network.yml , default-bpp-client.yml , default-bpp-network.yml files and deploy the protocol server.
telemetry:
network:
url: "TELEMETRY_SERVER_URL"
# raw events are optional
raw:
url: ""
batchSize: 100
# In minutes
syncInterval: 5
# Either can be LOCAL or REDIS
storageType: "LOCAL"
backupFilePath: "backups"
redis:
db: 4
messageProperties: []
service:
name: "network_service"
version: "1.0.0"The protocol server will generate telemetry events and send them in batches of 100 or every 5 minutes to ONEST Obsrv.
Note: For TELEMETRY_SERVER_URL, please reach out to ONEST team.
2. Using the Open Network Telemetry SDK
Open Network Telemetry SDK is a javascript SDK which takes API request message and context object and generates the telemetry events.
Please refer to the readme for more details.
3. Using the API
Use the following API to send telemetry events to ONEST Obsrv.
API curl command
curl --location '{TELEMETRY_SERVER_URL}' \
--header 'Content-Type: application/json' \
--data '{
"data": {
"id": "onest-batch-1",
"events": [{}]
}
}'Note: Send the telemetry events in a batch size of 50, 100 or send once in every 30 min.
Sample API Payloads
curl --location '{TELEMETRY_SERVER_URL}' \
--header 'Content-Type: application/json' \
--data '{
"data": {
"id": "onest-batch-1-1",
"events": [
{
"resourceSpans": [
{
"resource": {
"attributes": [
{
"key": "eid",
"value": {
"stringValue": "API"
}
},
{
"key": "producer",
"value": {
"stringValue": "fs-ps-bap-network.onest.network"
}
},
{
"key": "domain",
"value": {
"stringValue": "onest:financial-support"
}
}
]
},
"scopeSpans": [
{
"scope": {
"name": "network_service",
"version": "1.0.0",
"attributes": [
{
"key": "scope_uuid",
"value": {
"stringValue": "640a7f6438e722b31a8b44e23762767f"
}
},
{
"key": "checksum",
"value": {
"stringValue": ""
}
},
{
"key": "count",
"value": {
"intValue": 1
}
}
]
},
"spans": [
{
"name": "search",
"traceId": "15e956e5-2a8d-4a3a-8556-e33108e70226",
"spanId": "9509845c-5a32-4a8c-b5bd-78a1de071bd6",
"span_uuid": "b63cc6bcc452a73fd479f2e888a5b044",
"startTimeUnixNano": "1708691608401",
"endTimeUnixNano": "1708691609440",
"status": "Ok",
"attributes": [
{
"key": "sender.id",
"value": {
"stringValue": "fs-ps-bap-network.onest.network"
}
},
{
"key": "recipient.id",
"value": {
"stringValue": "fs-ps-bpp-network.onest.network"
}
},
{
"key": "sender.uri",
"value": {
"stringValue": "https://fs-ps-bap-network.onest.network/"
}
},
{
"key": "recipient.uri",
"value": {
"stringValue": "https://fs-ps-bpp-network.onest.network"
}
},
{
"key": "sender.type",
"value": {
"stringValue": "seeker"
}
},
{
"key": "recipient.type",
"value": {
"stringValue": "provider"
}
},
{
"key": "observedTimeUnixNano",
"value": {
"intValue": 1708691601754
}
},
{
"key": "http.request.method",
"value": {
"stringValue": "POST"
}
},
{
"key": "http.request.route",
"value": {
"stringValue": "/search"
}
},
{
"key": "http.request.host",
"value": {
"stringValue": "8da0-49-207-215-34.ngrok-free.app"
}
},
{
"key": "http.request.scheme",
"value": {
"stringValue": "http"
}
},
{
"key": "http.status.code",
"value": {
"intValue": 200
}
}
],
"events": []
}
]
}
]
}
]
}
]
}
}'Telemetry Event
{
"resourceSpans": [{
"resource": { // Required. Contextual information as per envelope structure
Attribute("eid", String), // Required. Event ID - must be API for all api events
Attribute("producer", String), // Required. Identifier of the system that produced this event
Attribute("domain", String), // Required. Domain where the event has occurred.
},
"scopeSpans": [{
"scope": { // Optional. Contextual information as per envelope structure
"name": String,
"version": String,
"attributes": [
Attribute("scope_uuid", String), // Optional. unique id for the batch for idempotency
Attribute("checksum", String), // Optional. checksum to enable tampering checks
Attribute("count", String), // Optional. Total count of spans in this batch
]
},
"spans": [{ // Required. One or more API events in detail
"name": String, // Required. API Name
"traceId": String, // Required. Unique ID of the entire transaction
"spanId": String, // Required. Unique ID of the API event call
"span_uuid": String, //Required. Unique identifier for this span record
"startTimeUnixNano": String, // Required. Start time of the API call in nano-seconds
"endTimeUnixNano": String, // Required. End time of the API call in nano-seconds
"status": String, // Required. one of Unset, Error, Ok
"attributes": [ // Required. List of attributes providing additional details about the span
Attribute("observedTimeUnixNano", String), // Required. Event generated time as ISO datetime
Attribute("sender.id", String), // Required. Identifier of the system initiated the API call
Attribute("recipient.id", String), // Required. Identifier of the system that is expected to be the recipient of the API call
Attribute("http.method", String), // Required. Http method one of GET/POST/PATCH/DELETE etc
Attribute("http.server_name", String), // Optional. Server name if any
Attribute("http.route", String), // Required. URL of the request
Attribute("http.scheme", String), // Optional. Is the call http or https
Attribute("http.host", String), // Required. Host ip or domain name
Attribute("http.status.code", Int), // Required. Status code of the API call
],
"events": [{ // Optional. Capture additional API specific data like errors, req & res data, or multiple events that occurred during the API processing
"name": String, // Required. Name of the additional data or event
"time": String, // Required. Event time as ISO datetime
"attributes": [Attribute] // Optional. List of attributes
}]
}]
}]
}]
}
Example Sample Event
[
{
"resourceSpans": [
{
"resource": {
"attributes": [
{
"key": "eid",
"value": {
"stringValue": "API"
}
},
{
"key": "producer",
"value": {
"stringValue": "fs-ps-bap-network.onest.network"
}
},
{
"key": "domain",
"value": {
"stringValue": "onest:financial-support"
}
}
]
},
"scopeSpans": [
{
"scope": {
"name": "network_service",
"version": "1.0.0",
"attributes": [
{
"key": "scope_uuid",
"value": {
"stringValue": "640a7f6438e722b31a8b44e23762767f"
}
},
{
"key": "checksum",
"value": {
"stringValue": ""
}
},
{
"key": "count",
"value": {
"intValue": 1
}
}
]
},
"spans": [
{
"name": "search",
"traceId": "15e956e5-2a8d-4a3a-8556-e33108e70226",
"spanId": "9509845c-5a32-4a8c-b5bd-78a1de071bd6",
"span_uuid": "b63cc6bcc452a73fd479f2e888a5b044",
"startTimeUnixNano": "1708691608401",
"endTimeUnixNano": "1708691609440",
"status": "Ok",
"attributes": [
{
"key": "sender.id",
"value": {
"stringValue": "fs-ps-bap-network.onest.network"
}
},
{
"key": "recipient.id",
"value": {
"stringValue": "fs-ps-bpp-network.onest.network"
}
},
{
"key": "sender.uri",
"value": {
"stringValue": "https://fs-ps-bap-network.onest.network/"
}
},
{
"key": "recipient.uri",
"value": {
"stringValue": "https://fs-ps-bpp-network.onest.network"
}
},
{
"key": "sender.type",
"value": {
"stringValue": "seeker"
}
},
{
"key": "recipient.type",
"value": {
"stringValue": "provider"
}
},
{
"key": "observedTimeUnixNano",
"value": {
"intValue": 1708691601754
}
},
{
"key": "http.request.method",
"value": {
"stringValue": "POST"
}
},
{
"key": "http.request.route",
"value": {
"stringValue": "/search"
}
},
{
"key": "http.request.host",
"value": {
"stringValue": "8da0-49-207-215-34.ngrok-free.app"
}
},
{
"key": "http.request.scheme",
"value": {
"stringValue": "http"
}
},
{
"key": "http.status.code",
"value": {
"intValue": 200
}
}
],
"events": []
}
]
}
]
}
]
},
{
"resourceSpans": [
{
"resource": {
"attributes": [
{
"key": "eid",
"value": {
"stringValue": "API"
}
},
{
"key": "producer",
"value": {
"stringValue": "fs-ps-bpp-network.onest.network"
}
},
{
"key": "domain",
"value": {
"stringValue": "onest:financial-support"
}
}
]
},
"scopeSpans": [
{
"scope": {
"name": "network_service",
"version": "1.0.0",
"attributes": [
{
"key": "scope_uuid",
"value": {
"stringValue": "640a7f6438e722b31a8b44e23762767f"
}
},
{
"key": "checksum",
"value": {
"stringValue": ""
}
},
{
"key": "count",
"value": {
"intValue": 1
}
}
]
},
"spans": [
{
"name": "on_search",
"traceId": "15e956e5-2a8d-4a3a-8556-e33108e70226",
"spanId": "9509845c-5a32-4a8c-b5bd-78a1de071bd6",
"span_uuid": "b63cc6bcc452a73fd479f2e888a5b044",
"startTimeUnixNano": "1708691608401",
"endTimeUnixNano": "1708691609440",
"status": "Ok",
"attributes": [
{
"key": "sender.id",
"value": {
"stringValue": "fs-ps-bpp-network.onest.network"
}
},
{
"key": "recipient.id",
"value": {
"stringValue": "fs-ps-bap-network.onest.network"
}
},
{
"key": "sender.uri",
"value": {
"stringValue": "https://fs-ps-bpp-network.onest.network/"
}
},
{
"key": "recipient.uri",
"value": {
"stringValue": "https://fs-ps-bap-network.onest.network"
}
},
{
"key": "sender.type",
"value": {
"stringValue": "provider"
}
},
{
"key": "recipient.type",
"value": {
"stringValue": "seeker"
}
},
{
"key": "observedTimeUnixNano",
"value": {
"intValue": 1708691601754
}
},
{
"key": "http.request.method",
"value": {
"stringValue": "POST"
}
},
{
"key": "http.request.route",
"value": {
"stringValue": "/on_search"
}
},
{
"key": "http.request.host",
"value": {
"stringValue": "8da0-49-207-215-34.ngrok-free.app"
}
},
{
"key": "http.request.scheme",
"value": {
"stringValue": "http"
}
},
{
"key": "http.status.code",
"value": {
"intValue": 200
}
}
],
"events": [
{
"name": "error",
"time": "2023-04-29T18:52:58.114561Z",
"attributes": [
{
"key": "type",
"value": {
"stringValue": "CORE-ERROR"
}
},
{
"key": "code",
"value": {
"stringValue": "ERR_BPP_1010"
}
},
{
"key": "msg",
"value": {
"stringValue": "Unable to fetch the catalog. Please try later"
}
}
]
}
]
}
]
}
]
}
]
}
]Refer to this repo, for more details about the open network telemetry specifications
Test Telemetry Generation
Use the https://webhook.site/ to generate an endpoint URL.
Set the web hook URL as Telemetry URL.
Ensure telemetry events are getting generated for all the relevant APIs in this format.
Last updated