01 Before you begin #
You need three things:
- A ZenPlus appliance you can reach over HTTP from the machine running your application.
- An account on that appliance that can create ingest keys.
- An application you can add an OpenTelemetry SDK to, or one already emitting OTLP.
Everything in this guide uses your appliance's own base URL — the same address you use to reach the dashboard. Throughout, http://APPLIANCE stands in for it. The APM Settings screen prints these snippets with your real address already filled in.
02 Step 1 — Create an ingest key #
Every producer authenticates with an ingest key. Go to APM → Settings → Ingest keys and choose Create ingest key.
- Name
- How you will recognise it later. Use the thing that will hold it:
prod-checkout-service,staging-collector. - Type
- SDK / Collector issues a
zpi_key for server-side telemetry. Browser RUM issues azpr_key; the trace receiver rejects RUM keys, so pick SDK unless you know otherwise. - Environment
- Stamps spans that do not declare their own
deployment.environment. An explicit attribute from the SDK always wins.
Only a SHA-256 hash is stored. If you lose the plaintext you cannot recover it — revoke the key and issue a new one. Scope keys narrowly enough that revoking one does not silence your whole estate.
03 Step 2 — Configure the SDK #
The appliance's built-in receiver speaks OTLP over HTTP with JSON encoding. Most SDKs default to protobuf, so the protocol must be set explicitly.
export OTEL_EXPORTER_OTLP_ENDPOINT="http://APPLIANCE"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/json"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer zpi_your_key_here"
export OTEL_SERVICE_NAME="checkout-service"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=prod,service.version=1.4.2"
Leaving the protocol at its default. An SDK exporting http/protobuf gets HTTP 415 and the spans are discarded — the application keeps running, no error surfaces in your app, and APM stays empty. If nothing appears after a few minutes, check this first.
Language examples
Python — zero code changes with auto-instrumentation:
pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http
opentelemetry-bootstrap -a install
OTEL_EXPORTER_OTLP_ENDPOINT=http://APPLIANCE \
OTEL_EXPORTER_OTLP_PROTOCOL=http/json \
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer zpi_your_key_here" \
OTEL_SERVICE_NAME=checkout-service \
opentelemetry-instrument python app.py
Node.js:
npm install @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node \
@opentelemetry/exporter-trace-otlp-http
OTEL_EXPORTER_OTLP_ENDPOINT=http://APPLIANCE \
OTEL_EXPORTER_OTLP_PROTOCOL=http/json \
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer zpi_your_key_here" \
OTEL_SERVICE_NAME=api-gateway \
node --require @opentelemetry/auto-instrumentations-node/register app.js
Java:
java -javaagent:opentelemetry-javaagent.jar \
-Dotel.exporter.otlp.endpoint=http://APPLIANCE \
-Dotel.exporter.otlp.protocol=http/json \
-Dotel.exporter.otlp.headers=Authorization=Bearer%20zpi_your_key_here \
-Dotel.service.name=payments-service \
-jar app.jar
04 Step 3 — Verify the first span #
Do not wait on your application. Send one span by hand and read the response:
curl -sS -X POST http://APPLIANCE/v1/traces \
-H "Content-Type: application/json" \
-H "Authorization: Bearer zpi_your_key_here" \
-d '{"resourceSpans":[{"resource":{"attributes":[
{"key":"service.name","value":{"stringValue":"hello-service"}}]},
"scopeSpans":[{"spans":[{
"traceId":"5b8efff798038103d269b633813fc60c",
"spanId":"eee19b7ec3c1b174","name":"GET /health","kind":2,
"startTimeUnixNano":"'$(date +%s)'000000000",
"endTimeUnixNano":"'$(date +%s)'100000000",
"status":{"code":1}}]}]}]}'
| Response | Meaning | What to do |
|---|---|---|
{"partialSuccess":{}} | Accepted | Nothing — the service appears within a minute |
401 | Key missing, malformed, revoked, or of the wrong kind | Check the header spelling and that the key is SDK-type and active |
415 | Protobuf body | Set the protocol to http/json |
400 | Body is not valid JSON | Check for shell quoting damage |
503 | Backpressure | Retry; if persistent, see Troubleshooting |
| SPA HTML | The request never reached the API | The appliance's reverse proxy is missing its /v1/ route |
kind: 2 is a SERVER span. Only SERVER and CONSUMER spans count as inbound requests, so a test span sent with the default kind (INTERNAL) is stored and appears in traces but contributes nothing to the service's rate, errors or duration. This is by design — see Services.
05 Step 4 — Confirm in the UI #
Within about a minute of the first SERVER span:
- APM → Services lists the service with rate, errors, duration and apdex.
- APM → Traces shows the trace in Live mode.
- APM → Settings → Ingest keys shows a last used timestamp on the key. A key that still reads never used has never successfully authenticated — the problem is upstream of ZenPlus.
06 What good instrumentation looks like #
APM reads a handful of OpenTelemetry attributes to do its job. Emitting them well is the difference between a usable module and a pile of spans.
| Attribute | Why ZenPlus needs it | Consequence if missing |
|---|---|---|
service.name | The identity of every row on every screen | Everything lands under unknown |
deployment.environment | Separates prod from staging in filters and SLO scope | Falls back to the key's environment, or unknown |
service.version | Version attribution on error issues | Cannot tell which release introduced an error |
http.route | The templated route, e.g. /orders/ | Every distinct URL becomes its own operation and the list explodes |
http.status_code | Shown on spans and in usage analytics | Status column reads 0 |
db.system, db.operation | Database spans are styled and labelled distinctly in waterfalls | DB calls look like any other span |
exception span events | Error grouping, stack traces, issue detail | Failures still counted, but with no type, message or stack |
http.route must be the route template, not the concrete URL. /orders/8842 as a route means every order id becomes a separate operation, which bloats the rollups and makes the operations table useless. Every mainstream auto-instrumentation gets this right by default — the risk is hand-rolled spans.
07 Sending through the ZenPlus agent #
Hosts that already run the ZenPlus agent can let it forward APM telemetry, so the application does not need to hold an ingest key at all. Agent forwarder health — spans per minute, export errors, spool depth — is reported under APM → Settings → Data quality.
For hosts you provision automatically, use an enrollment token instead of embedding a long-lived key in an image: it expires, is use-capped, and can be revoked before it is ever redeemed. See Settings.