ZenPlus APM

Getting started with APM

From an empty APM module to a service reporting golden signals: create an ingest key, point an OpenTelemetry SDK at the appliance, and verify the first span landed.

Applies to ZenPlus 1.6.0 Updated 2026-08-06 Audience Developers Reading time ~10 min

01 Before you begin #

You need three things:

  • A ZenPlus appliance you can reach over HTTP from the machine running your application.
  • An account on that appliance that can create ingest keys.
  • An application you can add an OpenTelemetry SDK to, or one already emitting OTLP.
Which endpoint do I use?

Everything in this guide uses your appliance's own base URL — the same address you use to reach the dashboard. Throughout, http://APPLIANCE stands in for it. The APM Settings screen prints these snippets with your real address already filled in.

02 Step 1 — Create an ingest key #

Every producer authenticates with an ingest key. Go to APM → Settings → Ingest keys and choose Create ingest key.

Name
How you will recognise it later. Use the thing that will hold it: prod-checkout-service, staging-collector.
Type
SDK / Collector issues a zpi_ key for server-side telemetry. Browser RUM issues a zpr_ key; the trace receiver rejects RUM keys, so pick SDK unless you know otherwise.
Environment
Stamps spans that do not declare their own deployment.environment. An explicit attribute from the SDK always wins.
The key is shown once

Only a SHA-256 hash is stored. If you lose the plaintext you cannot recover it — revoke the key and issue a new one. Scope keys narrowly enough that revoking one does not silence your whole estate.

03 Step 2 — Configure the SDK #

The appliance's built-in receiver speaks OTLP over HTTP with JSON encoding. Most SDKs default to protobuf, so the protocol must be set explicitly.

export OTEL_EXPORTER_OTLP_ENDPOINT="http://APPLIANCE"
export OTEL_EXPORTER_OTLP_PROTOCOL="http/json"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer zpi_your_key_here"
export OTEL_SERVICE_NAME="checkout-service"
export OTEL_RESOURCE_ATTRIBUTES="deployment.environment=prod,service.version=1.4.2"
The single most common setup failure

Leaving the protocol at its default. An SDK exporting http/protobuf gets HTTP 415 and the spans are discarded — the application keeps running, no error surfaces in your app, and APM stays empty. If nothing appears after a few minutes, check this first.

Language examples

Python — zero code changes with auto-instrumentation:

pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http
opentelemetry-bootstrap -a install

OTEL_EXPORTER_OTLP_ENDPOINT=http://APPLIANCE \
OTEL_EXPORTER_OTLP_PROTOCOL=http/json \
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer zpi_your_key_here" \
OTEL_SERVICE_NAME=checkout-service \
opentelemetry-instrument python app.py

Node.js:

npm install @opentelemetry/sdk-node @opentelemetry/auto-instrumentations-node \
            @opentelemetry/exporter-trace-otlp-http

OTEL_EXPORTER_OTLP_ENDPOINT=http://APPLIANCE \
OTEL_EXPORTER_OTLP_PROTOCOL=http/json \
OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer zpi_your_key_here" \
OTEL_SERVICE_NAME=api-gateway \
node --require @opentelemetry/auto-instrumentations-node/register app.js

Java:

java -javaagent:opentelemetry-javaagent.jar \
     -Dotel.exporter.otlp.endpoint=http://APPLIANCE \
     -Dotel.exporter.otlp.protocol=http/json \
     -Dotel.exporter.otlp.headers=Authorization=Bearer%20zpi_your_key_here \
     -Dotel.service.name=payments-service \
     -jar app.jar

04 Step 3 — Verify the first span #

Do not wait on your application. Send one span by hand and read the response:

curl -sS -X POST http://APPLIANCE/v1/traces \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer zpi_your_key_here" \
  -d '{"resourceSpans":[{"resource":{"attributes":[
        {"key":"service.name","value":{"stringValue":"hello-service"}}]},
      "scopeSpans":[{"spans":[{
        "traceId":"5b8efff798038103d269b633813fc60c",
        "spanId":"eee19b7ec3c1b174","name":"GET /health","kind":2,
        "startTimeUnixNano":"'$(date +%s)'000000000",
        "endTimeUnixNano":"'$(date +%s)'100000000",
        "status":{"code":1}}]}]}]}'
ResponseMeaningWhat to do
{"partialSuccess":{}}AcceptedNothing — the service appears within a minute
401Key missing, malformed, revoked, or of the wrong kindCheck the header spelling and that the key is SDK-type and active
415Protobuf bodySet the protocol to http/json
400Body is not valid JSONCheck for shell quoting damage
503BackpressureRetry; if persistent, see Troubleshooting
SPA HTMLThe request never reached the APIThe appliance's reverse proxy is missing its /v1/ route
kind: 2 matters

kind: 2 is a SERVER span. Only SERVER and CONSUMER spans count as inbound requests, so a test span sent with the default kind (INTERNAL) is stored and appears in traces but contributes nothing to the service's rate, errors or duration. This is by design — see Services.

05 Step 4 — Confirm in the UI #

Within about a minute of the first SERVER span:

  • APM → Services lists the service with rate, errors, duration and apdex.
  • APM → Traces shows the trace in Live mode.
  • APM → Settings → Ingest keys shows a last used timestamp on the key. A key that still reads never used has never successfully authenticated — the problem is upstream of ZenPlus.

06 What good instrumentation looks like #

APM reads a handful of OpenTelemetry attributes to do its job. Emitting them well is the difference between a usable module and a pile of spans.

AttributeWhy ZenPlus needs itConsequence if missing
service.nameThe identity of every row on every screenEverything lands under unknown
deployment.environmentSeparates prod from staging in filters and SLO scopeFalls back to the key's environment, or unknown
service.versionVersion attribution on error issuesCannot tell which release introduced an error
http.routeThe templated route, e.g. /orders/Every distinct URL becomes its own operation and the list explodes
http.status_codeShown on spans and in usage analyticsStatus column reads 0
db.system, db.operationDatabase spans are styled and labelled distinctly in waterfallsDB calls look like any other span
exception span eventsError grouping, stack traces, issue detailFailures still counted, but with no type, message or stack
Cardinality: use templated routes

http.route must be the route template, not the concrete URL. /orders/8842 as a route means every order id becomes a separate operation, which bloats the rollups and makes the operations table useless. Every mainstream auto-instrumentation gets this right by default — the risk is hand-rolled spans.

07 Sending through the ZenPlus agent #

Hosts that already run the ZenPlus agent can let it forward APM telemetry, so the application does not need to hold an ingest key at all. Agent forwarder health — spans per minute, export errors, spool depth — is reported under APM → Settings → Data quality.

For hosts you provision automatically, use an enrollment token instead of embedding a long-lived key in an image: it expires, is use-capped, and can be revoked before it is ever redeemed. See Settings.