Search docs
Jump to a page or section

Schema Management

How Telemetry Machine stores fields and evolves schemas for OpenTelemetry data.

Open in Claude

Telemetry Machine gives each traces, logs, or metrics dataset a schema for that signal type. You do not need to submit a schema before sending OTLP data.

This page describes OpenTelemetry datasets. Custom event datasets use a different ingestion and schema path.

How schemas evolve

Each signal type starts with fixed columns. Telemetry Machine then adds columns for resource and record attributes as it sees them.

For normal OTLP ingestion, schema growth is additive:

  1. A new attribute name creates a new nullable column.
  2. Rows that contain the attribute store its value.
  3. Rows that do not contain the attribute return null for that column.
  4. Existing rows are not rewritten when a new column appears.

For example:

Text
Day 1: service.name, http.request.method
Day 2: service.name, http.request.method, user.id

Day 1 rows have null for user.id.

Fixed columns

Fixed columns depend on the dataset's signal type. The schema reports native query types such as Utf8, Int64, Double, Boolean, and Timestamp(ns, UTC).

Common trace columns include:

ColumnQuery typeMeaning
_timeTimestamp(ns, UTC)Span start time used for time filtering
trace.trace_idUtf8Trace ID
trace.span_idUtf8Span ID
trace.parent_span_idUtf8Parent span ID
nameUtf8Span name
kindInt64OpenTelemetry span-kind enum, from 0 to 5
duration_nsInt64Span duration in nanoseconds
status_codeInt640=UNSET, 1=OK, 2=ERROR
resource.service.nameUtf8Service name

Logs have fixed columns such as log.body, log.severity_number, and log.severity_text. Metrics have fixed columns such as metric.name, metric.type, value_int, and value_double.

Attribute columns

Telemetry Machine flattens OpenTelemetry attributes into columns:

PrefixSource
attr.*Span, log-record, or metric-data-point attributes
resource.*Resource attributes

For example, an OpenTelemetry span attribute named http.request.method becomes attr.http.request.method.

Dynamic attr.* and resource.* values are stored as strings. This applies even when the OTLP AnyValue contains an integer, double, or boolean.

OTLP AnyValue fieldStored text
string_value: "GET"GET
int_value: 200200
double_value: 3.143.14
bool_value: truetrue
array_valueJSON array
kvlist_valueJSON object
bytes_valueBase64 text

The original scalar type is therefore not preserved for a dynamic attribute.

Attribute names

Use current OpenTelemetry semantic conventions where possible. Current HTTP span attributes include:

Text
attr.http.request.method
attr.url.path
attr.http.response.status_code

Resource attributes describe the observed service, process, container, host, or other entity. They should remain stable for the lifetime of that resource.

Text
resource.service.name
resource.deployment.environment.name
resource.cloud.region

Put values that change for each operation, such as request IDs and user IDs, on the span or log record instead of the resource.

Telemetry Machine stores attribute names as they arrive. It does not translate old and new semantic-convention names into aliases. For example, attr.http.status_code and attr.http.response.status_code are separate columns.

FAQ

Can I change a field's type?

Fixed column types come from the signal schema. Dynamic OpenTelemetry attributes are stored as strings. There is no field-type editor in the public product.

Can I rename or delete a field?

There is no per-field rename or delete control in the public product. Send the new attribute name from your instrumentation. The old column remains in the dataset and returns null for rows that no longer send it.

What happens to old data when I add a field?

Old rows return null for the new field. New rows contain the value when the producer sends it.

Can I send arrays or nested objects in attributes?

Yes. OTLP arrays and key-value lists are serialized as JSON strings. Byte arrays are stored as Base64 text.

Next steps