AppSignal for Ruby and OpenTelemetry

Collector mode only: this applies when AppSignal for Ruby runs in collector mode. It has no effect otherwise.

In collector mode, AppSignal builds its traces out of OpenTelemetry spans. A transaction becomes the root span of a trace. Every event you instrument becomes a child span inside it. Your own OpenTelemetry instrumentation therefore shares those traces. This page covers what that gives you, and how to describe a span in more detail than AppSignal does on its own.

Describing a span in OpenTelemetry’s terms

Attributes are key and value pairs on a span. The semantic conventions name the attributes to use for common work, such as a database query or an outgoing HTTP request. Use those names where they apply, so other tools can read what you record. Add attributes to the span AppSignal is currently recording with add_opentelemetry_attributes:

Appsignal.instrument("query.my_database") do
  Appsignal::Transaction.current.add_opentelemetry_attributes(
    "db.system.name" => "mysql",
    "db.namespace" => "orders"
  )
  run_the_query
end

The attributes are added to the innermost event that is open at the time, which is the query.my_database event in the example above. When no event is open, they are added to the transaction’s own span instead. A value must be a string, an integer, a float, or a boolean. Any other value is converted to a string.

Span kinds and instrumentation scopes

Span kind

A span’s kind says what role the work played in the trace. A span that served an incoming request is a :server span. A span that called out to another service is a :client span. Sending and receiving a message on a queue are :producer and :consumer spans. Work that stayed inside your application is an :internal span. An event’s span can be any of those five kinds. It is :internal unless you set one. The exception is Appsignal.instrument_sql, which records a :client span, because a query is an outgoing call to a datastore. A transaction’s span can be :server, :consumer, :producer, or :internal. It is :server unless you set one. A kind outside that list falls back to :server.

Appsignal.instrument("request.my_api", :opentelemetry_kind => :client) do
  call_the_api
end

Instrumentation scope

An instrumentation scope names the library a piece of instrumentation is written for, as a name and version pair. It is not always the library the work ran through. Instrumentation for a database adapter is scoped to the database library it talks to, not to the adapter. AppSignal breaks down the time events took and the allocations they made by instrumentation scope. The scope therefore decides which library a span’s cost is reported against. Spans are recorded under AppSignal’s default scope when you do not set one. Set a scope of your own when you instrument a library on its behalf, so that its cost is reported against that library.

Appsignal.instrument(
  "query.my_database",
  :opentelemetry_scope => ["my_database", "2.1.0"]
) do
  run_the_query
end

Which methods accept a kind or a scope

Pass opentelemetry_kind and opentelemetry_scope to any method that starts a transaction or records an event:

Appsignal.monitor
Appsignal.monitor_and_stop
Appsignal.send_error
Appsignal.report_error
Appsignal.instrument
Appsignal.instrument_sql

Set opentelemetry_kind to :internal on Appsignal.instrument_sql to override its :client default.

Continuing a trace from somewhere else

Work can reach your application from somewhere AppSignal does not instrument, carrying an OpenTelemetry context of its own. That context names the trace the work belongs to. Passing it lets both sides be read as one trace instead of two. Pass the context as opentelemetry_context. Say what the transaction’s span should do with it as opentelemetry_relationship:

:parent makes the transaction’s span a child of the incoming span. The work continues the same trace. This is what happens when you pass a context and say nothing else.
:link starts a new trace and records a link back to the incoming span. Use this when the work is its own unit rather than a continuation, which is usually the case for a background job.
:both parents the span and records the link.
:none ignores the incoming context.

context = OpenTelemetry.propagation.extract(message.headers)

Appsignal.monitor(
  :action => "MessageConsumer#perform",
  :namespace => "background_job",
  :opentelemetry_kind => :consumer,
  :opentelemetry_relationship => :link,
  :opentelemetry_context => context
) do
  handle(message)
end

Which methods accept a context and a relationship

Pass opentelemetry_context and opentelemetry_relationship to any method that starts a transaction:

Appsignal.monitor
Appsignal.monitor_and_stop
Appsignal.send_error
Appsignal.report_error

Appsignal.report_error is the exception. It starts a transaction only when none is open. Both arguments apply when it starts one, and are ignored when it adds the error to a transaction that is already open. You only need any of this for work AppSignal does not instrument. The libraries listed under distributed tracing are connected for you.

Using the OpenTelemetry SDK directly

You can create spans with the OpenTelemetry SDK inside code AppSignal is tracing. AppSignal makes its own span current while it records, so a span you create becomes a child of the AppSignal event you are inside.

tracer = OpenTelemetry.tracer_provider.tracer("my-app")

Appsignal.instrument("checkout") do
  # This span is recorded as a child of the "checkout" event.
  tracer.in_span("charge_card") do |span|
    span.set_attribute("payment.provider", "stripe")
    charge_the_card
  end
end

Use in_span rather than starting and finishing spans yourself. It makes the span current for the duration of the block, finishes it afterwards, and does both correctly when your code raises an error. Data you report through AppSignal is always recorded on a span AppSignal created, never on a span you created. An error reported to AppSignal inside your own span is recorded on the AppSignal event that surrounds it, or on the transaction’s span when no event is open. Your spans carry only what you record on them yourself.

Using third-party OpenTelemetry instrumentation

You can install instrumentation packages written by other people, such as the ones the OpenTelemetry project publishes for common libraries. Install and configure them as their own documentation describes. AppSignal needs no setup for them, and their spans appear in your traces alongside AppSignal’s own. This works because those packages record their work under whatever span is currently open, and AppSignal keeps its own span open while it records. So their spans land inside the request or job AppSignal is tracing, in the place you would expect. It does not work the other way around. AppSignal always starts a transaction as the beginning of a new trace, and does not look at what OpenTelemetry currently has open. If a package opens a span of its own and an AppSignal request or job begins inside it, that request does not join the package’s trace. You get two traces instead of one, with no connection between them. This matters when a package instruments the entry point into your application, because AppSignal instruments those entry points too. When a package instruments work that happens partway through a request AppSignal is already tracing, which is the usual case, the two fit together without you doing anything. When you do need to join a trace that AppSignal did not start, pass its context yourself. See continuing a trace from somewhere else.

SDKs & telemetry

Language SDKs

Telemetry tools

Describing a span in OpenTelemetry’s terms

Span kinds and instrumentation scopes

Span kind

Instrumentation scope

Which methods accept a kind or a scope

Continuing a trace from somewhere else

Which methods accept a context and a relationship

Using the OpenTelemetry SDK directly

Using third-party OpenTelemetry instrumentation

​Describing a span in OpenTelemetry’s terms

​Span kinds and instrumentation scopes

​Span kind

​Instrumentation scope

​Which methods accept a kind or a scope

​Continuing a trace from somewhere else

​Which methods accept a context and a relationship

​Using the OpenTelemetry SDK directly

​Using third-party OpenTelemetry instrumentation

Describing a span in OpenTelemetry’s terms

Span kinds and instrumentation scopes

Span kind

Instrumentation scope

Which methods accept a kind or a scope

Continuing a trace from somewhere else

Which methods accept a context and a relationship

Using the OpenTelemetry SDK directly

Using third-party OpenTelemetry instrumentation