Vidocq Documentation
Version

Humboldt runs on a simple principle: no runtime reflection on the hot path, no external bytecode library, anything computable at compile time is computed there. This page describes the internal pipeline (TracerProvider → Processor → Exporter), the ScopedValue vs ThreadLocal choice, the build-time wiring policy (and its one deliberate dynamic-proxy exception), and the deliberate absence of a gRPC stack.

Signal pipeline

Diagram

Each pillar is independent: one can enable traces=otlp, metrics=none, logs=in-memory without interference. Shared code (the OTLP/JSON encoders, retry, virtual-thread executor) lives in humboldt-exporter-otlp-http; the encoders sit in the non-exported io.vidocq.humboldt.exporter.otlp.http.internal package.

BatchSpanProcessor — virtual-thread worker

BatchSpanProcessor is the most delicate SDK component. Internal architecture:

  • Bounded queue (ArrayBlockingQueue<SpanData>). Default capacity: 2048. When the queue fills, spans are dropped and a System.Logger WARNING is emitted.

  • Dedicated worker — a single virtual thread (Thread.ofVirtual().name("humboldt-batch-span-processor")) looping:

    1. drainTo(batch, maxExportBatchSize)

    2. If batch.size() >= maxExportBatchSize OR delay elapsed OR flush requested → export.

    3. Calls exporter.export(batch) which returns a CompletableResultCode.

  • scheduleDelay — default 5 s. Guarantees an isolated span is flushed even without saturation.

  • Shutdown — final drain, awaits in-flight exports, propagates CompletableResultCode.

Picking a virtual thread over a platform thread follows directly from Humboldt’s I/O model: the HTTP exporter blocks on the network, and with a VT this blocking does not consume a carrier thread. No pool, no platform task queue.

BatchLogRecordProcessor — same pattern

BatchLogRecordProcessor shares the pattern: VT worker (named humboldt-batch-log-processor), bounded queue, threshold + scheduleDelay + flush + shutdown. The scheduleDelay (1 s by default, shorter than traces since logs are more frequent and critical during incidents) and the dedicated queue are the only differences.

PeriodicMetricReader

Metrics follow a different model: no event queue but periodic collection. The reader calls meterProvider.collectAllMetrics() every OTEL_METRIC_EXPORT_INTERVAL ms (default 60 s for otlp, 60 min for in-memory), then hands the Collection<MetricData> to the exporter.

Context storage: ThreadLocal vs ScopedValue

OpenTelemetry delegates the current Context storage to a ContextStorageProvider SPI. Humboldt ships its own: HumboldtContextStorageProvider.

MVP (M1, shipped) Post-MVP (M8 ADR)

Implementation

ThreadLocal<Context>

ScopedValue<Context> (JEP 506)

Virtual-thread pinning

None on pure-Java code (JEP 444)

None, with structural sharing

Memory cost per 100 K VTs

1 ThreadLocal entry per VT

1 global binding, O(1) per VT

Limitation

Imperative attach()/detach() mutations

Immutability, explicit scope (runWhere(…​))

The ThreadLocal MVP is OTel-spec-compliant and logs a WARNING on any out-of-order attach()/detach() — leak protection.

Build-time wiring — and one deliberate proxy exception

Humboldt follows the Vidocq philosophy: no bytecode agent, no third-party library like ASM or Byte Buddy, and no code generation of its own — Humboldt ships neither an annotation processor nor Class-File API (JEP 484) generators. What runs at build time is Vauban’s annotation processor (vauban-processor), which indexes the CDI beans of humboldt-cdi and humboldt-rest at compile time. The @WithSpan interceptor (WithSpanInterceptor) is a plain hand-written class registered through HumboldtBuildCompatibleExtension — step-debuggable, profileable like any other code.

There is one deliberate exception to the no-dynamic-proxy rule: HumboldtTelemetryProducers (in humboldt-cdi) builds the injected current-Span and current-Baggage beans with java.lang.reflect.Proxy.newProxyInstance. The MP Telemetry TCK requires an injected Span/Baggage to keep tracking the current context after injection, and a delegating proxy is the simplest spec-compliant way to do that. Everything else resolves statically.

Bootstrap sequence

  1. Java Modules startup — ServiceLoader loads HumboldtContextStorageProvider via provides …​ with (and META-INF/services fallback on classpath).

  2. HumboldtAutoConfigure.configure() is called by the application main().

  3. EnvConfig.system() reads System.getenv() + System.getProperties() (env vars win). If OTEL_SDK_DISABLED is true — the MP Telemetry 2.2 default — configuration stops here: providers are built with no processor or reader, so the OTel API stays fully usable but nothing is ever exported.

  4. Builds the Resource from OTEL_SERVICE_NAME + OTEL_RESOURCE_ATTRIBUTES.

  5. Based on OTEL_TRACES_EXPORTER — instantiates the exporter (Otlp, InMemory, Logging, or no-op).

  6. Builds the Sampler from OTEL_TRACES_SAMPLER + _ARG.

  7. Builds the SdkTracerProvider (Resource + Sampler + IdGenerator + Clock + appropriate processor).

  8. Same for SdkMeterProvider + PeriodicMetricReader + SdkLoggerProvider + BatchLogRecordProcessor.

  9. Composes an AutoConfiguredHumboldt implements OpenTelemetry, AutoCloseable.

  10. GlobalOpenTelemetry.set(sdk) in the application code. From there, every Tracer/Meter/Logger resolved through GlobalOpenTelemetry.getXxx() uses this SDK.

  11. On shutdown: sdk.close() → batch processors flush → exporter.shutdown() → httpClient released.

AOT compatibility

No Class.forName, Method.invoke, Class.getDeclaredFields() on the hot path. The only dynamic proxies are the two injected current-Span / current-Baggage CDI beans described above (a fixed, statically-known pair of interfaces). The remaining introspection goes through the ServiceLoader SPI, natively handled by GraalVM native-image and Leyden CDS through the standard JVM reachability metadata. Since the OTLP payload is plain JSON produced by an internal encoder, there is no protobuf runtime and no extra reflect-config.json to maintain.

Deliberate absence of gRPC

Humboldt only transports HTTP/1.1 and HTTP/2 (via java.net.http.HttpClient). No grpc-java, no netty-codec-http2. Consequence: no OTLP/gRPC transport in v1.

The future chappe-grpc module (cf. PLAN.md §3.5) will implement gRPC natively on top of the existing Chappe transport. At that point a humboldt-exporter-otlp-grpc module will ship, without grpc-java or Netty.

Internal metrology

There are no self-metrics yet: a full queue (dropped span or log record) and a failed or retried export are reported through System.Logger WARNINGs, not through dedicated humboldt.* instruments. Self-observability counters are a candidate for a later milestone.

Next: TCK.