JWeBot/docs/telemetry.md

Runtime Telemetry And ACK Semantics

Status: local instrumentation contract for beta 1.0.x.

What ACK Means

ACK is the short acknowledgement sent immediately after Edge accepts a request. It is not the final answer and it does not mean the expensive work has completed.

The ordered path is:

1. OCR/responder submits one normalized event to Edge.
2. Edge durably records the event and queues the requested work.
3. Edge returns the short ACK (喵...... or the cloud acknowledgement).
4. The recorder sends that ACK through the guarded GUI path.
5. The recorder calls Edge with fast_ack_sent or fast_ack_failed.
6. Expensive work may run concurrently, but its final reply cannot be delivered until ACK delivery is confirmed.

This separates perceived response latency from completion latency while preserving message order. A failed ACK prevents the later final reply from appearing without its expected acknowledgement.

Privacy Boundary

Telemetry never records message bodies, sender names, raw group identifiers, reply text, secrets, environment values, or service payloads.

Event and group identifiers are represented only by 12-character SHA-256 prefixes. Plugin IDs, route names, queue states, operation names, counts, bounded error class names, and durations are allowed.

Every structured log uses schema version 1:

{
  "schema_version": 1,
  "name": "edge.ack.confirmed",
  "status": "ok",
  "component": "edge",
  "event_hash": "0123456789ab",
  "elapsed_ms": 83.417,
  "ack_kind": "chat",
  "delivered": true
}

Instrumented Stages

Stage	Events
Capture and OCR	ocr.warmup, ocr.capture, ocr.frame.wait, ocr.scan, ocr.pipeline
OCR queue	ocr.queue.wait, ocr.queue.process
Recorder to Edge	recorder.edge.request
ACK GUI and confirmation	recorder.ack.gui, recorder.ack.end_to_end, recorder.ack.confirm, edge.ack.prepared, edge.ack.confirmed
Fast ACK dedicated view	recorder.fast_ack.gui, recorder.fast_ack.end_to_end, recorder.fast_ack.confirm, edge.fast_ack.prepared, edge.fast_ack.confirmed
Edge routing	edge.route.classified, edge.local.complete, edge.alias.complete
Extension startup and execution	edge.extensions.startup, edge.extension.execute
Host process and IPC	host.process.spawn, host.runtime.load, host.request.received, host.service.request, host.service.wait, host.service.complete, host.invoke, host.process.complete
Kernel broker	edge.service.call
Model and cloud providers	edge.provider.complete
Durable queues	edge.queue.wait, edge.queue.claim, edge.queue.work
Final GUI delivery	edge.delivery.process, edge.delivery.complete, edge.delivery.end_to_end, delivery.gui.send
Guarded GUI send internals	gui.send.step, gui.bridge.step, gui.send.total

gui.send.step is emitted by the parent OCR process around safety boundaries: GUI lock acquisition, pre-draft screenshot, pre-draft group check, bridge process calls, post-draft screenshot, post-draft group check, callbacks, and total completion. gui.bridge.step is emitted from the child bridge timing metadata and covers VNC focus clicks, fixed settle waits, screenshot polling, draft confirmation, send-button confirmation, and Enter fallback.

The Kernel also retains process-local low-cardinality counts and average/maximum latency summaries. They are exposed through Edge health output for diagnosis and reset when the process restarts.

edge.route.classified records one bounded execution_path value: local-command, deepseek, or openclaw. OCR scan events record cache hits, misses, and periodic refresh without recording image or message content. Direct-request queue events use queue=direct_generation for parallel provider or Steam work and queue=direct_delivery for the single ordered GUI worker. Their bounded worker field distinguishes generation-0, generation-1, and delivery-0 without exposing event content.

Error Rules

• Record only the exception class, never exception text from plugin or user data boundaries.
• Distinguish controlled errors, retries, uncertain delivery, startup fallback, timeouts, and final failures.
• Do not replay an active Host invocation after a service call may have committed state.
• Keep existing durable queue state as the source of truth; telemetry is observational and must never control delivery.
• Telemetry observer and logger failures are swallowed so instrumentation cannot break the reply path.

Performance Analysis

The current points support separate measurement of:

• screen capture delay;
• OCR recognition cost and bubble count;
• stable-frame/debounce delay;
• recorder-to-Edge socket latency;
• ACK GUI latency and ACK confirmation latency;
• Host process startup and plugin load cost;
• service broker round trips;
• model/provider execution;
• queue waiting and retry time;
• final guarded GUI delivery.

Future optimization can then trade compute for latency deliberately, for example:

• narrower OCR regions or separate sender/body recognizers;
• more frequent capture with bounded coalescing;
• selective high-cost OCR only for likely incoming bubbles;
• warm or persistent Extension Hosts;
• concurrent ACK delivery and expensive computation;
• queue prioritization by route and age;
• preloaded models and cached layout/title fingerprints.

No such optimization should be accepted without comparing end-to-end ACK and final-delivery percentiles and checking duplicate/ordering regressions.

Run tools/performance-baseline.py for repository-only microbenchmarks or pass one or more private --telemetry logs to calculate stage distributions.