Cost
Storing full prompts and replies takes space. OpenTelemetry says recording them can be a problem because of storage cost and privacy rules. Start with the step names and the error. Turn on prompt text only where you are allowed to keep it.
Map · Ship & Maintain · 02 · Observability
Concept card · Ship & Maintain
Observability
Observability is a trace of the prompt, the retrieval, the tool calls, and the reply, kept so a failure can be inspected after it happens.
Logs and traces of prompts, tool calls, and replies so a failure can be inspected.
What changed
Oct 18: First card. The picture opens a trace on one wrong reply. Sources are the OpenTelemetry GenAI span conventions and LangSmith's observability docs.
1 What it is
A trace is the record of one run. OpenTelemetry's generative AI span conventions say a span should cover a logical operation from the moment it starts until the response is fully received or the operation stops on an error. Those conventions name separate spans for inference, retrieval, and executing a tool. The same document says instructions, inputs, and outputs are sensitive, and that instrumentations should not capture them by default. You opt in, or you store the text somewhere with its own access control and put a reference on the span. LangSmith's observability docs call traces the record of what agents did in production, and say to use them to debug failures. The OpenTelemetry conventions are marked Development. The picture on this card is an illustration of one made-up run.
2 Why it matters
Quality
Without the trace, a wrong close time looks like a model mistake. With the trace, you can see that the reply followed the tool result.
Risk
A trace that stores customer text where the whole team can read it is a privacy problem. OpenTelemetry says not to capture inputs and outputs by default, and to use separate access control when you do store them.
3 How to apply · try today
On the next failed request, write four lines: the user question, what retrieval returned, what each tool returned, and the reply. Leave out names, emails, and account numbers.
Common mistake
Logging only the final reply. The failure is often in the retrieved text or the tool result, and those are gone if you did not keep them.
Self-check
Why keep a trace instead of only the final reply?
Coming soon. One failed request, written as question, retrieval, tool result, and reply, with identifiers removed.
Coming soon. How the OpenTelemetry conventions separate inference, retrieval, and tool spans, and why prompt text is off by default.
Questions
- What is observability?
- Observability is a trace of the prompt, the retrieval, the tool calls, and the reply, kept so a failure can be inspected after it happens.
- Should a trace store the full prompt by default?
- OpenTelemetry's GenAI span conventions say instrumentations should not capture instructions, inputs, and outputs by default. Opt in, or store the text with its own access control.
- Is a trace the same as the reply?
- The reply is one step. The trace also keeps retrieval and tool results, which is where a wrong answer often starts.