# Agno
Source: https://docs.agnost.ai/agno
Capture traces from Agno agents and teams
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this Agno application.
Org ID: your-org-id
Instrument the real agent.run() path and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash theme={null}
pip install agnost
```
```python theme={null}
import agnost
agnost.init("your-org-id")
interaction = agnost.begin(user_id="u-42", agent_name="agno-support", input=prompt)
try:
response = agent.run(prompt)
interaction.end(output=response.content)
except Exception as exc:
interaction.end(output=str(exc), success=False)
raise
finally:
agnost.shutdown()
```
## Manual setup: OpenTelemetry
### Install
```bash theme={null}
pip install agno openinference-instrumentation-agno \
opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```
### Setup
```python theme={null}
import os
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from openinference.instrumentation.agno import AgnoInstrumentor
from openinference.instrumentation import using_attributes
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://otel.agnost.ai"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "X-Agnost-Org-ID="
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
AgnoInstrumentor().instrument()
agent = Agent(
model=OpenAIChat(id="gpt-4o-mini"),
session_id="sess-123", # Agno's first-class session
user_id="u-42",
)
with using_attributes(session_id="sess-123", user_id="u-42"):
agent.print_response("Hello")
```
Agno itself supports `session_id` and `user_id` as `Agent` constructor params. Spans carry OpenInference attributes plus `agno.agent.name` / `agno.team.name`. Instrument once at process start; child agents in teams inherit.
### Alternative: OpenLit
`openlit.init(...)` is also supported and auto-instruments common providers in one shot.
### Verify
Run one `agent.print_response` call, then open **Events** in Agnost AI. Confirm spans include `session_id`, `user_id`, and `agno.agent.name`.
### Troubleshooting
* Instrument once at process start before agent calls.
* Confirm `OTEL_EXPORTER_OTLP_HEADERS` contains `X-Agnost-Org-ID=`.
* If child team spans are missing, confirm the parent Agno agent is instrumented before teams are constructed.
### References
* [Enable OpenTelemetry export](https://docs.agno.com/observability/openlit)
* [Add custom metadata](https://docs.agno.com/tracing/overview)
## Next steps
* [Conversations](/using-conversations): review the complete Agno interaction.
* [Events](/using-events): inspect the spans Agnost AI received.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Agnost AI in Claude and Cursor
Source: https://docs.agnost.ai/agnost-mcp-server
Query your Agnost AI dashboard from Claude Desktop, Cursor, and any MCP client
Agnost AI runs a hosted, OAuth-protected MCP server at **`https://mcp.agnost.ai/mcp`**. Add it to any MCP-aware client and ask natural-language questions about conversations, events, tool calls, intents, and violations.
## Connect
### Claude Desktop / Cursor
```json theme={null}
{
"mcpServers": {
"agnost-ai": {
"url": "https://mcp.agnost.ai/mcp"
}
}
}
```
No API keys, no copy-pasted tokens. The first time you call a tool, the client opens a browser tab, you sign in with Google through the Agnost AI dashboard, and the tab closes.
### Other MCP clients
Anything that speaks streamable-HTTP MCP and supports OAuth 2.1 dynamic client registration works the same way. The server publishes its OAuth metadata at:
```
https://mcp.agnost.ai/.well-known/oauth-authorization-server
https://mcp.agnost.ai/.well-known/oauth-protected-resource
```
Both discovery documents advertise the supported `agnost` OAuth scope so clients can request the server's current access level without guessing. Organization membership and permissions are still derived from the signed-in Agnost AI account.
## What you can ask
* "Which tools are failing most often this week?"
* "Show me recent conversations where users had to repeat their request."
* "Show me today's violations."
* "Summarize the top intents in this org."
## Verify
After adding the server to your MCP client, call any Agnost AI tool. The client should open a browser for OAuth the first time, then return dashboard data after login.
## Troubleshooting
* Confirm the MCP client supports streamable-HTTP MCP and OAuth 2.1 dynamic client registration.
* If OAuth does not open, remove the server from the client config and add it again.
* If a query returns no data, confirm you are logged into the right Agnost AI organization in the dashboard.
## Next steps
* [Conversations](/using-conversations): understand the interaction data you can query.
* [Intents](/using-intents): learn how user goals are represented.
* [Violations](/using-violations): learn how expected behavior is evaluated.
# Anthropic
Source: https://docs.agnost.ai/anthropic
Track Anthropic SDK calls from Python or TypeScript
## Language support
| Setup option | Python | TypeScript |
| --------------- | :-------: | :--------: |
| Agnost AI skill | ✓ | ✓ |
| Agnost AI SDK | ✓ | ✓ |
| OpenTelemetry | Automatic | Manual |
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this Anthropic application.
Org ID: your-org-id
Instrument the real messages.create() path and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash Python theme={null}
pip install agnost
```
```bash TypeScript theme={null}
npm install agnostai
```
```python Python theme={null}
import agnost
from anthropic import Anthropic
agnost.init("your-org-id")
client = Anthropic()
interaction = agnost.begin(user_id="u-42", agent_name="anthropic-support", input=prompt)
try:
response = client.messages.create(model="claude-sonnet-4-5", max_tokens=512,
messages=[{"role": "user", "content": prompt}])
interaction.end(output="".join(block.text for block in response.content if block.type == "text"))
except Exception as exc:
interaction.end(output=str(exc), success=False)
raise
finally:
agnost.shutdown()
```
```typescript TypeScript theme={null}
import Anthropic from '@anthropic-ai/sdk';
import * as agnost from 'agnostai';
agnost.init('your-org-id');
const client = new Anthropic();
const interaction = agnost.begin({
userId: 'u-42',
agentName: 'anthropic-support',
input: prompt,
});
try {
const response = await client.messages.create({
model: 'claude-sonnet-4-5',
max_tokens: 512,
messages: [{ role: 'user', content: prompt }],
});
const output = response.content
.filter((block) => block.type === 'text')
.map((block) => block.text)
.join('');
interaction.end(output);
} catch (error) {
interaction.end(String(error), false);
throw error;
} finally {
await agnost.shutdown();
}
```
## Manual setup: OpenTelemetry
### Python: automatic instrumentation
### Install
```bash theme={null}
pip install anthropic openinference-instrumentation-anthropic \
opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```
### Setup
```python theme={null}
import os
from anthropic import Anthropic
from openinference.instrumentation.anthropic import AnthropicInstrumentor
from openinference.instrumentation import using_attributes
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://otel.agnost.ai"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "X-Agnost-Org-ID="
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
AnthropicInstrumentor().instrument(tracer_provider=provider)
client = Anthropic()
with using_attributes(session_id="sess-123", user_id="u-42"):
client.messages.create(
model="claude-sonnet-4-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Hi"}],
)
```
Tool-use blocks are unpacked. You get `llm.input_messages.*`, `llm.output_messages.*`, `tool.name`, `tool.parameters`, and `llm.token_count.{prompt,completion}`.
### Alternative
The Traceloop alternative (`opentelemetry-instrumentation-anthropic`) emits `gen_ai.*` and `traceloop.*`: also recognized by Agnost AI.
### Verify
Run one `client.messages.create` call, then open **Events** in Agnost AI. Confirm message and tool-use attributes appear as expected.
### Troubleshooting
* Confirm `AnthropicInstrumentor().instrument(...)` runs before Anthropic calls.
* Confirm `OTEL_EXPORTER_OTLP_HEADERS` contains `X-Agnost-Org-ID=`.
* Confirm `using_attributes` wraps the call you want grouped.
### References
* [Enable OpenTelemetry export](https://github.com/Arize-ai/openinference/tree/main/python/instrumentation/openinference-instrumentation-anthropic)
* [Add custom metadata](https://arize.com/docs/phoenix/tracing/how-to-tracing/add-metadata/customize-spans)
### TypeScript: manual instrumentation
Install the OpenTelemetry packages alongside your existing Anthropic SDK:
```bash theme={null}
npm install @opentelemetry/api @opentelemetry/sdk-node \
@opentelemetry/exporter-trace-otlp-proto
```
Start OpenTelemetry before creating the Anthropic client, then wrap the call you
want to track:
```typescript theme={null}
import Anthropic from '@anthropic-ai/sdk';
import { SpanStatusCode, trace } from '@opentelemetry/api';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
const sdk = new NodeSDK({
traceExporter: new OTLPTraceExporter({
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
}),
});
await sdk.start();
const client = new Anthropic();
const tracer = trace.getTracer('anthropic-app');
const response = await tracer.startActiveSpan('anthropic.messages.create', async (span) => {
span.setAttribute('gen_ai.operation.name', 'chat');
span.setAttribute('gen_ai.provider.name', 'anthropic');
span.setAttribute('gen_ai.request.model', 'claude-sonnet-4-5');
span.setAttribute('gen_ai.conversation.id', 'conversation-123');
span.setAttribute('user.id', 'user-42');
span.setAttribute('input.value', prompt);
try {
const message = await client.messages.create({
model: 'claude-sonnet-4-5',
max_tokens: 512,
messages: [{ role: 'user', content: prompt }],
});
const output = message.content
.filter((block) => block.type === 'text')
.map((block) => block.text)
.join('');
span.setAttribute('output.value', output);
return message;
} catch (error) {
span.recordException(error as Error);
span.setStatus({ code: SpanStatusCode.ERROR });
throw error;
} finally {
span.end();
}
});
await sdk.shutdown();
```
Run one message call, then open **Events** in Agnost AI and confirm the input,
output, model, user, and conversation fields are present.
## Next steps
* [Conversations](/using-conversations): review the complete Anthropic interaction.
* [Events](/using-events): inspect the SDK or OpenTelemetry records.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Python Anthropic SDK (low-level)
Source: https://docs.agnost.ai/anthropic-low-level
Add Agnost AI analytics to your low-level Anthropic MCP server
## Install
```bash pip theme={null}
pip install agnost-mcp
```
```bash uv theme={null}
uv add agnost-mcp
```
## Integrate
Call `track` after defining your server and handlers, before running:
```python theme={null}
from mcp.server import Server
from agnost_mcp import track, config
server = Server("your-server-name")
# ... your list_tools and call_tool handlers ...
track(server, "your-org-id", config(
endpoint="https://api.agnost.ai",
disable_input=False,
disable_output=False,
))
```
Get your org ID from [app.agnost.ai](https://app.agnost.ai).
## Identify users
Pass an `identify` function to resolve a user from the incoming request context. It receives the raw request object and the process environment, and should return a dict with at least a `userId` key:
```python theme={null}
from agnost_mcp import track, config
track(server, "your-org-id", config(
identify=lambda req, env: {
"userId": req.get("headers", {}).get("x-user-id") or env.get("USER_ID", "anonymous"),
"email": req.get("headers", {}).get("x-user-email") or env.get("USER_EMAIL"),
"role": req.get("headers", {}).get("x-user-role") or env.get("USER_ROLE", "user"),
}
))
```
The function can also be `async`. Return `None` to skip user attribution for a request.
## Options
| Field | Type | Default | Description |
| ---------------- | ---------- | ----------------------- | ----------------------------------------------------------------------------- |
| `endpoint` | `str` | `https://api.agnost.ai` | API endpoint |
| `disable_input` | `bool` | `False` | Skip capturing tool input arguments |
| `disable_output` | `bool` | `False` | Skip capturing tool output / result |
| `log_level` | `str` | `"INFO"` | Log verbosity: `DEBUG`, `INFO`, `WARNING`, `ERROR` |
| `identify` | `Callable` | `None` | Function `(request, env) → UserIdentity` to resolve user identity per request |
`UserIdentity` is a `Dict[str, Any]` that **must contain a `userId` key**. Other fields are optional and forwarded as user traits.
## Checkpoints
Per-step latency checkpoints are currently available only in the [TypeScript SDK](/typescript-sdk#checkpoints). Tool-level latency is captured automatically across all SDKs.
## What appears in Agnost AI
* **Tool calls** for MCP tool invocations.
* **Events** for every tracked call.
* Failed tool calls remain visible in **Events** with their error details.
## Verify
Call one MCP tool from your Anthropic client, then open [app.agnost.ai](https://app.agnost.ai). Check **Events** first, then **Tool calls**.
## Troubleshooting
* Call `track(...)` after registering handlers and before serving.
* Confirm the org ID is correct.
## Next steps
* [Tool calls](/using-tool-calls): analyze calls from your low-level MCP server.
* [Events](/using-events): verify call fields and execution details.
* [Alerts](/using-alerts): monitor important MCP tool conditions.
# Apify Actors
Source: https://docs.agnost.ai/apify
Add Agnost AI analytics to your Apify Actors
## What are Apify Actors?
[Apify Actors](https://docs.apify.com/platform/actors) are serverless cloud programs that can perform anything from a simple action, like filling out a web form, to a complex operation, like crawling an entire website. They are programs packaged as Docker images, which accept a well-defined JSON input, perform an action, and optionally produce a well-defined JSON output.
### Key Features
* **Serverless Execution**: No infrastructure management required, just deploy and run
* **Stateful Operations**: Actors can maintain state across executions, enabling runs from seconds to hours, or even indefinitely
* **Flexible I/O**: Accept JSON input and produce JSON output with well-defined schemas
* **Built-in Storage**: Access to datasets, key-value stores, and request queues
* **Composable**: Actors can call and interact with each other to build complex systems
### Actor Structure
Each Apify Actor consists of:
1. **Dockerfile**: Manages code location, build process, and execution instructions
2. **Source Code**: Your application logic (Python, Node.js, or other languages)
3. **Input/Output Schemas**: Define required inputs and produced results
4. **README Documentation**: Explains functionality for users
5. **Metadata**: Actor name, description, author, and version
Actors can be run via the web console, API, CLI, or scheduled triggers. They can be private for personal use or published to the [Apify Store](https://apify.com/store) for monetization.
Learn more: [Apify Actors Development](https://docs.apify.com/platform/actors/development)
## Getting Started
Add Agnost AI analytics to your Apify Actor in four simple steps:
### 1. Get Your Organization ID
Get your organization ID from [app.agnost.ai](https://app.agnost.ai)
### 2. Install Agnost AI
```bash theme={null}
pip install agnost-mcp
```
```bash theme={null}
npm install agnost
```
### 3. Add One Line of Code
```python theme={null}
from apify import Actor
from agnost_mcp import track
from mcp.server import Server
async def main():
async with Actor:
# Create MCP server
server = Server("your-mcp-server")
# Enable analytics
track(server, "your-org-id")
# Your actor logic here
actor_input = await Actor.get_input() or {}
# ... rest of your code
```
```javascript theme={null}
import { Actor } from 'apify';
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { trackMCP } from 'agnost';
await Actor.main(async () => {
// Create MCP server
const server = new Server({ name: 'your-mcp-server', version: '1.0.0' },
{ capabilities: { tools: {} } });
// Enable analytics
trackMCP(server, "your-org-id");
// Your actor logic here
const input = await Actor.getInput();
// ... rest of your code
});
```
That's it! Your Apify Actor is now tracked with default settings.
### 4. View Metrics
Visit [app.agnost.ai](https://app.agnost.ai) to view your analytics:
* Monitor Actor runs in real-time
* Track performance and latency
* Analyze success and failure rates
* Identify optimization opportunities
## Configuration
Customize tracking behavior with configuration options:
```python theme={null}
from agnost_mcp import track, config
from mcp.server import Server
server = Server("your-mcp-server")
track(server, "your-org-id", config(
endpoint="https://api.agnost.ai",
disable_input=False,
disable_output=False
))
```
```javascript theme={null}
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { trackMCP, createConfig } from 'agnost';
const server = new Server({ name: 'your-mcp-server', version: '1.0.0' },
{ capabilities: { tools: {} } });
trackMCP(server, "your-org-id", createConfig({
endpoint: "https://api.agnost.ai",
disableInput: false,
disableOutput: false
}));
```
### Configuration Options
| Option | Type | Default | Description |
| ---------------------------------- | -------- | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
| `endpoint` | string | `"https://api.agnost.ai"` | API endpoint URL |
| `disable_input` / `disableInput` | boolean | `false` | Disable tracking of input parameters |
| `disable_output` / `disableOutput` | boolean | `false` | Disable tracking of output responses |
| `log_level` | string | `"INFO"` | Python only. Log verbosity: `DEBUG`, `INFO`, `WARNING`, `ERROR` |
| `identify` | function | `None` / `undefined` | Function `(request, env) → UserIdentity` to identify users from request context. See [user identification](#user-identification). |
`UserIdentity` is a `dict` (Python) or object (Node) that **must contain a
`userId` key**. Other fields such as `email` and `role` are optional and forwarded
as user traits.
## User Identification
Track analytics per user to understand usage patterns across different customers or organizations running your Actor.
```python theme={null}
from agnost_mcp import track, config
from mcp.server import Server
async def main():
async with Actor:
server = Server("your-mcp-server")
# With user identification
track(server, "your-org-id", config(
identify=lambda req, env: {
"userId": env.get("APIFY_USER_ID") or "anonymous",
"actorId": env.get("APIFY_ACT_ID"),
"runId": env.get("APIFY_ACT_RUN_ID"),
}
))
# Your actor logic here
```
Using Actor input for identification:
```python theme={null}
from agnost_mcp import track, config
from mcp.server import Server
async def main():
async with Actor:
actor_input = await Actor.get_input() or {}
server = Server("your-mcp-server")
track(server, "your-org-id", config(
identify=lambda req, env: {
"userId": actor_input.get("customer_id", "anonymous"),
"email": actor_input.get("customer_email"),
"organization": actor_input.get("organization_name"),
}
))
# Your actor logic here
```
```javascript theme={null}
import { Actor } from 'apify';
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { trackMCP } from 'agnost';
await Actor.main(async () => {
const server = new Server({ name: 'your-mcp-server', version: '1.0.0' },
{ capabilities: { tools: {} } });
// With user identification
trackMCP(server, "your-org-id", {
identify: (req, env) => ({
userId: env.APIFY_USER_ID || 'anonymous',
actorId: env.APIFY_ACT_ID,
runId: env.APIFY_ACT_RUN_ID
})
});
// Your actor logic here
});
```
Using Actor input for identification:
```javascript theme={null}
import { Actor } from 'apify';
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { trackMCP } from 'agnost';
await Actor.main(async () => {
const input = await Actor.getInput();
const server = new Server({ name: 'your-mcp-server', version: '1.0.0' },
{ capabilities: { tools: {} } });
trackMCP(server, "your-org-id", {
identify: (req, env) => ({
userId: input.customer_id || 'anonymous',
email: input.customer_email,
organization: input.organization_name
})
});
// Your actor logic here
});
```
### Common Use Cases
* **Multi-tenant Actors**: Track which customer or organization is using the Actor
* **Subscription Tiers**: Segment analytics by free, pro, or enterprise users
* **Cost Attribution**: Understand usage and costs per customer
* **Customer Success**: Monitor individual customer health and engagement
See the [Python setup guide](/python-conversation) for advanced identification patterns.
## Complete Example
```python theme={null}
from apify import Actor
from agnost_mcp import track, config
from mcp.server import Server
async def main():
async with Actor:
actor_input = await Actor.get_input() or {}
# Create MCP server
server = Server("your-mcp-server")
# Enable analytics with user identification
track(server, "your-org-id", config(
identify=lambda req, env: {
"userId": actor_input.get("customer_id", "anonymous"),
"organization": actor_input.get("org_name"),
"subscriptionTier": actor_input.get("tier", "free"),
}
))
# Your actor logic
url = actor_input.get('url')
Actor.log.info(f'Scraping URL: {url}')
# Scraping logic
data = [] # ... scrape data
# Save results
await Actor.push_data(data)
Actor.log.info(f'Successfully scraped {len(data)} items')
```
```javascript theme={null}
import { Actor } from 'apify';
import { Server } from '@modelcontextprotocol/sdk/server/index.js';
import { trackMCP } from 'agnost';
await Actor.main(async () => {
const input = await Actor.getInput();
// Create MCP server
const server = new Server({ name: 'your-mcp-server', version: '1.0.0' },
{ capabilities: { tools: {} } });
// Enable analytics with user identification
trackMCP(server, "your-org-id", {
identify: (req, env) => ({
userId: input.customer_id || 'anonymous',
organization: input.org_name,
subscriptionTier: input.tier || 'free'
})
});
// Your actor logic
const { url } = input;
console.log(`Scraping URL: ${url}`);
// Scraping logic
const data = []; // ... scrape data
// Save results
await Actor.pushData(data);
console.log(`Successfully scraped ${data.length} items`);
});
```
## Why Add Analytics to Your Actor?
### Performance Optimization
* Identify slow operations and bottlenecks
* Track execution time trends over versions
* Optimize resource usage and costs
### Usage Insights
* Monitor how customers use your Actor
* Understand which features are most popular
* Identify usage patterns and trends
### Error Tracking
* Track failure rates and error patterns
* Debug issues faster with detailed context
* Monitor service reliability
### Business Intelligence
* Segment users by subscription tier
* Track adoption and retention
* Make data-driven decisions for features and pricing
## Verify
Run one Actor invocation, then open [app.agnost.ai](https://app.agnost.ai). Check **Events** first, then **Tool calls** for MCP tool calls emitted by the Actor.
## Troubleshooting
* Confirm `track(...)` or `trackMCP(...)` runs before Actor tool calls.
* Confirm the org ID is correct.
* Confirm Actor input fields used in `identify` are present.
## Resources
* [Python setup](/python-conversation): SDK configuration and identification
* [Apify Actors Documentation](https://docs.apify.com/platform/actors): Learn about Apify Actors
* [Apify Actors Development](https://docs.apify.com/platform/actors/development): Development guide
## Need Help?
* Email: [founders@agnost.ai](mailto:founders@agnost.ai)
* [Book a call](https://call.agnost.ai/)
## Next steps
* [Tool calls](/using-tool-calls): analyze Actor tool usage and outcomes.
* [Events](/using-events): inspect the records emitted by an Actor run.
* [Alerts](/using-alerts): monitor important Actor conditions.
# Authentication
Source: https://docs.agnost.ai/authentication
How Agnost AI authenticates ingestion, dashboard, and API requests
Agnost AI uses different authentication models for ingestion and dashboard/API access.
## SDK ingestion
SDK ingestion endpoints under `/api/v1/*` use your organization ID in the
`x-org-id` header. SDK configuration and environment variables may format it as
`org_id`, `orgId`, or `AGNOST_ORG_ID`; all refer to the same organization ID.
The org ID is a public routing identifier. It does not grant dashboard read access by itself.
## Dashboard and API access
Dashboard API endpoints use one of these:
| Method | Headers | Use case |
| ------- | -------------------------------------------- | ------------------------------------ |
| JWT | `Authorization: Bearer ` and `x-org-id` | Browser/dashboard access after login |
| API key | `x-api-key` and optional `x-org-id` | Programmatic dashboard/API access |
API keys use the `agnost_<64-hex>` format and are issued from **Settings -> API Keys**.
## Common mistakes
* Sending `x-org-id` without a JWT or API key to dashboard endpoints.
* Using the wrong org ID when a user belongs to multiple organizations.
* Forgetting the `X-Agnost-Org-ID` header on OpenTelemetry exporters.
* Treating the org ID as a secret. Rotate API keys if they leak; org IDs are routing identifiers.
## Related pages
* [Quickstart](/quickstart)
* [Capture Session](/capture-session)
* [Capture Event](/capture-event)
## Next steps
* [Packages and versions](/sdks): choose the package that matches your application.
* [Capture a session](/capture-session): create a conversation boundary through the API.
* [Capture an event](/capture-event): send activity into an existing session.
# Capture Event
Source: https://docs.agnost.ai/capture-event
POST /api/v1/capture-event
Record one turn-pair or tool call within a session
Generate `event_id` client-side (UUID) so child events can reference their parent before the parent's response returns.
## Body
| Field | Type | Required | Description |
| ---------------- | --------------- | -------- | ----------------------------------------------------------------------------------------------------- |
| `event_id` | `string` (UUID) | Required | Client-generated UUID. Use as `parent_id` on child events. |
| `session_id` | `string` (UUID) | Required | Session UUID from `capture-session`. |
| `primitive_name` | `string` | Required | Agent name (turn-pair) or tool name (tool call). |
| `args` | `string` | Required | Input: user message or JSON-encoded tool args. |
| `result` | `string` | Required | Output: assistant reply or JSON-encoded tool result. |
| `success` | `boolean` | Optional | Defaults to `true`. |
| `latency` | `integer` (ms) | Optional | Execution time in milliseconds. |
| `timestamp` | `integer` (ms) | Optional | Unix time in ms when the event occurred. Defaults to server time. |
| `parent_id` | `string` (UUID) | Optional | Parent event UUID. Set on tool calls to point at the agent turn (or parent tool) that triggered them. |
| `metadata` | `object` | Optional | Free-form event metadata. |
## Core rules
* **One session per conversation**: reuse `session_id` on every event.
* **One event per turn-pair**: `primitive_name` is the agent name, `args` is the user input, `result` is the assistant output. Don't emit separate output events.
* **Merge assistant turns**: if one user turn is followed by N assistant turns (with tool calls between), concatenate the N texts into a single `result`.
* **Tool calls = own event**: `primitive_name` is the tool name, `args`/`result` are plain text or JSON-encoded strings.
* **Sub-tools chain**: when a tool invokes another tool, set `parent_id` to the parent **tool's** `event_id`.
## Verify
Send one event, then open **Events** in Agnost AI. Confirm `primitive_name`, `args`, `result`, `success`, and `latency` look right.
## Troubleshooting
* `event_id` and `session_id` must be UUIDs.
* Create the session first with [Capture Session](/capture-session).
* For tool calls, set `parent_id` when the tool was triggered by another agent/tool event.
## Next steps
* [Events](/using-events): inspect accepted events in Agnost AI.
* [Capture a session](/capture-session): review the required conversation boundary.
* [Ingestion errors](/errors): troubleshoot rejected requests.
# Capture Session
Source: https://docs.agnost.ai/capture-session
POST /api/v1/capture-session
Start a session at the beginning of a conversation
Call once per conversation. Reuse `session_id` on every event that follows.
## Body
| Field | Type | Required | Description |
| --------------- | --------------- | -------- | ---------------------------------------------------------------- |
| `session_id` | `string` (UUID) | Required | Session UUID. Reuse on every event in this conversation. |
| `user_data` | `object` | Required | End-user identity. Must contain `user_id`. |
| `metadata` | `object` | Optional | Free-form session metadata. |
| `timestamp` | `integer` (ms) | Optional | Unix time in ms when the session began. Defaults to server time. |
| `client_config` | `string` | Optional | Free-form client/SDK label. |
## Verify
After creating a session and sending at least one event, open **Events** in Agnost AI and confirm the `session_id` appears on the event.
## Troubleshooting
* `session_id` must be a UUID.
* `user_data.user_id` is required.
* Reuse the same `session_id` for every event in the conversation.
## Next steps
* [Capture an event](/capture-event): add activity to the session you created.
* [Authentication](/authentication): review organization routing and API access.
* [Ingestion errors](/errors): troubleshoot rejected requests.
# CrewAI
Source: https://docs.agnost.ai/crewai
Capture traces from CrewAI crews with OpenLit
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this CrewAI application.
Org ID: your-org-id
Instrument the real crew.kickoff() path and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash theme={null}
pip install agnost
```
```python theme={null}
import agnost
agnost.init("your-org-id")
interaction = agnost.begin(user_id="u-42", agent_name="crewai-research", input=prompt)
try:
result = crew.kickoff(inputs={"prompt": prompt})
interaction.end(output=str(result))
except Exception as exc:
interaction.end(output=str(exc), success=False)
raise
finally:
agnost.shutdown()
```
## Manual setup: OpenTelemetry
OpenLit is the recommended path: it auto-instruments CrewAI plus the underlying LLM provider.
### Install
```bash theme={null}
pip install crewai openlit
```
### Setup
```python theme={null}
import os, openlit
from crewai import Agent, Crew, Task
from openinference.instrumentation import using_attributes
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://otel.agnost.ai"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "X-Agnost-Org-ID="
openlit.init(application_name="my-crew", environment="production")
with using_attributes(session_id="sess-123", user_id="u-42"):
Crew(agents=[...], tasks=[...]).kickoff()
```
Spans use OTel GenAI semconv (`gen_ai.prompt.{n}.content`, `gen_ai.completion.{n}.content`, `gen_ai.usage.*`) plus `gen_ai.agent.name` and `gen_ai.operation.name` on agent/task spans.
### Caveats
* Call `openlit.init()` before constructing any `Crew` / `Agent` objects.
* CrewAI's own anonymized telemetry (sent to CrewAI's servers) is unrelated and can be left enabled.
### Verify
Run one `Crew(...).kickoff()` call, then open **Events** in Agnost AI. Confirm CrewAI and provider spans are present.
### Troubleshooting
* Call `openlit.init()` before constructing crews or agents.
* Confirm `OTEL_EXPORTER_OTLP_HEADERS` contains `X-Agnost-Org-ID=`.
* Wrap runs with `using_attributes` if you need explicit user/session grouping.
### References
* [Enable OpenTelemetry export](https://docs.crewai.com/en/observability/openlit)
* [Add custom metadata](https://docs.crewai.com/en/api-reference/kickoff)
## Next steps
* [Conversations](/using-conversations): review the complete CrewAI execution.
* [Events](/using-events): inspect crew, task, and model activity.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Data Governance
Source: https://docs.agnost.ai/data-governance
How to send production conversation data to Agnost AI while minimizing sensitive data exposure
Agnost AI analyzes the conversation and event data you send to it. That usually includes user prompts, agent outputs, tool calls, error messages, user identifiers, and metadata. If your product handles job seekers, patients, students, employees, or other sensitive user groups, treat this data as sensitive before it leaves your system.
This page explains how to instrument Agnost AI safely today.
Agnost AI does not currently provide automatic PII redaction or DLP before ingestion. If a field can contain personal data, confidential business data, or regulated data, redact, pseudonymize, or omit it in your application before sending it to Agnost AI.
## What Agnost AI receives
Depending on the integration path, Agnost AI may receive:
| Data type | Examples | Send it? |
| ------------------- | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| User identifiers | `user_id`, account ID, tenant ID | Yes, but use stable pseudonymous IDs instead of raw emails or names. |
| Conversation input | User prompts, chat turns, voice transcripts | Yes when needed for analysis; redact sensitive fields first. |
| Agent output | Assistant replies, tool results, generated actions | Yes when needed for analysis; redact secrets and personal data first. |
| Tool/event metadata | Tool name, model, latency, success flag, intent, plan | Yes. Prefer allowlisted operational metadata. |
| User traits | Plan, role, company segment, cohort | Yes if useful; avoid name, email, phone, address, resume, SSN, health, financial, or job-application details unless you have approved that data flow. |
## Recommended defaults
Use these defaults unless your legal/security review approves something broader:
1. Use an internal stable `user_id` instead of email, phone, or full name.
2. Send only metadata fields you intentionally allowlist.
3. Redact obvious PII from `input`, `output`, tool arguments, and tool results.
4. Do not send secrets, API keys, access tokens, passwords, private keys, or auth headers.
5. Do not send resumes, full job applications, government IDs, health records, payment card data, or other regulated data unless you have a specific agreement and retention plan.
6. Keep a local mapping from your internal user ID to the real person in your own system, not in Agnost AI metadata.
## Pseudonymous user identity
Prefer this:
```python theme={null}
agnost.identify("user_8f3a91", {
"plan": "team",
"role": "recruiter",
"account_segment": "mid_market"
})
```
Avoid this unless explicitly approved:
```python theme={null}
agnost.identify("alice@example.com", {
"name": "Alice Smith",
"email": "alice@example.com",
"phone": "+1-555-0100"
})
```
The first example still lets you analyze behavior by user, plan, and segment. It does not expose directly identifying traits in Agnost AI.
## Redact before sending
Add a small scrubber around your instrumentation layer. Keep it close to the code that calls Agnost AI so every integration path uses the same policy.
```python theme={null}
import re
EMAIL_RE = re.compile(r"\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b", re.I)
PHONE_RE = re.compile(r"\+?\d[\d\s().-]{7,}\d")
def redact_for_agnost(text: str | None) -> str:
if not text:
return ""
text = EMAIL_RE.sub("[redacted_email]", text)
text = PHONE_RE.sub("[redacted_phone]", text)
return text
interaction = agnost.begin(
user_id="user_8f3a91",
agent_name="support-agent",
input=redact_for_agnost(user_message),
properties={
"plan": "team",
"intent_source": "support_chat"
}
)
result = call_agent(user_message)
interaction.end(output=redact_for_agnost(result))
```
For structured tool calls, redact before JSON serialization or remove sensitive keys entirely:
```python theme={null}
SAFE_KEYS = {"tool_name", "status", "error_code", "model", "latency_ms"}
def allowlisted_metadata(metadata: dict) -> dict:
return {key: value for key, value in metadata.items() if key in SAFE_KEYS}
```
## Metadata allowlist
Metadata is often more useful than raw personal data. Start with operational fields:
```json theme={null}
{
"model": "gpt-4.1",
"plan": "team",
"agent_version": "2026-07-02",
"surface": "onboarding",
"intent": "setup_friction",
"success": false
}
```
Avoid free-form user traits like:
```json theme={null}
{
"email": "alice@example.com",
"resume_text": "...",
"home_address": "...",
"cover_letter": "..."
}
```
## OpenTelemetry integrations
Many OTel integrations capture prompts, messages, tool parameters, and tool results automatically. Before enabling full traces in production:
1. Review what your framework exports.
2. Disable or scrub message/tool attributes that contain sensitive data.
3. Keep `user.id`, `session.id`, and tenant metadata pseudonymous.
4. Test with one staging trace and inspect the raw event in Agnost AI before rolling out broadly.
## If you handle regulated or high-risk data
If your users may enter health data, payment card data, government IDs, children's data, candidate/job-application data, or other regulated information, do not enable raw input/output capture until your team has reviewed the data flow.
Use one of these patterns instead:
| Pattern | When to use |
| --------------------------- | ----------------------------------------------------------------------------- |
| Metadata-only capture | You only need latency, success, agent name, model, and intent labels. |
| Redacted transcript capture | You need conversation analysis but can remove direct identifiers and secrets. |
| Sampled capture | You need debugging coverage on a limited subset of traffic. |
| Customer-approved capture | You have explicit contractual approval for the data categories being sent. |
For security or data-processing questions, contact [founders@agnost.ai](mailto:founders@agnost.ai).
## Instrumentation checklist
Before going live:
* [ ] Replace raw emails/names with internal user IDs.
* [ ] Remove secrets from inputs, outputs, tool args, and tool results.
* [ ] Allowlist metadata keys.
* [ ] Redact obvious PII from text fields.
* [ ] Confirm whether raw transcripts are necessary, or whether metadata-only events are enough.
* [ ] Run one test conversation and inspect the event in the Agnost AI dashboard.
* [ ] Document internally which fields your integration sends.
## Summary
Agnost AI is most useful when it can see real production behavior, but you control the data boundary. Send enough context to debug and improve your agent, and keep directly identifying or regulated data in your own system unless it has been explicitly approved for ingestion.
## Next steps
* [Security](/security): review the broader security and procurement model.
* [Authentication](/authentication): understand organization routing and API access.
* [Quickstart](/quickstart): configure and verify one interaction.
# Agnost AI developer resources
Source: https://docs.agnost.ai/developer-resources
Canonical API, authentication, SDK, webhook, and MCP resources for Agnost AI
Use this page as the canonical index for integrating with Agnost AI by API, SDK,
OpenTelemetry, or Model Context Protocol (MCP).
## Machine-readable contracts
* [Agnost AI OpenAPI specification](https://docs.agnost.ai/openapi.yaml) — the canonical REST API contract in YAML.
* [Agnost AI OpenAPI specification](https://docs.agnost.ai/openapi.json) — the same REST API contract in JSON.
* [OAuth authorization-server metadata](https://mcp.agnost.ai/.well-known/oauth-authorization-server) — OAuth 2.1 endpoints and supported scopes for the hosted MCP server.
* [OAuth protected-resource metadata](https://mcp.agnost.ai/.well-known/oauth-protected-resource) — resource-server discovery for MCP clients.
* [Agent documentation index](https://docs.agnost.ai/llms.txt) — the machine-readable documentation map.
## Integration guides
* [Authentication](/authentication) — ingestion, JWT, and API-key authentication.
* [Packages and versions](/sdks) — official Python, TypeScript, and Go packages.
* [Capture Session API](/capture-session) and [Capture Event API](/capture-event) — direct ingestion endpoints.
* [Hosted Agnost AI MCP server](/agnost-mcp-server) — connect Claude, Cursor, and other MCP clients to `https://mcp.agnost.ai/mcp`.
* [Vapi webhook integration](/vapi) — send voice-agent end-of-call reports to Agnost AI.
## Agent-readable website responses
Request `https://agnost.ai/` with `Accept: text/markdown` to receive the Agnost
AI agent index. Missing website paths preserve their `404` or `410` status and
return a short Markdown recovery document when the same media type is requested.
Negotiated responses include `Vary: Accept, Accept-Encoding` so shared caches do
not mix HTML and Markdown variants.
## When to use Agnost AI
Use Agnost AI when a team needs to inspect production conversations and traces,
find silent failures or unmet user intent, monitor expected agent behavior, or
turn recurring evidence into reviewed agent improvements. Use the SDK or
OpenTelemetry guides to send data; use the REST API or hosted MCP server to query
and operate on the resulting analytics.
# DSPy
Source: https://docs.agnost.ai/dspy
Capture traces from DSPy modules and optimizers
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this DSPy application.
Org ID: your-org-id
Instrument the real DSPy module call and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash theme={null}
pip install agnost
```
```python theme={null}
import agnost
agnost.init("your-org-id")
interaction = agnost.begin(user_id="u-42", agent_name="dspy-qa", input=question)
try:
result = qa(question=question)
interaction.end(output=result.answer)
except Exception as exc:
interaction.end(output=str(exc), success=False)
raise
finally:
agnost.shutdown()
```
## Manual setup: OpenTelemetry
### Install
```bash theme={null}
pip install dspy-ai openinference-instrumentation-dspy \
opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```
### Setup
```python theme={null}
import os, dspy
from openinference.instrumentation.dspy import DSPyInstrumentor
from openinference.instrumentation import using_attributes
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://otel.agnost.ai"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "X-Agnost-Org-ID="
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
DSPyInstrumentor().instrument(tracer_provider=provider)
dspy.settings.configure(lm=dspy.LM("openai/gpt-4o-mini"))
qa = dspy.Predict("question -> answer")
with using_attributes(session_id="sess-123", user_id="u-42"):
qa(question="What is OTel?")
```
Module/program/optimizer spans nest properly and carry `dspy.module`, `dspy.signature`, plus standard OpenInference `input.value` / `output.value`. Pair with a provider instrumentation (e.g. `OpenAIInstrumentor`) for the deepest trace tree.
### Alternative: MLflow
If you already use MLflow, `mlflow.dspy.autolog()` works too: Agnost AI reads `mlflow.spanInputs` / `mlflow.spanOutputs`.
### Verify
Run one DSPy module call, then open **Events** in Agnost AI. Confirm `dspy.module` and input/output attributes are present.
### Troubleshooting
* Instrument DSPy before calling modules.
* Pair DSPy instrumentation with provider instrumentation if you need LLM-level spans.
* Confirm `OTEL_EXPORTER_OTLP_HEADERS` contains `X-Agnost-Org-ID=`.
### References
* [Enable OpenTelemetry export](https://dspy.ai/tutorials/observability/)
* [Add custom metadata](https://mlflow.org/docs/latest/genai/tracing/app-instrumentation/manual-tracing/)
## Next steps
* [Conversations](/using-conversations): review the complete DSPy interaction.
* [Events](/using-events): inspect module and model activity.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Errors
Source: https://docs.agnost.ai/errors
Common API and ingestion errors
Most Agnost AI API errors return a JSON body with an `error` string:
```json theme={null}
{
"error": "Invalid request body"
}
```
## Common errors
| Symptom | Likely cause | Fix |
| ------------------------ | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
| No events appear | Wrong org ID or endpoint | Confirm org ID in Settings -> Organization and use `https://api.agnost.ai` or `https://otel.agnost.ai/v1/traces`. |
| 401/403 on dashboard API | Missing JWT/API key or wrong org scope | Use `Authorization: Bearer ` or `x-api-key` plus the right `x-org-id`. |
| 400 on ingestion | Missing required field or invalid UUID | Validate `session_id`, `event_id`, and `user_data.user_id`. |
| OTel traces missing | Header not attached or exporter endpoint is wrong | Set `OTEL_EXPORTER_OTLP_HEADERS=X-Agnost-Org-ID=` and use `/v1/traces` when the SDK expects a trace URL. |
| Tool inputs missing | Input capture disabled or framework did not export content | Check SDK config and framework telemetry settings. |
## Debugging checklist
1. Send one test event.
2. Open **Events** in the Agnost AI dashboard.
3. Confirm the event has the expected org ID, user ID, session/conversation ID, and agent/tool name.
4. Check **Tool calls** or **Conversations** depending on the integration.
5. If using OpenTelemetry, confirm your app can reach `otel.agnost.ai`.
## Related pages
* [Authentication](/authentication)
* [OpenTelemetry troubleshooting](/otel#troubleshooting)
## Next steps
* [Quickstart](/quickstart): compare your setup with the recommended path.
* [Authentication](/authentication): verify headers and organization routing.
* [Events](/using-events): confirm whether telemetry reached Agnost AI.
# Python FastMCP
Source: https://docs.agnost.ai/fastmcp
Add Agnost AI analytics to your FastMCP server
## Install
```bash pip theme={null}
pip install agnost-mcp
```
```bash uv theme={null}
uv add agnost-mcp
```
## Integrate
Call `track` after defining your server and tools, before running:
```python theme={null}
from mcp.server.fastmcp import FastMCP
from agnost_mcp import track, config
server = FastMCP("Your Server Name")
# ... your tools ...
track(server, "your-org-id", config(
endpoint="https://api.agnost.ai",
disable_input=False,
disable_output=False,
))
if __name__ == "__main__":
server.run()
```
Get your org ID from [app.agnost.ai](https://app.agnost.ai).
## Identify users
Pass an `identify` function to resolve a user from the incoming request context. It receives the raw request object and the process environment, and should return a dict with at least a `userId` key:
```python theme={null}
from agnost_mcp import track, config
track(server, "your-org-id", config(
identify=lambda req, env: {
"userId": req.get("headers", {}).get("x-user-id") or env.get("USER_ID", "anonymous"),
"email": req.get("headers", {}).get("x-user-email") or env.get("USER_EMAIL"),
"role": req.get("headers", {}).get("x-user-role") or env.get("USER_ROLE", "user"),
}
))
```
The function can also be `async`. Return `None` to skip user attribution for a request.
## Options
| Field | Type | Default | Description |
| ---------------- | ---------- | ----------------------- | ----------------------------------------------------------------------------- |
| `endpoint` | `str` | `https://api.agnost.ai` | API endpoint |
| `disable_input` | `bool` | `False` | Skip capturing tool input arguments |
| `disable_output` | `bool` | `False` | Skip capturing tool output / result |
| `log_level` | `str` | `"INFO"` | Log verbosity: `DEBUG`, `INFO`, `WARNING`, `ERROR` |
| `identify` | `Callable` | `None` | Function `(request, env) → UserIdentity` to resolve user identity per request |
`UserIdentity` is a `Dict[str, Any]` that **must contain a `userId` key**. Other fields (e.g. `email`, `role`) are optional and forwarded as user traits. The identify callable can be sync or async, and may return `None` to skip attribution for a request.
## What appears in Agnost AI
* **Tool calls** for FastMCP tool invocations.
* **Events** for every tracked call.
* Failed tool calls remain visible in **Events** with their error details.
## Verify
Call one FastMCP tool from your MCP client, then open [app.agnost.ai](https://app.agnost.ai). Check **Events** first, then **Tool calls**.
## Troubleshooting
* Call `track(...)` after defining tools and before `server.run()`.
* Confirm the org ID is correct.
## Next steps
* [Tool calls](/using-tool-calls): analyze calls from your FastMCP server.
* [Events](/using-events): verify call fields and execution details.
* [Alerts](/using-alerts): monitor important MCP tool conditions.
# Go MCP SDK
Source: https://docs.agnost.ai/golang-sdk
Add Agnost AI analytics to your Go MCP server
## Install
```bash theme={null}
go get github.com/agnostai/agnost-go/agnost
```
## Integrate
Call `agnost.Track` after adding your tools, before serving:
```go theme={null}
import "github.com/agnostai/agnost-go/agnost"
// your existing server setup...
err := agnost.Track(s, "your-org-id", &agnost.Config{
Endpoint: "https://api.agnost.ai",
DisableInput: false,
DisableOutput: false,
})
if err != nil {
log.Printf("analytics init failed: %v", err)
}
server.ServeStdio(s)
```
Get your org ID from [app.agnost.ai](https://app.agnost.ai).
## Options
| Field | Type | Default | Description |
| --------------- | -------- | ----------------------- | ----------------------------------------- |
| `Endpoint` | `string` | `https://api.agnost.ai` | API endpoint |
| `DisableInput` | `bool` | `false` | Skip tracking tool inputs |
| `DisableOutput` | `bool` | `false` | Skip tracking tool outputs |
| `Identify` | `func` | `nil` | Return user identity from request context |
| `LogLevel` | `string` | `"info"` | `debug`, `info`, `warning`, `error` |
## What appears in Agnost AI
* **Tool calls** for MCP tool invocations.
* **Events** for every tracked call.
* Failed tool calls remain visible in **Events** with their error details.
## Verify
Call one MCP tool from your client, then open [app.agnost.ai](https://app.agnost.ai). Check **Events** first, then **Tool calls**.
## Troubleshooting
* Call `agnost.Track` after adding tools and before serving.
* Confirm the org ID is correct.
## Next steps
* [Tool calls](/using-tool-calls): analyze calls from your Go server.
* [Events](/using-events): verify call fields and execution details.
* [Alerts](/using-alerts): monitor important MCP tool conditions.
# Understand your AI agents in production
Source: https://docs.agnost.ai/index
See what users want, where agents break expected behavior, and what to improve next
Agnost AI helps teams understand and improve AI agents using real production
interactions. It shows what users are trying to accomplish, identifies where
agents violate expected behavior, and connects every finding to the conversation,
event, or tool call behind it.
Connect an agent and see your first interaction in Agnost AI.
## Why teams use Agnost AI
Use intents to see what users are trying to accomplish across real conversations.
Find conversations where an agent did not follow an expected rule.
Trace an outcome back to the events and tool calls that produced it.
Turn production evidence into alerts and concrete improvements.
## How it works
Use the Agnost AI skill, an Agnost AI SDK, or OpenTelemetry with your existing framework.
Analyze conversations through intents and violations, with the supporting evidence attached.
Monitor important conditions with alerts and turn recurring issues into improvements.
## The data behind every finding
| Data | What it represents |
| ----------------- | --------------------------------------------------------------------------- |
| **Conversations** | Complete user interactions grouped into one timeline. |
| **Events** | Agent turns, model generations, handoffs, guardrails, and other operations. |
| **Tool calls** | Tool execution details, outcomes, and latency. |
Agnost AI analyzes this production data into **intents** and **violations**. You
can always open the supporting conversation and inspect the underlying events
before deciding what to change.
## See your first interaction
Use the [Quickstart](/quickstart) for guided setup, or select your existing
framework under **Agent frameworks** in the sidebar.
Connect an agent, trigger one real interaction, and verify that it reached Agnost AI.
## Next steps
* [Quickstart](/quickstart): send your first interaction to Agnost AI.
* [Conversations](/using-conversations): learn how related activity is grouped.
* [Intents](/using-intents): organize conversations by what users wanted.
# LangChain
Source: https://docs.agnost.ai/langchain
Capture traces from LangChain / LangGraph via LangSmith's OTel mode
## Language support
| Setup option | Python | TypeScript |
| ------------------------------- | :----: | :--------: |
| Agnost AI skill | ✓ | ✓ |
| Agnost AI SDK | ✓ | ✓ |
| OpenTelemetry through LangSmith | ✓ | ✓ |
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this LangChain or LangGraph application.
Org ID: your-org-id
Instrument the real chain, graph, or agent invocation and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash Python theme={null}
pip install agnost
```
```bash TypeScript theme={null}
npm install agnostai
```
```python Python theme={null}
import agnost
agnost.init("your-org-id")
interaction = agnost.begin(user_id="u-42", agent_name="langchain-support", input=prompt)
try:
result = chain.invoke({"input": prompt})
interaction.end(output=str(result))
except Exception as exc:
interaction.end(output=str(exc), success=False)
raise
finally:
agnost.shutdown()
```
```typescript TypeScript theme={null}
import * as agnost from 'agnostai';
agnost.init('your-org-id');
const interaction = agnost.begin({ userId: 'u-42', agentName: 'langchain-support', input: prompt });
try {
const result = await chain.invoke({ input: prompt });
interaction.end(JSON.stringify(result));
} catch (error) {
interaction.end(String(error), false);
throw error;
} finally {
await agnost.shutdown();
}
```
## Manual setup: OpenTelemetry
LangChain emits OTel through LangSmith. Register the OTel exporter pointing at Agnost AI before building chains or agents, enable LangSmith's OTel mode, then pass `session_id` / `user_id` per call.
### 1. Install
**Already have LangSmith and an OTLP exporter installed?** Skip.
**No setup yet?**
```bash Python theme={null}
pip install "langsmith[otel]" langchain \
opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```
```bash TypeScript theme={null}
npm install langsmith @opentelemetry/api @opentelemetry/context-async-hooks \
@opentelemetry/sdk-trace-base @opentelemetry/exporter-trace-otlp-proto
```
### 2. Wire LangSmith OTel pointing at Agnost AI
The provider must be registered before importing or constructing LangChain objects.
**Already have an OTel TracerProvider?** Append Agnost AI as an additional span processor:
```python Python theme={null}
import os
from opentelemetry import trace
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
trace.get_tracer_provider().add_span_processor(
BatchSpanProcessor(
OTLPSpanExporter(
endpoint="https://otel.agnost.ai/v1/traces",
headers={"X-Agnost-Org-ID": os.environ["AGNOST_ORG_ID"]},
)
)
)
os.environ["LANGSMITH_OTEL_ENABLED"] = "true"
os.environ["LANGSMITH_TRACING"] = "true"
os.environ["LANGSMITH_OTEL_ONLY"] = "true"
```
**No OTel yet?** Full setup:
```python Python theme={null}
import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
provider = TracerProvider()
provider.add_span_processor(
BatchSpanProcessor(
OTLPSpanExporter(
endpoint="https://otel.agnost.ai/v1/traces",
headers={"X-Agnost-Org-ID": os.environ["AGNOST_ORG_ID"]},
)
)
)
trace.set_tracer_provider(provider)
os.environ["LANGSMITH_OTEL_ENABLED"] = "true"
os.environ["LANGSMITH_TRACING"] = "true"
os.environ["LANGSMITH_OTEL_ONLY"] = "true"
# Import LangChain AFTER the provider is registered.
from langchain_openai import ChatOpenAI
```
```typescript TypeScript theme={null}
import { trace, SpanStatusCode } from '@opentelemetry/api';
process.env.LANGSMITH_TRACING = 'true';
process.env.LANGCHAIN_TRACING_V2 = 'true';
process.env.LANGSMITH_TRACING_MODE = 'otel';
const { initializeOTEL } = await import('langsmith/experimental/otel/setup');
initializeOTEL({
exporterConfig: {
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
},
});
```
For TypeScript, `langsmith/experimental/vercel` is the Vercel AI SDK 7 adapter. LangChain JS/TS should use `langsmith/experimental/otel/setup`.
For LangChain JS/TS, keep one active OTel span around the actual invoke. The
LangSmith OTel translator attaches LangChain run data to that active span, and
the GenAI attributes below give Agnost AI exact Chat View input/output text.
### 3. Pass user\_id / session\_id per call
```python Python theme={null}
result = agent.invoke(
{"messages": [{"role": "user", "content": "Hello"}]},
config={
"metadata": {
"session_id": "conv-abc123",
"user_id": "user-42",
},
},
)
```
```typescript TypeScript theme={null}
const tracer = trace.getTracer('langchain-ts');
const userMessage = 'Hello';
const result = await tracer.startActiveSpan('langchain.invoke', {
attributes: {
'langsmith.traceable': 'true',
'langsmith.metadata.session_id': 'conv-abc123',
'langsmith.metadata.user_id': 'user-42',
'gen_ai.operation.name': 'chat',
'gen_ai.system': 'langchain',
'gen_ai.input.messages': JSON.stringify([{ role: 'user', content: userMessage }]),
},
}, async (span) => {
try {
const result = await chain.invoke(
{ messages: [{ role: 'user', content: userMessage }] },
{
metadata: {
session_id: 'conv-abc123',
user_id: 'user-42',
},
tags: ['agnost', 'langchain-ts'],
},
);
const answer = typeof result.content === 'string' ? result.content : JSON.stringify(result);
span.setAttribute('gen_ai.output.messages', JSON.stringify([{ role: 'assistant', content: answer }]));
span.setStatus({ code: SpanStatusCode.OK });
return result;
} catch (error) {
span.recordException(error as Error);
span.setStatus({ code: SpanStatusCode.ERROR, message: error instanceof Error ? error.message : String(error) });
throw error;
} finally {
span.end();
}
});
```
LangChain prefixes metadata as `langsmith.metadata.*`: Agnost AI reads `langsmith.metadata.session_id` and `langsmith.metadata.user_id` natively for user / session grouping.
For Chat View, keep the assistant answer in the normal LangChain output:
message content, `generations`, or a chain result wrapper such as `answer`,
`output`, or `result`. Agnost AI stores the raw OTel payload, then collapses those
common output shapes to readable text for the dashboard.
### What appears in Agnost AI
* **Conversations** grouped by `session_id`.
* **User-level analytics** grouped by `user_id`.
* **Events** for LangSmith OTel spans.
### Verify
Run one `agent.invoke`, then open **Events** and confirm `langsmith.metadata.session_id` and `langsmith.metadata.user_id` exist.
### Troubleshooting
* Register the `TracerProvider` before importing LangChain.
* Set `LANGSMITH_OTEL_ENABLED=true` and `LANGSMITH_TRACING=true`.
* Set `LANGSMITH_OTEL_ONLY=true` if you do not want to also ship to LangSmith.
### References
* [Enable OpenTelemetry export](https://docs.langchain.com/langsmith/trace-with-opentelemetry)
* [Add custom metadata](https://docs.langchain.com/langsmith/add-metadata-tags)
## Next steps
* [Conversations](/using-conversations): review the complete chain or graph execution.
* [Events](/using-events): inspect LangChain and LangGraph spans.
* [Intents](/using-intents): organize production conversations by what users wanted.
# LiveKit Agents
Source: https://docs.agnost.ai/livekit
Export native LiveKit Agents traces to Agnost AI over OpenTelemetry
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this LiveKit Agents application.
Org ID: your-org-id
Use LiveKit's native OpenTelemetry traces and verify one fresh agent session.
```
LiveKit Agents already instruments agent sessions, turns, model requests, tool calls,
speech, and transcription with OpenTelemetry. Point those native spans at Agnost AI;
no webhook or Agnost SDK wrapper is required.
## 1. Configure the OTLP destination
Set these variables in the agent worker environment:
```bash theme={null}
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://otel.agnost.ai/v1/traces
OTEL_EXPORTER_OTLP_HEADERS="X-Agnost-Org-ID=your-org-id"
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
```
## 2. Register one batch exporter
Call LiveKit's tracer-provider hook before `AgentSession.start()`.
```python Python theme={null}
from livekit.agents.telemetry import set_tracer_provider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
set_tracer_provider(provider, metadata={
"session.id": conversation_id,
"user.id": user_id,
})
```
```typescript TypeScript theme={null}
import { telemetry } from '@livekit/agents';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { BatchSpanProcessor, NodeTracerProvider } from '@opentelemetry/sdk-trace-node';
const fanout = new telemetry.FanoutSpanProcessor();
const provider = new NodeTracerProvider({
spanProcessors: [
new BatchSpanProcessor(new OTLPTraceExporter()),
fanout,
],
});
provider.register();
telemetry.setTracerProvider(provider, {
metadata: {
'session.id': conversationId,
'user.id': userId,
},
registerSpanProcessor: (processor) => fanout.add(processor),
});
```
Install the OTel packages if the application does not already have them:
```bash Python theme={null}
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```
```bash TypeScript theme={null}
npm install @opentelemetry/api @opentelemetry/exporter-trace-otlp-http @opentelemetry/sdk-trace-node
```
The TypeScript `FanoutSpanProcessor` is required by the current OTel 2.x API so
LiveKit can attach its metadata processor (and its Cloud exporter, when enabled)
after provider construction.
## 3. Add standard conversation context
Pass metadata to `set_tracer_provider`; LiveKit copies it to every native span.
Use standard attributes rather than Agnost-specific names:
```python theme={null}
set_tracer_provider(provider, metadata={
"session.id": ctx.room.name,
"user.id": customer_id,
"user.plan": "enterprise",
"user.locale": "en-US",
"conversation.channel": "voice",
"conversation.environment": "production",
})
```
| Attribute | Agnost behavior |
| ------------------------------------- | ---------------------------------------- |
| `session.id` | Groups every span into one conversation. |
| `user.id` | Groups conversations into one user. |
| `user.*` or `enduser.*` | Promoted to user metadata. |
| `conversation.*` or `session.*` | Promoted to conversation metadata. |
| Any other attribute, including `lk.*` | Preserved on the event. |
If `session.id` is absent, Agnost falls back to LiveKit's `lk.job_id`, then
`lk.room_name`. For user identity, explicit `user.id` is best;
the participant identity on a native user-turn span is a safe fallback.
## What appears in Agnost AI
* `agent_turn` spans become the ordered user/assistant conversation turns.
* `function_tool` spans become tool calls with native arguments, output, and error state.
* LLM request spans retain model, token, and `gen_ai.*` attributes.
* STT, TTS, speaking, evaluator, session, job, and activity spans remain visible in the trace.
* Native `lk.*`, standard `gen_ai.*`, and your custom attributes remain available on events.
LiveKit webhooks are useful for room and participant lifecycle events, but they do
not carry the complete agent turn, model, tool, and speech span tree. Use native
OTLP export for observability.
## Flush on shutdown
Batch export is asynchronous. Register a shutdown callback so the worker sends
the final turns before it exits:
```python Python theme={null}
async def flush_traces():
provider.force_flush()
ctx.add_shutdown_callback(flush_traces)
```
```typescript TypeScript theme={null}
ctx.addShutdownCallback(async () => {
await provider.shutdown();
});
```
## Verify
Run one real LiveKit agent session, then open **Users** in Agnost AI. Confirm:
1. The user appears under the value sent in `user.id`.
2. The conversation shows alternating user and assistant turns.
3. The trace contains the LiveKit agent, LLM, tool, STT, and TTS spans used by the session.
4. Tool details show their native input and output.
## Troubleshooting
* Register the provider before `AgentSession.start()`; earlier spans cannot be recovered.
* Use the traces URL ending in `/v1/traces` with the HTTP/protobuf exporter.
* Confirm `X-Agnost-Org-ID` is present in `OTEL_EXPORTER_OTLP_HEADERS`.
* Do not point a generic OTLP exporter at LiveKit's private observability URL.
* Flush the provider when a short-lived worker or test exits.
## References
* [LiveKit: Export traces](https://docs.livekit.io/deploy/observability/tracing/)
* [LiveKit Python OTel example](https://github.com/livekit/agents/blob/main/examples/voice_agents/otel_trace.py)
* [LiveKit TypeScript OTel example](https://github.com/livekit/agents-js/blob/main/examples/src/otel_trace.ts)
* [LiveKit TypeScript `setTracerProvider` reference](https://docs.livekit.io/reference/agents-js/functions/agents.telemetry.setTracerProvider.html)
## Next steps
* [Conversations](/using-conversations): review complete voice interactions.
* [Events](/using-events): inspect native LiveKit spans and attributes.
* [Tool calls](/using-tool-calls): analyze tool arguments, results, errors, and latency.
# Mastra
Source: https://docs.agnost.ai/mastra
Capture traces from Mastra agents with the Observability + OtelExporter API
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this Mastra application.
Org ID: your-org-id
Instrument the real agent.generate() path and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash theme={null}
npm install agnostai
```
```typescript theme={null}
import * as agnost from 'agnostai';
agnost.init('your-org-id');
const interaction = agnost.begin({ userId: 'u-42', agentName: 'mastra-support', input: prompt });
try {
const result = await agent.generate([{ role: 'user', content: prompt }]);
interaction.end(result.text);
} catch (error) {
interaction.end(String(error), false);
throw error;
} finally {
await agnost.shutdown();
}
```
## Manual setup: OpenTelemetry
Mastra ships an `OtelExporter` that targets any OTLP endpoint. Wire it into your Mastra instance, then pass `userId` / `conversationId` per call.
### 1. Install
**Already have `@mastra/observability` and `@mastra/otel-exporter`?** Skip.
**No Observability set up yet?**
```bash theme={null}
npm install @mastra/observability @mastra/otel-exporter
```
### 2. Wire the OtelExporter pointing at Agnost AI
**Already have Mastra Observability?** Append Agnost AI to the existing `exporters` array: Mastra fans traces out to every exporter in the list:
```typescript theme={null}
import { OtelExporter } from '@mastra/otel-exporter';
const agnostExporter = new OtelExporter({
provider: {
custom: {
endpoint: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
protocol: 'http/protobuf',
},
},
});
// Inside your existing Observability config:
exporters: [
// ...your existing exporters,
agnostExporter,
]
```
**No Observability yet?** Full setup:
```typescript theme={null}
import { Mastra } from '@mastra/core';
import { Observability } from '@mastra/observability';
import { OtelExporter } from '@mastra/otel-exporter';
export const mastra = new Mastra({
agents: { agent },
observability: new Observability({
configs: {
default: {
serviceName: 'my-mastra-app',
exporters: [
new OtelExporter({
provider: {
custom: {
endpoint: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
protocol: 'http/protobuf',
},
},
}),
],
},
},
}),
});
```
### 3. Pass userId / conversationId per call
```typescript theme={null}
await agent.generate('Hello', {
tracingOptions: {
metadata: {
userId: 'user-42',
conversationId: 'conv-abc123',
},
},
});
```
Reserved keys `userId`, `conversationId`, and `threadId` in `tracingOptions.metadata` are read by Agnost AI for user / session grouping.
### What appears in Agnost AI
* **Conversations** grouped by `conversationId` or `threadId`.
* **User-level analytics** grouped by `userId`.
* **Events** for spans exported by Mastra.
### Verify
Run one `agent.generate` call, then open **Events** in Agnost AI. Confirm the span has your `userId` and `conversationId`.
### Troubleshooting
* Confirm the `OtelExporter` is included in the active Mastra observability config.
* Confirm `protocol: 'http/protobuf'` and `/v1/traces` are used.
* Confirm metadata keys are spelled `userId`, `conversationId`, or `threadId`.
### References
* [Enable OpenTelemetry export](https://mastra.ai/docs/observability/tracing/exporters/otel)
* [Add custom metadata](https://mastra.ai/docs/observability/tracing/overview)
## Next steps
* [Conversations](/using-conversations): review the complete Mastra interaction.
* [Events](/using-events): inspect agent and model activity.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Overview
Source: https://docs.agnost.ai/mcp-overview
Track MCP tool calls, latency, outcomes, and execution details
MCP server analytics shows how clients use the tools exposed by your MCP server.
Agnost AI records each tool call with its name, input, output, latency, outcome,
and conversation context.
## How it works
```
MCP client → Your MCP server → Agnost AI
```
Each tool invocation becomes a **tool call** and an underlying **event**. Related
calls share a conversation identifier so you can inspect the complete interaction.
## Choose your server language
MCP server in TypeScript / JavaScript
MCP server with FastMCP
MCP server in Go
Low-level Anthropic Python SDK
## Next steps
* [Tool calls](/using-tool-calls): learn what MCP server analytics makes visible.
* [Events](/using-events): inspect the underlying records for each call.
* [Alerts](/using-alerts): monitor important MCP tool conditions.
# MCP Toolbox for Databases
Source: https://docs.agnost.ai/mcp-toolbox
Add Agnost AI analytics to Google's MCP Toolbox for Databases
## Prerequisites
* [MCP Toolbox](https://github.com/googleapis/genai-toolbox) installed
* A `tools.yaml` configuration file for your toolbox setup
## Installation
First, check the current version:
```bash theme={null}
./toolbox version
```
Choose your preferred installation method:
```bash theme={null}
export VERSION=0.28.0 # or your preferred version
curl -O https://storage.googleapis.com/genai-toolbox/v$VERSION/linux/amd64/toolbox
chmod +x toolbox
```
```bash theme={null}
export VERSION=0.28.0 # or your preferred version
docker pull us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:$VERSION
```
```bash theme={null}
brew install mcp-toolbox
```
## Enable Telemetry
Set this environment variable before running the server. Use whichever mechanism
your environment provides, such as a shell configuration, `.env` file, container
configuration, or deployment-platform settings:
```dotenv theme={null}
OTEL_EXPORTER_OTLP_HEADERS="X-Agnost-Org-ID="
```
Then run your MCP Toolbox with telemetry enabled and point it to Agnost AI:
```bash theme={null}
./toolbox \
--tools-file "tools.yaml" \
--telemetry-otlp="otel.agnost.ai"
```
Replace `` with the organization ID from your
[Agnost AI dashboard](https://app.agnost.ai).
### Docker Usage
If using Docker:
```bash theme={null}
docker run \
-e OTEL_EXPORTER_OTLP_HEADERS="X-Agnost-Org-ID=" \
-v $(pwd)/tools.yaml:/app/tools.yaml \
us-central1-docker.pkg.dev/database-toolbox/toolbox/toolbox:$VERSION \
--tools-file "/app/tools.yaml" \
--telemetry-otlp="otel.agnost.ai"
```
## That's It!
Your MCP Toolbox is now sending telemetry data to Agnost AI. Visit your [Agnost AI dashboard](https://app.agnost.ai) to:
* Monitor tool invocations in real-time
* Track performance metrics and response times
* Analyze usage patterns and optimization opportunities
* Set up alerts for performance issues
## Verify
Call one Toolbox tool, then open **Events** and **Tool calls** in Agnost AI. You should see the tool name, latency, success/failure, and exported metadata.
## Troubleshooting
* Confirm `OTEL_EXPORTER_OTLP_HEADERS` contains `X-Agnost-Org-ID=`.
* Confirm Toolbox is started with `--telemetry-otlp="otel.agnost.ai"`.
* If running Docker, confirm the env var is passed into the container.
## Need Help?
* Check your [dashboard](https://app.agnost.ai) for incoming data
* Contact us at [founders@agnost.ai](mailto:founders@agnost.ai) if you need assistance
* [Book a call](https://call.agnost.ai/) for personalized setup support
## MCP Toolbox Support Resources
* Visit the [MCP Toolbox documentation](https://mcp-toolbox.dev/documentation/getting-started/) for detailed guides
## Next steps
* [Tool calls](/using-tool-calls): analyze Toolbox usage, outcomes, and latency.
* [Events](/using-events): inspect the exported OpenTelemetry records.
* [Alerts](/using-alerts): monitor important tool conditions.
# OpenAI SDK
Source: https://docs.agnost.ai/openai
Capture traces from the OpenAI Python or TypeScript SDK with OpenInference
## Language support
| Setup option | Python | TypeScript |
| ---------------------------------- | :----: | :--------: |
| Agnost AI skill | ✓ | ✓ |
| Agnost AI SDK | ✓ | ✓ |
| OpenTelemetry auto-instrumentation | ✓ | ✓ |
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this OpenAI SDK application.
Org ID: your-org-id
Instrument the real Responses API call and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash Python theme={null}
pip install agnost
```
```bash TypeScript theme={null}
npm install agnostai
```
```python Python theme={null}
import agnost
from openai import OpenAI
agnost.init("your-org-id")
client = OpenAI()
interaction = agnost.begin(user_id="u-42", agent_name="openai-support", input=prompt)
try:
response = client.responses.create(model="gpt-4.1-mini", input=prompt)
interaction.end(output=response.output_text)
except Exception as exc:
interaction.end(output=str(exc), success=False)
raise
finally:
agnost.shutdown()
```
```typescript TypeScript theme={null}
import OpenAI from 'openai';
import * as agnost from 'agnostai';
agnost.init('your-org-id');
const client = new OpenAI();
const interaction = agnost.begin({ userId: 'u-42', agentName: 'openai-support', input: prompt });
try {
const response = await client.responses.create({ model: 'gpt-4.1-mini', input: prompt });
interaction.end(response.output_text);
} catch (error) {
interaction.end(String(error), false);
throw error;
} finally {
await agnost.shutdown();
}
```
## Manual setup: OpenTelemetry
The `openai` SDK ships no first-party OTel: use OpenInference's auto-instrumentation. Pick your language in the code blocks below; the choice persists across the page.
### 1. Install
**Already have OpenInference + an OTLP exporter wired up?** Skip.
**No setup yet?**
```bash Python theme={null}
pip install openai openinference-instrumentation-openai \
opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```
```bash TypeScript theme={null}
npm install openai @arizeai/openinference-instrumentation-openai \
@arizeai/openinference-core @opentelemetry/sdk-node \
@opentelemetry/exporter-trace-otlp-proto @opentelemetry/api
```
### 2. Wire OpenInference + OTLP exporter pointing at Agnost AI
**Already have OpenInference (or any OTel TracerProvider) running?** Append Agnost AI as an additional span processor on the existing provider:
```python Python theme={null}
from opentelemetry import trace
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
trace.get_tracer_provider().add_span_processor(
BatchSpanProcessor(
OTLPSpanExporter(
endpoint="https://otel.agnost.ai/v1/traces",
headers={"X-Agnost-Org-ID": os.environ["AGNOST_ORG_ID"]},
)
)
)
```
```typescript TypeScript theme={null}
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
provider.addSpanProcessor(
new BatchSpanProcessor(
new OTLPTraceExporter({
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
}),
),
);
```
**No OTel yet?** Full setup:
```python Python theme={null}
import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from openinference.instrumentation.openai import OpenAIInstrumentor
provider = TracerProvider(resource=Resource.create({"service.name": "openai-py"}))
provider.add_span_processor(
BatchSpanProcessor(
OTLPSpanExporter(
endpoint="https://otel.agnost.ai/v1/traces",
headers={"X-Agnost-Org-ID": os.environ["AGNOST_ORG_ID"]},
)
)
)
trace.set_tracer_provider(provider)
OpenAIInstrumentor().instrument(tracer_provider=provider)
```
```typescript TypeScript theme={null}
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
import { OpenAIInstrumentation } from '@arizeai/openinference-instrumentation-openai';
const openAIInstrumentation = new OpenAIInstrumentation();
const sdk = new NodeSDK({
traceExporter: new OTLPTraceExporter({
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
}),
instrumentations: [openAIInstrumentation],
});
sdk.start();
// Import openai AFTER sdk.start() and apply the ESM patch.
import OpenAI from 'openai';
openAIInstrumentation.manuallyInstrument(OpenAI as never);
```
### 3. Pass userId / sessionId per call
```python Python theme={null}
from openinference.instrumentation import using_attributes
with using_attributes(user_id="user-42", session_id="conv-abc123"):
client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello"}],
)
```
```typescript TypeScript theme={null}
import { context } from '@opentelemetry/api';
import { setUser, setSession } from '@arizeai/openinference-core';
let ctx = context.active();
ctx = setUser(ctx, { userId: 'user-42' });
ctx = setSession(ctx, { sessionId: 'conv-abc123' });
await context.with(ctx, () =>
client.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'user', content: 'Hello' }],
}),
);
```
`using_attributes` (Python) and `setUser` / `setSession` (TS) propagate via OTel context, landing as `user.id` / `session.id` on every span the OpenAI SDK emits inside the block.
### What appears in Agnost AI
* **Conversations** grouped by `session.id`.
* **User-level analytics** grouped by `user.id`.
* **Events** containing OpenInference spans.
* **Tool calls** when OpenAI tool-use spans are emitted.
### Verify
Run one OpenAI call inside the context block, then open **Events** in Agnost AI. Confirm `user.id`, `session.id`, model, and message attributes are present.
### Troubleshooting
* Import and initialize instrumentation before creating the OpenAI client.
* Confirm the OTLP exporter points at `https://otel.agnost.ai/v1/traces`.
* For TypeScript, confirm the `openai` and `@arizeai/openinference-instrumentation-openai` versions match the table below.
### TypeScript version compatibility
Pin matching majors: version mismatches throw `does not provide an export named 'APIPromise'` at import time:
| `openai` | `@arizeai/openinference-instrumentation-openai` |
| ------------------ | ----------------------------------------------- |
| `^6.7.0` | `^4.0.0` |
| `^4.95.0` – `^5.x` | `~2.3.1` |
### References
* [Enable OpenTelemetry export](https://arize.com/docs/phoenix/tracing/integrations-tracing/openai)
* [Add custom metadata](https://arize.com/docs/phoenix/tracing/how-to-tracing/add-metadata/customize-spans)
## Next steps
* [Conversations](/using-conversations): review the complete OpenAI interaction.
* [Events](/using-events): inspect SDK or OpenTelemetry activity.
* [Intents](/using-intents): organize production conversations by what users wanted.
# OpenAI Agents SDK
Source: https://docs.agnost.ai/openai-agents
Capture traces from the OpenAI Agents SDK over OTLP
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this OpenAI Agents application.
Org ID: your-org-id
Instrument the real Runner call and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash theme={null}
pip install agnost
```
```python theme={null}
import agnost
from agents import Runner
agnost.init("your-org-id")
interaction = agnost.begin(user_id="u-42", agent_name="openai-agent", input=prompt)
try:
result = Runner.run_sync(agent, prompt)
interaction.end(output=str(result.final_output))
except Exception as exc:
interaction.end(output=str(exc), success=False)
raise
finally:
agnost.shutdown()
```
## Manual setup: OpenTelemetry
### Install
```bash theme={null}
pip install openai-agents openinference-instrumentation-openai-agents \
opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```
### Setup
```python theme={null}
import os
from agents import set_tracing_disabled
from openinference.instrumentation.openai_agents import OpenAIAgentsInstrumentor
from openinference.instrumentation import using_attributes
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://otel.agnost.ai"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "X-Agnost-Org-ID="
set_tracing_disabled(True) # turn off OpenAI's hosted trace backend
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(provider)
OpenAIAgentsInstrumentor().instrument(tracer_provider=provider)
with using_attributes(session_id="sess-123", user_id="u-42"):
await Runner.run(agent, "Hello")
```
You get agent / handoff / guardrail spans plus the underlying LLM call spans with full message and tool data (`llm.input_messages.*`, `llm.output_messages.*`, `tool.name`, `tool.parameters`, `llm.token_count.*`).
### Caveats
Without `set_tracing_disabled(True)`, traces also flow to OpenAI's hosted dashboard.
### Verify
Run one `Runner.run` call, then open **Events** in Agnost AI. Confirm agent, handoff, guardrail, and LLM spans are present.
### Troubleshooting
* Call `set_tracing_disabled(True)` if you do not want dual-send to OpenAI's hosted trace backend.
* Confirm `OTEL_EXPORTER_OTLP_HEADERS` contains `X-Agnost-Org-ID=`.
* Confirm `using_attributes` wraps the call you want grouped.
### References
* [Enable OpenTelemetry export](https://openai.github.io/openai-agents-python/tracing/)
* [Add custom metadata](https://openai.github.io/openai-agents-python/tracing/#properties)
## Next steps
* [Conversations](/using-conversations): review the complete agent run.
* [Events](/using-events): inspect model calls, handoffs, guardrails, and tools.
* [Intents](/using-intents): organize production conversations by what users wanted.
# OpenTelemetry (OTel)
Source: https://docs.agnost.ai/otel
Send traces from any OTel-instrumented AI framework directly to Agnost AI
Agnost AI runs a hosted OTLP collector at `otel.agnost.ai`. Any framework that emits OpenTelemetry spans can ship them straight to Agnost AI: no SDK required.
```
Your AI App → otel.agnost.ai → Agnost AI Dashboard
```
## Quick Setup
Set two environment variables before starting your app:
```bash theme={null}
OTEL_EXPORTER_OTLP_ENDPOINT=https://otel.agnost.ai
OTEL_EXPORTER_OTLP_HEADERS="X-Agnost-Org-ID="
```
Get your org ID from [app.agnost.ai](https://app.agnost.ai).
For SDKs that take an explicit URL, use `https://otel.agnost.ai/v1/traces`.
## Framework Guides
* [LiveKit Agents](/livekit)
* [Vercel AI SDK](/vercel-ai)
* [Mastra](/mastra)
* [Spectrum-TS](/spectrum-ts)
* [VoltAgent](/voltagent)
* [OpenAI SDK](/openai)
* [LangChain](/langchain)
## Custom Tool Spans
If your stack doesn't have a framework integration, wrap each agent / tool call in an OTel span. Name the span with a `tool.` prefix so Agnost AI classifies it as a tool call.
### 1. Install
**Already have an OTel SDK and the OTLP exporter installed?** Skip.
**No OTel yet?**
```bash Python theme={null}
pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
```
```bash TypeScript theme={null}
npm install @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-proto @opentelemetry/api
```
### 2. Wire the OTLP exporter pointing at Agnost AI
**Already have an OTel TracerProvider?** Append Agnost AI as an additional span processor:
```python Python theme={null}
from opentelemetry import trace
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
trace.get_tracer_provider().add_span_processor(
BatchSpanProcessor(
OTLPSpanExporter(
endpoint="https://otel.agnost.ai/v1/traces",
headers={"X-Agnost-Org-ID": os.environ["AGNOST_ORG_ID"]},
)
)
)
```
```typescript TypeScript theme={null}
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
provider.addSpanProcessor(
new BatchSpanProcessor(
new OTLPTraceExporter({
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
}),
),
);
```
**No OTel yet?** Full setup:
```python Python theme={null}
import os
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
provider = TracerProvider(resource=Resource.create({"service.name": "my-app"}))
provider.add_span_processor(
BatchSpanProcessor(
OTLPSpanExporter(
endpoint="https://otel.agnost.ai/v1/traces",
headers={"X-Agnost-Org-ID": os.environ["AGNOST_ORG_ID"]},
)
)
)
trace.set_tracer_provider(provider)
```
```typescript TypeScript theme={null}
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
new NodeSDK({
traceExporter: new OTLPTraceExporter({
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
}),
}).start();
```
### 3. Wrap each tool call in a span with userId / sessionId
```python Python theme={null}
tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("tool.search_web") as span:
span.set_attribute("agnost.user_id", "user-42")
span.set_attribute("agnost.session_id", "conv-abc123")
span.set_attribute("input", query)
try:
result = search_web(query)
span.set_attribute("output", str(result))
except Exception as exc:
span.record_exception(exc)
raise
```
```typescript TypeScript theme={null}
import { trace, SpanStatusCode } from '@opentelemetry/api';
const tracer = trace.getTracer('my-app');
const span = tracer.startSpan('tool.search_web');
span.setAttribute('agnost.user_id', 'user-42');
span.setAttribute('agnost.session_id', 'conv-abc123');
span.setAttribute('input', query);
try {
const result = await searchWeb(query);
span.setAttribute('output', JSON.stringify(result));
} catch (err) {
span.setStatus({ code: SpanStatusCode.ERROR });
throw err;
} finally {
span.end();
}
```
## Additional properties
Agnost AI recognizes these OTel fields and preserves non-reserved custom span
attributes as event metadata:
| Type | Fields |
| --------------------------- | ---------------------------------------------------------------------------------------------------------- |
| Parent span | `parentSpanId` on the child span |
| GenAI convention attributes | `gen_ai.request.model`, `gen_ai.response.model`, `gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens` |
| Custom metadata | `customer.tier`, `feature.name`, or another non-reserved OTel attribute |
> **Note:** If you don't set `agnost.session_id` or another supported session
> attribute, Agnost AI uses the trace ID as the session ID. Each trace therefore
> appears as a separate session. Reuse a stable session ID to group multiple
> traces into one conversation.
### Send a trace with curl
This sends one completion span with its input/output and GenAI attributes, plus
a child `tool.get_weather` span with its own input/output:
```bash theme={null}
curl https://otel.agnost.ai/v1/traces \
-X POST \
-H "X-Agnost-Org-ID: $AGNOST_ORG_ID" \
-H "Content-Type: application/json" \
--data-raw '{
"resourceSpans": [{
"resource": {
"attributes": [{
"key": "service.name",
"value": { "stringValue": "support-agent" }
}]
},
"scopeSpans": [{
"scope": { "name": "custom-otel-example" },
"spans": [{
"traceId": "0102030405060708090a0b0c0d0e0f10",
"spanId": "0102030405060708",
"name": "chat.completion",
"startTimeUnixNano": "1700000000000000000",
"endTimeUnixNano": "1700000002000000000",
"attributes": [{
"key": "agnost.user_id",
"value": { "stringValue": "user-42" }
}, {
"key": "agnost.session_id",
"value": { "stringValue": "conversation-abc123" }
}, {
"key": "input",
"value": { "stringValue": "What is the weather in San Francisco?" }
}, {
"key": "output",
"value": { "stringValue": "It is 72°F and sunny." }
}, {
"key": "gen_ai.request.model",
"value": { "stringValue": "gpt-5" }
}, {
"key": "gen_ai.response.model",
"value": { "stringValue": "gpt-5-2025-08-07" }
}, {
"key": "gen_ai.usage.input_tokens",
"value": { "intValue": "120" }
}, {
"key": "gen_ai.usage.output_tokens",
"value": { "intValue": "48" }
}, {
"key": "customer.tier",
"value": { "stringValue": "enterprise" }
}]
}, {
"traceId": "0102030405060708090a0b0c0d0e0f10",
"spanId": "1112131415161718",
"parentSpanId": "0102030405060708",
"name": "tool.get_weather",
"startTimeUnixNano": "1700000000200000000",
"endTimeUnixNano": "1700000001800000000",
"attributes": [{
"key": "agnost.session_id",
"value": { "stringValue": "conversation-abc123" }
}, {
"key": "input",
"value": { "stringValue": "San Francisco" }
}, {
"key": "output",
"value": { "stringValue": "72°F and sunny" }
}, {
"key": "feature.name",
"value": { "stringValue": "weather" }
}]
}]
}]
}]
}'
```
A successful export returns HTTP `200`.
## What appears in Agnost AI
* **Events**: every received span/event.
* **Conversations**: spans grouped by `agnost.session_id`, `session.id`, or framework-specific session metadata.
* **Tool calls**: spans named with `tool.*` or framework tool-call attributes.
* **Errors**: spans marked failed or exceptions recorded on spans.
## Verify
1. Start your app with the OTel exporter configured.
2. Run one agent turn or tool call.
3. Open [app.agnost.ai](https://app.agnost.ai).
4. Check **Events** first, then **Conversations** or **Tool calls**.
## Troubleshooting
* **No data**: confirm the endpoint is `https://otel.agnost.ai/v1/traces` when your SDK expects a trace URL.
* **Wrong org**: confirm `X-Agnost-Org-ID` is attached to the OTLP exporter.
* **No input/output text**: your framework may not export message content by default, or you may have disabled it.
## Next steps
* [Events](/using-events): verify the spans Agnost AI received.
* [Tool calls](/using-tool-calls): inspect tool reliability and latency.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Pydantic AI
Source: https://docs.agnost.ai/pydantic-ai
Capture traces from Pydantic AI by pointing Logfire at Agnost AI
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this Pydantic AI application.
Org ID: your-org-id
Instrument the real agent.run_sync() or agent.run() path and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash theme={null}
pip install agnost
```
```python theme={null}
import agnost
agnost.init("your-org-id")
interaction = agnost.begin(user_id="u-42", agent_name="pydantic-support", input=prompt)
try:
result = agent.run_sync(prompt)
interaction.end(output=str(result.output))
except Exception as exc:
interaction.end(output=str(exc), success=False)
raise
finally:
agnost.shutdown()
```
## Manual setup: OpenTelemetry
Pydantic AI uses Logfire for instrumentation. Point Logfire's OTLP exporter at Agnost AI and disable its hosted backend.
### Install
```bash theme={null}
pip install pydantic-ai logfire
```
### Setup
```python theme={null}
import os, logfire
from pydantic_ai import Agent
from opentelemetry import trace
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://otel.agnost.ai"
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = "X-Agnost-Org-ID="
logfire.configure(send_to_logfire=False) # required: don't dual-send to Logfire
logfire.instrument_pydantic_ai()
agent = Agent("anthropic:claude-sonnet-4-5", system_prompt="...")
tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("chat-turn") as span:
span.set_attribute("gen_ai.conversation.id", "sess-123")
span.set_attribute("user.id", "u-42")
agent.run_sync("Hello")
```
Spans use OTel GenAI semconv (`gen_ai.input.messages`, `gen_ai.output.messages`, `gen_ai.tool.call.arguments`, `gen_ai.usage.*`) plus `pydantic_ai.all_messages`. `gen_ai.conversation.id` is emitted natively when message history is provided to the agent.
### Caveats
Do **not** set `LOGFIRE_TOKEN`: that re-enables the hosted backend.
### Verify
Run one `agent.run_sync` call, then open **Events** in Agnost AI. Confirm `gen_ai.*` and `pydantic_ai.*` attributes are present.
### Troubleshooting
* Keep `send_to_logfire=False` if you do not want dual-send to Logfire.
* Do not set `LOGFIRE_TOKEN` unless you explicitly want hosted Logfire enabled.
* Confirm `OTEL_EXPORTER_OTLP_HEADERS` contains `X-Agnost-Org-ID=`.
### References
* [Enable OpenTelemetry export](https://pydantic.dev/docs/ai/integrations/logfire/#otel-without-logfire)
* [Add custom metadata](https://pydantic.dev/docs/ai/integrations/logfire/#adding-custom-metadata)
## Next steps
* [Conversations](/using-conversations): review the complete Pydantic AI interaction.
* [Events](/using-events): inspect agent, model, and tool activity.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Python
Source: https://docs.agnost.ai/python-conversation
Track any AI interaction with begin() / end()
## Install
```bash pip theme={null}
pip install agnost
```
```bash uv theme={null}
uv add agnost
```
## Integrate
```python theme={null}
import agnost
agnost.init("your-org-id")
```
Get your org ID from [app.agnost.ai](https://app.agnost.ai).
## Track interactions
```python theme={null}
interaction = agnost.begin(user_id="u-123", agent_name="my-agent", input="...")
# ... your AI call ...
interaction.end(output="...")
```
Latency is auto-calculated. For errors:
```python theme={null}
interaction.end(output="Error: timeout", success=False)
```
For simple fire-and-forget cases:
```python theme={null}
agnost.track(user_id="u-123", input="...", output="...", agent_name="my-agent")
```
## Group into conversations
```python theme={null}
import uuid
conversation_id = str(uuid.uuid4())
agnost.track(user_id="u-123", input="Hello", output="Hi!", conversation_id=conversation_id)
agnost.track(user_id="u-123", input="Follow-up", output="Sure!", conversation_id=conversation_id)
```
## Identify users
```python theme={null}
agnost.identify("u-123", {"plan": "pro", "role": "admin"})
```
## Custom properties
```python theme={null}
interaction = agnost.begin(user_id="u-123", agent_name="my-agent", input="...")
# Set one at a time
interaction.set_property("model", "gpt-4")
# Or set many at once
interaction.set_properties({"tokens": 150, "cost": 0.045})
interaction.end(output="...")
```
## Configuration
```python theme={null}
agnost.init("your-org-id", endpoint="https://api.agnost.ai", debug=True)
```
| Parameter | Type | Default | Description |
| ---------- | ------ | ----------------------- | -------------------- |
| `org_id` | `str` | — | Your org ID |
| `endpoint` | `str` | `https://api.agnost.ai` | API endpoint |
| `debug` | `bool` | `False` | Enable debug logging |
## Cleanup
```python theme={null}
agnost.shutdown() # flushes and cleans up
```
## What appears in Agnost AI
* **Conversations** grouped by `conversation_id`.
* **User-level analytics** grouped by `user_id`.
* **Events** for every tracked interaction.
* Unsuccessful interactions remain visible in **Events** with `success=False`.
## Verify
Run one `begin()` / `end()` interaction, then open [app.agnost.ai](https://app.agnost.ai). Check **Events** first, then **Conversations**.
## Troubleshooting
* Confirm `agnost.init("your-org-id")` runs before tracking.
* Call `agnost.flush()` or `agnost.shutdown()` before short-lived scripts exit.
* Confirm the endpoint is `https://api.agnost.ai` unless you intentionally use another endpoint.
## Next steps
* [Conversations](/using-conversations): review how Python interactions are grouped.
* [Events](/using-events): inspect the exact records the SDK sent.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Quickstart
Source: https://docs.agnost.ai/quickstart
Send your first AI interaction to Agnost AI
Use this page to get one tool invocation or AI interaction visible in the [Agnost AI dashboard](https://app.agnost.ai). The Agnost AI skill detects the right integration route and applies the smallest matching change.
## 1. Get your organization ID
1. Sign in at [app.agnost.ai](https://app.agnost.ai).
2. Open **Settings > Organization**. Copy your organization ID: it's a UUID.
In these docs, `org_id` and `AGNOST_ORG_ID` both mean your organization ID. It
routes telemetry to your workspace but does not grant dashboard read access.
## 2. Set up your application
Install the skill in your agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Then ask your coding agent to use it from your app root:
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics.
Org ID: your-org-id
Target app: .
Verification: local app
Entrypoint: the real chat route, agent call, MCP server startup, or tool call
```
The skill asks for missing facts, detects whether your app needs SDK, MCP, or
OpenTelemetry instrumentation, and verifies the resulting setup.
After the edit, restart or deploy your app, trigger one real chat, agent action, or MCP tool call, then confirm the event in the dashboard.
***
## Setup guides
Continue in the **Get started** sidebar and select the framework or application type
you already use. Each AI framework guide keeps all three approaches together:
* **Agnost AI skill** — recommended; it inspects, instruments, and verifies your app.
* **Agnost AI SDK** — manually wrap the framework call you want to analyze.
* **OpenTelemetry** — connect existing framework telemetry to Agnost AI.
Use one approach for a given call path at first. Enabling the SDK and OpenTelemetry
around the same call can create duplicate interactions.
Set up Agnost AI for Agno, Anthropic, CrewAI, DSPy, LangChain, Mastra, OpenAI, and more.
Track custom Python or TypeScript AI interactions, or connect an existing OTel pipeline.
Track tool, resource, and prompt calls from a dedicated MCP server.
Set up MCP Toolbox or Apify telemetry.
***
## Verify
Open [app.agnost.ai](https://app.agnost.ai) and check:
* **Conversations** for SDK-tracked interactions
* **Tool calls** for MCP tool invocations
* **Events** for the unfiltered event stream
Events usually appear within a few seconds. Nothing showing up? Check the org ID, restart/deploy state, and whether the real app path was exercised.
## Next steps
* [Conversations](/using-conversations): review the complete interaction you sent.
* [Events](/using-events): inspect the underlying telemetry and verify its fields.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Retell
Source: https://docs.agnost.ai/retell
Send analyzed Retell calls to Agnost AI
Agnost AI accepts Retell's standard `call_analyzed` webhook and turns the final transcript into a conversation with caller/agent events and nested tool calls. Retell does not expose a documented OTLP export, so use this webhook instead of Retell Live Monitoring or Analytics.
## Configure the webhook
Configure the account-level or agent-level webhook URL in Retell:
```text theme={null}
https://api.agnost.ai/api/voice/v1/webhooks/retell?org_id=
```
Enable the `call_analyzed` event. Agnost acknowledges other Retell lifecycle and transcript-update events without ingesting them.
Get your organization ID from [app.agnost.ai](https://app.agnost.ai). It must be a UUID.
## Data mapping
* Retell `call_id` becomes the Agnost conversation ID.
* `metadata.user_id` or `retell_llm_dynamic_variables.user_id` becomes the user ID when supplied.
* Phone calls fall back to the inbound caller or outbound destination number. Anonymous web calls fall back to `call_id`.
* `agent_name`, then `agent_id`, becomes the agent name.
* `transcript_object` preserves word timestamps. Agnost falls back to the plain `transcript` field.
* `call_analysis`, latency, status, timing, and safe recording/log URLs become conversation metadata.
* Retell's combined cost is converted from cents to USD and attached to one event.
* Tool invocations and results in `transcript_with_tool_calls` become nested tool events.
Agnost never stores Retell's call `access_token`.
Retell recording and log URLs may expire when signed URLs are enabled in Retell. Agnost stores the URL supplied with the webhook; it does not refresh expired Retell URLs.
## Delivery behavior
Successful analyzed calls return `200`. Events that Agnost intentionally ignores, and analyzed calls without a usable transcript, return `204`. Invalid payloads return `400` or `422`; downstream ingestion failures return `502` so Retell can retry.
Repeated successful deliveries map to the same conversation but can duplicate events. Avoid replaying a webhook that already received a `2xx` response.
Retell signs webhooks with `X-Retell-Signature`, but direct signature verification requires storing each organization's Retell webhook key. This initial integration follows the same organization-routing trust model as the existing voice webhook adapters and does not verify that signature. Treat the webhook URL as sensitive until per-organization signature verification is available.
## Next steps
* [Conversations](/using-conversations): review complete Retell calls.
* [Events](/using-events): inspect each caller/agent exchange and tool call.
* [Intents](/using-intents): understand why callers contacted the agent.
# Packages and versions
Source: https://docs.agnost.ai/sdks
Published Agnost AI SDK and server packages
Pick the SDK or integration path based on what you are instrumenting.
| Use case | Language/framework | Page |
| ------------------------------------- | ------------------ | ---------------------------------------------------- |
| AI interactions from application code | Python | [Python setup](/python-conversation) |
| AI interactions from application code | TypeScript/Node | [TypeScript setup](/typescript-conversation) |
| MCP server tool calls | TypeScript/Node | [TypeScript MCP SDK](/typescript-sdk) |
| MCP server tool calls | Python FastMCP | [Python FastMCP](/fastmcp) |
| MCP server tool calls | Go | [Go MCP SDK](/golang-sdk) |
| Existing AI framework traces | OpenTelemetry | [OpenTelemetry](/otel) |
| Query Agnost AI from Claude or Cursor | MCP client | [Agnost AI in Claude and Cursor](/agnost-mcp-server) |
## Package names
| Package | Install |
| --------------------------- | --------------------------------------------- |
| Python conversation SDK | `pip install agnost` |
| TypeScript conversation SDK | `npm install agnostai` |
| Python MCP SDK | `pip install agnost-mcp` |
| TypeScript MCP SDK | `npm install agnost` |
| Go MCP SDK | `go get github.com/agnostai/agnost-go/agnost` |
## First step
If you are not sure which integration to use, start with the [Quickstart](/quickstart).
## Next steps
* [Quickstart](/quickstart): use the Agnost AI skill for guided setup.
* [OpenAI](/openai): see the structure of an agent-framework guide.
* [MCP server analytics](/mcp-overview): instrument an MCP server.
# Security & Trust
Source: https://docs.agnost.ai/security
Security, privacy, and procurement guidance for Agnost AI
Agnost AI is a production telemetry and analytics system for AI agents. That means it can receive prompts, completions, tool calls, metadata, user identifiers, and errors from your application. Treat the integration as a third-party data processor and decide what data should leave your system before enabling production traffic.
## Current security posture
This public page is intentionally conservative. It documents what customers can rely on from these docs without assuming controls that are not stated here.
| Area | Public guidance |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Transport | Use the HTTPS endpoints shown in these docs: `https://api.agnost.ai` and `https://otel.agnost.ai/v1/traces`. |
| Ingestion scope | SDK ingestion uses your organization ID as a routing identifier. It does not grant dashboard read access by itself. |
| Dashboard/API access | Dashboard APIs require a JWT or API key. See [Authentication](/authentication). |
| Sensitive data | Agnost AI does not currently provide automatic PII redaction/DLP before ingestion. Redact or pseudonymize before sending. |
| Regulated data | Do not send HIPAA, PCI, government ID, full job application, resume, children's data, or other regulated data unless your team has an explicit agreement and data-processing plan. |
| Security review | For current security questionnaires, subprocessors, DPA requests, or compliance status, contact [founders@agnost.ai](mailto:founders@agnost.ai). |
## Customer-side controls
Use these controls before production rollout:
* Use stable pseudonymous user IDs instead of raw names, emails, or phone numbers when possible.
* Allowlist metadata keys.
* Redact secrets, API keys, access tokens, passwords, private keys, and auth headers.
* For MCP integrations, set `disable_input` / `disable_output` or `disableInput` / `disableOutput` if tool args or results may contain sensitive data.
* For OpenTelemetry integrations, review which message, prompt, completion, tool parameter, and tool result attributes your framework exports.
* Run one staging trace and inspect the raw event before enabling broad production traffic.
## Procurement notes
If your buyer asks about data governance, the honest answer is:
> Agnost AI can process production conversation data, so customers control what they send. The docs recommend pseudonymization, metadata allowlisting, and redaction before ingestion. Automatic PII redaction before ingestion is not currently documented as a built-in feature.
For formal procurement, request the current security packet from [founders@agnost.ai](mailto:founders@agnost.ai).
## Related pages
* [Data Governance](/data-governance)
* [Authentication](/authentication)
## Next steps
* [Data governance](/data-governance): define the data boundary for your application.
* [Quickstart](/quickstart): configure and verify one interaction.
* [Authentication](/authentication): understand organization routing and API access.
# Spectrum-TS
Source: https://docs.agnost.ai/spectrum-ts
Capture traces from Photon Spectrum-TS with native OpenTelemetry telemetry
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this Spectrum-TS application.
Org ID: your-org-id
Instrument the real message-handling path and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash theme={null}
npm install agnostai
```
Wrap the handler that turns an incoming provider message into the reply sent by
your Spectrum application:
```typescript theme={null}
import * as agnost from 'agnostai';
agnost.init('your-org-id');
const interaction = agnost.begin({
userId: message.userId,
conversationId: message.conversationId,
agentName: 'spectrum-assistant',
input: message.text,
});
try {
const reply = await createReply(message);
interaction.end(reply.text);
return reply;
} catch (error) {
interaction.end(String(error), false);
throw error;
}
```
Call `await agnost.shutdown()` during application shutdown so buffered events flush.
## Manual setup: OpenTelemetry
Spectrum-TS has built-in OpenTelemetry instrumentation. Enable it with `telemetry: true`, then route its OTLP exporter to Agnost AI with standard `OTEL_EXPORTER_OTLP_*` environment variables.
Note: Spectrum telemetry is exported by the `spectrum-ts` package itself, so the app must have `spectrum-ts` installed. The umbrella package includes the official provider packages; install a scoped `@spectrum-ts/*` package only if the app imports that provider directly.
### 1. Install
**Already have `spectrum-ts` installed?** Skip.
```bash theme={null}
npm install spectrum-ts
```
Official provider imports:
| Provider | Umbrella import | Direct package |
| ----------------- | ----------------------------------------- | -------------------------------- |
| Telegram | `spectrum-ts/providers/telegram` | `@spectrum-ts/telegram` |
| Slack | `spectrum-ts/providers/slack` | `@spectrum-ts/slack` |
| WhatsApp Business | `spectrum-ts/providers/whatsapp-business` | `@spectrum-ts/whatsapp-business` |
| iMessage | `spectrum-ts/providers/imessage` | `@spectrum-ts/imessage` |
| Terminal | `spectrum-ts/providers/terminal` | `@spectrum-ts/terminal` |
### 2. Point Spectrum telemetry at Agnost AI
Set these environment variables before starting the Spectrum app. Use your
runtime's environment configuration, a `.env` file, container configuration, or
your deployment platform:
```dotenv theme={null}
AGNOST_ORG_ID=
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://otel.agnost.ai/v1/traces
OTEL_EXPORTER_OTLP_HEADERS="X-Agnost-Org-ID="
```
### 3. Enable Spectrum telemetry
```typescript theme={null}
import { Spectrum } from "spectrum-ts";
import { telegram } from "spectrum-ts/providers/telegram";
const app = await Spectrum({
projectId: process.env.PROJECT_ID!,
projectSecret: process.env.PROJECT_SECRET!,
providers: [
// Telegram is only an example. Any official Spectrum provider works with
// the same telemetry config.
telegram.config({ botToken: process.env.TELEGRAM_BOT_TOKEN! }),
],
telemetry: true,
});
// Optional smoke-test loop only. Spectrum telemetry is enabled above; keep this
// only if you want the sample app to reply to messages.
for await (const [space, message] of app.messages) {
if (message.content.type !== "text") continue;
const output = `Echo: ${message.content.text}`;
await space.send(output);
}
```
`telemetry: true` sends Spectrum's native spans for provider, space, message, content type, and lifecycle metadata. Current Spectrum native spans do not include raw transcript text. If Agnost AI Chat View must render the actual user/bot messages, emit one additional app-level turn span with `agnost.session_id`, `agnost.user_id`, `input`, and `output` from your message loop.
### Local Sample
```bash theme={null}
npm create spectrum-project@latest clank -- --providers telegram
cd clank
```
Set the sample's environment variables using the mechanism appropriate for your
local runtime:
```dotenv theme={null}
PROJECT_ID=
PROJECT_SECRET=
TELEGRAM_BOT_TOKEN=
AGNOST_ORG_ID=
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://otel.agnost.ai/v1/traces
OTEL_EXPORTER_OTLP_HEADERS="X-Agnost-Org-ID="
```
Use the provider you need, for example `telegram`, `slack`, `whatsapp-business`, `imessage`, or `terminal`. Then set `telemetry: true` in the generated `Spectrum(...)` config. Native Spectrum telemetry is enough for Trace View and Metadata. Add a separate turn span only if you need transcript text in Chat View.
### References
* [Spectrum-TS telemetry](https://photon.codes/docs/spectrum-ts/getting-started#telemetry)
* [Spectrum-TS repository](https://github.com/photon-hq/spectrum-ts)
## Next steps
* [Conversations](/using-conversations): review the complete Spectrum interaction.
* [Events](/using-events): inspect provider and message lifecycle activity.
* [Intents](/using-intents): organize production conversations by what users wanted.
# Telnyx
Source: https://docs.agnost.ai/telnyx-voice
Capture Telnyx Voice STT and TTS activity in Agnost AI
Telnyx Voice uses the Agnost AI voice webhook service. No Agnost AI SDK import is required.
## Data Sources
| Telnyx data | Who has it | Agnost AI capture |
| ----------------------- | ----------------------------------- | ---------------------------------------- |
| In-call transcription | Telnyx webhook `call.transcription` | Forward the webhook |
| REST/WebSocket STT text | Your app receives the transcript | Post `kind: "stt"` |
| TTS / speak text | Your app sends the text to Telnyx | Post `kind: "tts"` after Telnyx succeeds |
## Minimal Server Integration
Keep Telnyx pointed at your app. Add one small forwarder for final STT and one for TTS:
```text theme={null}
POST https://api.agnost.ai/api/voice/v1/events/telnyx
X-Org-Id:
```
Forward final transcription events:
```json theme={null}
{
"kind": "stt",
"data": {
"event_type": "call.transcription",
"payload": {
"call_session_id": "call-session-id",
"from": "+15551234567",
"transcription_data": {
"is_final": true,
"transcript": "hello from Telnyx"
}
}
}
}
```
Record TTS where your app already calls Telnyx speak/playback:
```json theme={null}
{
"kind": "tts",
"text": "Sure, I can help with billing.",
"session_id": "call-session-id",
"user_id": "caller-id",
"mode": "in_call_playback"
}
```
## Copy-Paste Middleware
Use one server-side helper. STT can run inside the incoming Telnyx webhook handler.
TTS must run beside the existing Telnyx speak/playback call because that is where your backend has the bot text.
```ts theme={null}
const agnostVoiceUrl = "https://api.agnost.ai";
const agnostOrgId = process.env.AGNOST_ORG_ID;
async function sendAgnostTelnyx(body: Record) {
await fetch(`${agnostVoiceUrl}/api/voice/v1/events/telnyx`, {
method: "POST",
headers: {
"Content-Type": "application/json",
"X-Org-Id": agnostOrgId!,
},
body: JSON.stringify(body),
});
}
```
Forward final STT from your Telnyx webhook:
```ts theme={null}
app.post("/webhooks/telnyx", async (req, res) => {
const payload = req.body;
const transcript = payload?.data?.payload?.transcription_data?.transcript;
const isFinal = payload?.data?.payload?.transcription_data?.is_final !== false;
const sessionId = payload?.data?.payload?.call_session_id;
if (payload?.data?.event_type === "call.transcription" && isFinal && transcript) {
await sendAgnostTelnyx({
kind: "stt",
session_id: sessionId,
event_id: payload.data.id,
user_id: payload.data.payload.from,
agent_name: "telnyx_voice",
text: transcript,
});
}
res.sendStatus(200);
});
```
Forward TTS where your product already speaks to the caller:
```ts theme={null}
const replyText = await agentReply(userText);
await telnyx.calls.actions.speak(callControlId, { payload: replyText, voice: "female" });
await sendAgnostTelnyx({
kind: "tts",
session_id: callSessionId,
event_id: crypto.randomUUID(),
user_id: callerId,
agent_name: "telnyx_voice",
text: replyText,
});
```
## Telnyx SDK Users
Using the Telnyx Node or Python SDK does not replace the STT webhook. Keep `webhook_url`
configured on the call so final `call.transcription` events still reach your backend.
Then wrap the SDK speak call to record TTS.
Node.js:
```ts theme={null}
import Telnyx from "telnyx";
import crypto from "node:crypto";
const telnyx = new Telnyx({ apiKey: process.env.TELNYX_API_KEY });
async function speakWithAgnost({
callControlId,
callSessionId,
callerId,
text,
}: {
callControlId: string;
callSessionId: string;
callerId: string;
text: string;
}) {
const eventId = crypto.randomUUID();
await telnyx.calls.actions.speak(callControlId, {
payload: text,
voice: "female",
command_id: eventId,
});
await sendAgnostTelnyx({
kind: "tts",
session_id: callSessionId,
event_id: eventId,
user_id: callerId,
agent_name: "telnyx_voice",
text,
});
}
```
Python:
```python theme={null}
import os
import uuid
import json
import urllib.request
from telnyx import Telnyx
telnyx = Telnyx(api_key=os.environ.get("TELNYX_API_KEY"))
def send_agnost_telnyx(body: dict) -> None:
request = urllib.request.Request(
"https://api.agnost.ai/api/voice/v1/events/telnyx",
data=json.dumps(body).encode("utf-8"),
headers={
"Content-Type": "application/json",
"X-Org-Id": os.environ["AGNOST_ORG_ID"],
},
method="POST",
)
with urllib.request.urlopen(request, timeout=5):
pass
def speak_with_agnost(call_control_id: str, call_session_id: str, caller_id: str, text: str) -> None:
event_id = str(uuid.uuid4())
telnyx.calls.actions.speak(
call_control_id=call_control_id,
payload=text,
voice="female",
command_id=event_id,
)
send_agnost_telnyx({
"kind": "tts",
"session_id": call_session_id,
"event_id": event_id,
"user_id": caller_id,
"agent_name": "telnyx_voice",
"text": text,
})
```
Use the same `session_id` for STT and TTS events from the same call/session.
It can be the Telnyx `conversation_id`, `call_session_id`, or `call_control_id`; Agnost AI turns non-UUID Telnyx IDs into stable dashboard UUIDs and keeps the original ID in metadata.
Pass a stable `event_id` when your middleware can retry the same STT/TTS event.
The Agnost AI voice service stores final STT in memory for up to 10 minutes until the next TTS event for the same `session_id`.
That TTS write becomes one dashboard event with user speech in `args` and bot speech in `result`.
If a TTS event arrives without a queued STT, or after that queue expires, Agnost AI records it as output-only.
## Troubleshooting
* **Only bot messages appear**: TTS is arriving before final STT, more than 10 minutes after STT, or with a different `session_id`.
* **No user input appears**: you are forwarding partial STT only. Agnost AI ignores STT where `final` or `is_final` is `false`.
* **TTS appears in a separate conversation**: pass the same Telnyx conversation or call ID as `session_id`.
* **The accepted `session_id` looks different from the Telnyx ID**: use the returned `conversation_id` in dashboard/debug links. The original Telnyx ID is stored as metadata.
* **No events appear**: confirm `X-Org-Id` is set and your server can reach the Agnost AI voice service.
* **Missing bot output**: you are forwarding STT only. Post the TTS text after the Telnyx speak/playback request succeeds.
## Telnyx references
* [Telnyx Voice API webhooks](https://developers.telnyx.com/docs/voice/programmable-voice/voice-api-webhooks)
* [Telnyx TTS overview](https://developers.telnyx.com/docs/voice/tts/overview)
* [Telnyx STT models](https://developers.telnyx.com/docs/voice/stt/models)
* [Telnyx Node.js SDK](https://developers.telnyx.com/development/sdk/node/index)
* [Telnyx Python SDK](https://developers.telnyx.com/development/sdk/python)
## Next steps
* [Conversations](/using-conversations): review complete Telnyx calls.
* [Events](/using-events): inspect captured speech and agent responses.
* [Intents](/using-intents): understand what callers wanted.
# Trellus
Source: https://docs.agnost.ai/trellus
Send Trellus call transcripts to Agnost AI
Agnost AI accepts the default Trellus call webhook and turns each call into a conversation with events for each user and agent exchange. Trellus already provides the transcript, so this setup does not run speech-to-text again.
## Configure the webhook
In Trellus, create a webhook with:
```text theme={null}
URL: https://api.agnost.ai/api/voice/v1/webhooks/trellus
Method: POST
Header: X-Org-Id:
Body: Default Trellus payload
```
Get your organization ID from [app.agnost.ai](https://app.agnost.ai). It must be a UUID.
Do not add a custom payload mapping. Agnost AI reads the standard Trellus fields directly.
## Required data
The webhook needs:
* A call ID in `session_id`, `unique_id`, or `custom_id`.
* A prospect identity in `contact_name`, `target_number`, or both.
* Call text in `transcript` or, as a fallback, `summary`.
Agnost AI uses the Trellus call ID as the stable conversation ID. Repeated deliveries for the same call map to the same conversation, but may duplicate its events, so do not manually replay a successful webhook.
## Example payload
```json theme={null}
{
"session_id": "sess_abc123def456",
"duration": 245,
"direction": "outbound",
"call_status": "answered",
"rep_id": "rep_001",
"rep_name": "John Smith",
"rep_number": "+15551234567",
"contact_id": "contact_001",
"contact_name": "Jane Doe",
"target_number": "+15559876543",
"disposition": "Connected",
"sentiment": "Positive",
"summary": "Discussed product features and pricing.",
"audio_url": "https://recordings.example.com/calls/sess_abc.mp3",
"transcript": "Jane Doe: Hi, who is this?\nJohn Smith: Hi Jane, this is John from Acme."
}
```
Trellus speaker labels such as `Rep`, `Contact`, and `Prospect` are mapped automatically. Rep and contact names are also recognized when they appear as transcript labels.
## Response
A successful request returns:
```json theme={null}
{
"status": "accepted",
"session_id": "sess_abc123def456",
"user_id": "jane-doe:+15559876543",
"event_count": 1
}
```
The rep name becomes the Agnost AI agent name. The contact name and target number form a stable user ID, with either field used alone when only one is present.
## What appears in Agnost AI
* One conversation for the Trellus call.
* Conversation events created from user and rep turns.
* Call metadata such as direction, disposition, sentiment, duration, summary, rep, and contact fields.
* Safe `http` or `https` audio, transcript, and platform links when provided.
## Troubleshooting
* **`422` for a missing header**: add `X-Org-Id` to the Trellus webhook.
* **`400` for the organization ID**: confirm `X-Org-Id` is a valid UUID.
* **`400` for identity**: include a call ID and either `contact_name` or `target_number`.
* **`422` for an empty transcript**: include `transcript` or `summary`.
* **The call has no events**: confirm transcript turns use `Speaker: text` lines. Agnost AI recognizes `Rep`, `Contact`, `Prospect`, and the rep/contact names.
## Next steps
* [Conversations](/using-conversations): review complete Trellus calls.
* [Events](/using-events): inspect each caller and rep exchange.
* [Intents](/using-intents): understand what prospects wanted.
# TypeScript
Source: https://docs.agnost.ai/typescript-conversation
Track any AI interaction with begin() / end()
## Install
```bash npm theme={null}
npm install agnostai
```
```bash pnpm theme={null}
pnpm add agnostai
```
```bash yarn theme={null}
yarn add agnostai
```
## Integrate
```typescript theme={null}
import * as agnost from 'agnostai';
agnost.init('your-org-id');
```
Get your org ID from [app.agnost.ai](https://app.agnost.ai).
## Track interactions
```typescript theme={null}
const interaction = agnost.begin({ userId: 'u-123', agentName: 'my-agent', input: '...' });
// ... your AI call ...
interaction.end('...');
```
Latency is auto-calculated. For errors:
```typescript theme={null}
interaction.end('Error: timeout', false);
```
## Group into conversations
```typescript theme={null}
const conversationId = crypto.randomUUID();
const t1 = agnost.begin({ userId: 'u-123', agentName: 'my-agent', input: 'Hello', conversationId });
t1.end('Hi!');
const t2 = agnost.begin({ userId: 'u-123', agentName: 'my-agent', input: 'Follow-up', conversationId });
t2.end('Sure!');
```
## Identify users
```typescript theme={null}
agnost.identify('u-123', { plan: 'pro', role: 'admin' });
```
## Custom properties
```typescript theme={null}
const interaction = agnost.begin({ userId: 'u-123', agentName: 'my-agent', input: '...' });
interaction.setProperties({ model: 'gpt-4', tokens: '150' });
interaction.end('...');
```
## Retrieve an interaction by ID
Useful when you need to complete an interaction from a different function or callback:
```typescript theme={null}
agnost.begin({ userId: 'u-123', agentName: 'my-agent', input: '...', interactionId: 'req-1' });
// elsewhere...
const interaction = agnost.getInteraction('req-1');
interaction?.end('...');
```
## Configuration
```typescript theme={null}
agnost.init('your-org-id', { endpoint: 'https://api.agnost.ai', debug: true });
```
| Option | Type | Default | Description |
| ---------- | --------- | ----------------------- | -------------------- |
| `endpoint` | `string` | `https://api.agnost.ai` | API endpoint |
| `debug` | `boolean` | `false` | Enable debug logging |
## Cleanup
```typescript theme={null}
await agnost.shutdown(); // flushes and cleans up
```
## What appears in Agnost AI
* **Conversations** grouped by `conversationId`.
* **User-level analytics** grouped by `userId`.
* **Events** for every tracked interaction.
* Unsuccessful interactions remain visible in **Events** with `success=false`.
## Verify
Run one `begin()` / `end()` interaction, then open [app.agnost.ai](https://app.agnost.ai). Check **Events** first, then **Conversations**.
## Troubleshooting
* Confirm `agnost.init('your-org-id')` runs before tracking.
* Await `agnost.shutdown()` before short-lived scripts exit.
* Confirm the endpoint is `https://api.agnost.ai` unless you intentionally use another endpoint.
## Next steps
* [Conversations](/using-conversations): review how TypeScript interactions are grouped.
* [Events](/using-events): inspect the exact records the SDK sent.
* [Intents](/using-intents): organize production conversations by what users wanted.
# TypeScript MCP SDK
Source: https://docs.agnost.ai/typescript-sdk
Add Agnost AI analytics to your TypeScript MCP server
## Install
```bash npm theme={null}
npm install agnost
```
```bash pnpm theme={null}
pnpm add agnost
```
```bash yarn theme={null}
yarn add agnost
```
## Integrate
Call `trackMCP` after creating your server, before connecting transport:
```typescript theme={null}
import { trackMCP } from 'agnost';
// your existing server setup...
trackMCP(server, 'your-org-id');
```
Get your org ID from [app.agnost.ai](https://app.agnost.ai).
## Options
```typescript theme={null}
trackMCP(server, 'your-org-id', {
disableInput: false, // set true to skip tracking tool inputs
disableOutput: false, // set true to skip tracking tool outputs
identify: (request, env) => ({
userId: request?.headers?.['x-user-id'] || 'anonymous',
email: request?.headers?.['x-user-email'],
}),
});
```
## Checkpoints
Use `checkpoint()` inside tool handlers to get per-step latency breakdowns:
```typescript theme={null}
import { trackMCP, checkpoint } from 'agnost';
server.setRequestHandler(CallToolRequestSchema, async (request) => {
checkpoint('db_query_start');
const rows = await db.query(/* ... */);
checkpoint('db_query_done', { rowCount: rows.length });
checkpoint('format_start');
const result = format(rows);
checkpoint('format_done');
return result;
});
trackMCP(server, 'your-org-id');
```
Checkpoints appear as a timeline in your dashboard.
## What appears in Agnost AI
* **Tool calls** for MCP tool invocations.
* **Events** for every tracked call.
* **Checkpoint timeline** for calls that use `checkpoint()`.
* Failed tool calls remain visible in **Events** with their error details.
## Verify
Call one MCP tool from your client, then open [app.agnost.ai](https://app.agnost.ai). Check **Events** first, then **Tool calls**.
## Troubleshooting
* Call `trackMCP` after creating your server and before connecting transport.
* Confirm the org ID is correct.
## Next steps
* [Tool calls](/using-tool-calls): analyze calls from your TypeScript server.
* [Events](/using-events): verify call fields and execution details.
* [Alerts](/using-alerts): monitor important MCP tool conditions.
# Alerts
Source: https://docs.agnost.ai/using-alerts
Monitor important production conditions and notify the right destination
An **alert** sends important production signals to email or Slack. Custom alerts
evaluate a saved condition on a schedule. Proactive intent alerts are event-driven:
they run when Agnost discovers a new user intent, not on the scheduled alert poller.
## Slack defaults
Before Slack is connected, the Alerts page shows the available alert examples as
suggestions you can try, including **Daily Summary** and **Proactive Intent Summary**.
Connecting a Slack channel enables those two summaries by default. You can pause
or resume either summary from its alert card.
## Create an alert
Open **Alerts** in the Agnost AI sidebar, then choose a template or create a custom
condition. Configure:
* A descriptive alert name.
* The condition and threshold.
* How often Agnost AI should check it.
* Email recipients or a connected Slack channel.
* The action or investigation context recipients need.
You can also open an intent or violation and choose **Alert**. Agnost AI fills in
the selected classification, keeps the counting logic fixed, and lets you set the
count threshold, lookback window, recent-message count, schedule, and destination.
## Alert design
Prefer conditions that are actionable and stable enough to avoid noise. Include
the workspace, time window, affected intent/tool/rule, supporting count, and a
direct path back to Agnost AI.
Test the alert before enabling it. Revisit thresholds after traffic or agent
behavior changes.
## Next steps
* [Intents](/using-intents): choose a user goal worth monitoring.
* [Violations](/using-violations): choose an expected rule worth monitoring.
* [Improvements](/using-improvements): investigate and address recurring conditions.
# Conversations
Source: https://docs.agnost.ai/using-conversations
Review complete user interactions and understand where an agent succeeded or failed
A **conversation** is the user-facing view of one related sequence of agent
interactions. SDKs may call the same boundary a `conversation_id` or `session_id`.
Agnost AI groups those events into one timeline so you can review the user's goal,
the agent's responses, tool activity, and outcome together.
## Open a conversation
1. Open **User Stories** in the Agnost AI sidebar.
2. Apply the workspace, time-range, and metadata filters you need.
3. Select a row to inspect its messages, spans, errors, and available audio.
The dashboard currently labels this surface **User Stories**. In these docs,
“conversation” is the canonical data term; a user story is the dashboard's
narrative view of that conversation.
## Conversation identity
Reuse one stable conversation identifier across every turn that belongs together.
The integration path determines the field name:
| Integration | Conversation field |
| -------------------------- | ------------------------------------------------------------------ |
| Python direct tracking | `conversation_id` |
| TypeScript direct tracking | `conversationId` |
| Ingestion API | `session_id` |
| OpenTelemetry | `session.id`, `conversation.id`, or a documented framework mapping |
Starting a new identifier on every turn fragments the timeline. Reusing one
identifier across unrelated users combines data that should remain separate.
## What to inspect
* Whether the user’s goal was resolved.
* Repeated questions, corrections, and abandoned flows.
* Failed or unexpectedly slow tool calls.
* The intent and violation evidence attached to the conversation.
* Agent-version, plan, surface, or cohort metadata that explains the outcome.
For the underlying records, continue to [Events](/using-events).
## Next steps
* [Events](/using-events): inspect the records behind a conversation.
* [Intents](/using-intents): group conversations by what users wanted.
* [Violations](/using-violations): find where expected behavior was not followed.
# Events
Source: https://docs.agnost.ai/using-events
Inspect the atomic operations Agnost AI received from SDK and OpenTelemetry instrumentation
An **event** is one captured operation inside a conversation. Depending on the
instrumentation, it may represent an agent turn, model generation, tool call,
handoff, guardrail, or another traced operation.
## Inspect events
Open **Events** in the Agnost AI sidebar to review the unaggregated telemetry stream.
Use it first when verifying a new setup: an event can arrive before higher-level
classification and aggregation finishes.
Confirm that a test event contains:
* The expected workspace and conversation identifier.
* A stable user identifier.
* The correct agent, model, framework, or operation name.
* The expected input and output.
* Success state, latency, timestamp, and useful metadata.
## Events and conversations
```text theme={null}
Conversation
├── Agent turn event
├── Tool call event
└── Agent turn event
```
The ingestion API calls the conversation boundary a session. Create the session
once, then send its events with the same `session_id`. See
[Capture Session](/capture-session) and [Capture Event](/capture-event).
## Troubleshooting
* Missing event: verify the endpoint, organization ID, and org header.
* Fragmented conversation: reuse the same conversation/session identifier.
* Missing text: check framework content-capture settings.
* Duplicate operation: avoid enabling two instrumentation paths around the same
call unless you intentionally want both records.
## Next steps
* [Conversations](/using-conversations): see how related events become one interaction.
* [Tool calls](/using-tool-calls): analyze tool-specific execution details.
* [Ingestion errors](/errors): troubleshoot events that do not arrive as expected.
# Improvements
Source: https://docs.agnost.ai/using-improvements
Review proposed agent changes and follow their delivery and outcome
An **improvement** is a proposed change based on production evidence. **Auto
Improve** is the Agnost AI workflow that investigates a finding, gathers repository
context, prepares a change, and tracks its review and outcome.
## Review improvements
Open **Auto Improve** in the Agnost AI sidebar to inspect:
* The production finding that triggered the work.
* The baseline evidence and affected conversations.
* Investigation and implementation status.
* The repository branch or pull request when one was created.
* Outcome measurements available after deployment.
Treat generated changes like any other engineering contribution: review the diff,
run the repository's tests, confirm security and data boundaries, and deploy
through your normal process.
## Next steps
* [Conversations](/using-conversations): review the production evidence behind a change.
* [Alerts](/using-alerts): continue monitoring after deployment.
* [Events](/using-events): compare production behavior after a change.
# Intents
Source: https://docs.agnost.ai/using-intents
See what users are trying to accomplish and inspect the evidence behind each intent
An **intent** describes what the user was trying to accomplish. Agnost AI analyzes
conversation evidence against the intents configured for the workspace and
aggregates the matches over time.
## Work with intents
Open **Intents** in the Agnost AI sidebar to:
* Review intent volume and activity.
* Search and inspect configured intents.
* Open the conversations that support an intent.
* Create a threshold alert for the selected intent.
* Create, edit, or remove workspace-specific intent definitions.
* Generate suggested intents from the product description and observed traffic.
## Interpret the result
An intent count is a classified match, not proof that the user succeeded. Review
the supporting conversation and pair intent data with violations, outcomes, and
tool activity before prioritizing work.
Use names that describe the user goal, such as “reset a password” or “compare
plans,” rather than internal feature or route names.
## Next steps
* [Conversations](/using-conversations): inspect the evidence behind an intent.
* [Alerts](/using-alerts): monitor important changes in intent volume.
* [Improvements](/using-improvements): turn recurring user needs into product changes.
# Tool calls
Source: https://docs.agnost.ai/using-tool-calls
Analyze tool usage, unsuccessful calls, latency, and execution details
A **tool call** is an event representing the execution of a tool. Agnost AI keeps it
separate from the surrounding agent turn so you can compare tool reliability and
latency without losing conversation context.
## Open tool-call analytics
Open **Tool Calls** in the Agnost AI sidebar. Use the filters to compare tools by
volume, success rate, error distribution, and latency over time.
## What good instrumentation includes
* A stable tool name.
* The parent conversation or agent-turn identifier.
* Success or failure state.
* End-to-end latency.
* Arguments and results when content capture is enabled.
* Checkpoints for meaningful internal phases when the SDK supports them.
## Client-side versus server-side visibility
Application OpenTelemetry records the calls made by that application. Agnost AI MCP
server analytics instruments an MCP server itself and can observe calls from all
of its clients. Choose the boundary that matches the question you need to answer;
avoid recording the same call twice without a deliberate reason.
## Next steps
* [Events](/using-events): inspect the complete record behind a tool call.
* [Alerts](/using-alerts): monitor important tool conditions.
* [Improvements](/using-improvements): turn recurring tool issues into changes.
# Violations
Source: https://docs.agnost.ai/using-violations
Define expected agent behavior and review evidence when it is not followed
An **SOP** is an expected rule for agent behavior. A **violation** means an
analyzed conversation did not follow that rule.
## Configure rules
Open **Violations** in the Agnost AI sidebar. Create rules that are specific enough
to evaluate from a conversation, for example:
* Confirm identity before exposing account information.
* State the cancellation policy before completing cancellation.
* Escalate when the user reports a safety issue.
Avoid rules that depend on information Agnost AI never receives.
## Review violations
For each rule, compare the number of violations with the number of analyzed
conversations. Open **Evidence** to read the supporting messages before treating a
match as a product issue. Choose **Alert** on a rule to monitor when its violation
count crosses a threshold.
A violation is a classified finding, not an enforcement action. Keep critical
safety and authorization checks in application code.
## Next steps
* [Conversations](/using-conversations): inspect the evidence behind a violation.
* [Alerts](/using-alerts): monitor important rules and thresholds.
* [Improvements](/using-improvements): turn repeated violations into agent changes.
# Vapi
Source: https://docs.agnost.ai/vapi
Send Vapi end-of-call reports to Agnost AI
Agnost AI accepts Vapi's standard `end-of-call-report` payload and turns its transcript into a conversation with events for each caller and assistant exchange. Vapi already provides the transcript, so Agnost AI does not run speech-to-text again. Internal Vapi function calls are also captured as nested tool events.
## Configure the webhook
Configure the Vapi assistant or phone number server with:
```text theme={null}
URL: https://api.agnost.ai/api/voice/v1/webhooks/vapi
Method: POST
Header: X-Org-Id:
Server message: end-of-call-report
```
Get your organization ID from [app.agnost.ai](https://app.agnost.ai). It must be a UUID.
If the assistant or phone number already sends events to another server URL, forward the unchanged Vapi request body from that server to Agnost AI. Vapi selects one server URL for each event rather than broadcasting it to every configured URL.
## Required data
The report must contain:
* `message.type` equal to `end-of-call-report`.
* A stable call ID in `message.call.id`.
* The caller number in `message.call.customer.number` or `message.customer.number`.
* Structured turns in `message.artifact.messages`, or a transcript in `message.artifact.transcript` or `message.transcript`.
The call ID becomes the Agnost AI conversation ID, the caller number becomes the user ID, and the Vapi assistant name becomes the agent name. `AI`, `Assistant`, and `Bot` transcript labels map to the agent; `User`, `Customer`, and `Caller` map to the user.
Repeated deliveries map to the same conversation but can duplicate its events. Avoid manually replaying a successful report; retry only reports that received a non-2xx response.
## Internal tool calls
When `message.artifact.messages` or `message.artifact.messagesOpenAIFormatted` contains a Vapi function call, Agnost AI keeps the existing synthetic `TOOL CALL: ` agent event and stores the exact function call beneath it as a child `tool` event. The child records the Vapi tool name, arguments, result, success state, source call ID and type, timestamp, and measured call-to-result latency. Calls from the two Vapi message representations are correlated by tool-call ID rather than captured twice.
Tool arguments and results can contain sensitive application data. Only return fields that are appropriate to retain in your Agnost AI organization; redact credentials and secrets in the Vapi tool implementation before they enter the end-of-call report.
## Response
```json theme={null}
{
"status": "accepted",
"session_id": "01a01158-b8d1-7000-aafb-c3bd72914986",
"user_id": "+15559876543",
"event_count": 3
}
```
The conversation metadata includes available Vapi call status, timing, end reason, cost, summary, structured analysis, assistant and phone-number identifiers, and safe recording or log URLs. Agnost also attaches Vapi's completed total cost to exactly one event so the conversation cost appears without being multiplied by the number of exchanges. The call is timestamped from Vapi's `startedAt` value.
## Troubleshooting
* **`422` for a missing header**: add `X-Org-Id` to the Vapi server configuration or forwarded request.
* **`400` for the organization ID**: confirm `X-Org-Id` is a valid UUID.
* **`400` for the event type**: enable `end-of-call-report` in Vapi's server messages.
* **`400` for identity**: ensure `message.call.id` and the caller number are present.
* **`422` for an empty transcript**: ensure Vapi transcript artifacts are enabled for the call.
## Next steps
* [Conversations](/using-conversations): review complete Vapi calls.
* [Events](/using-events): inspect each caller and assistant exchange.
* [Intents](/using-intents): understand why callers contacted the agent.
# Vercel AI SDK
Source: https://docs.agnost.ai/vercel-ai
Capture traces from the Vercel AI SDK with experimental_telemetry
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this Vercel AI SDK application.
Org ID: your-org-id
Instrument the real generateText() or streamText() path and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash theme={null}
npm install agnostai
```
```typescript theme={null}
import { generateText } from 'ai';
import * as agnost from 'agnostai';
agnost.init('your-org-id');
const interaction = agnost.begin({ userId: 'u-42', agentName: 'vercel-ai-support', input: prompt });
try {
const { text } = await generateText({ model, prompt });
interaction.end(text);
} catch (error) {
interaction.end(String(error), false);
throw error;
} finally {
await agnost.shutdown();
}
```
## Manual setup: OpenTelemetry
The Vercel AI SDK emits `ai.*` spans natively. Wire an OTel exporter pointing at Agnost AI, then enable telemetry per call with `userId` / `sessionId`.
### 1. Install
**Already have `@opentelemetry/sdk-node` (or another OTel SDK) and `@opentelemetry/exporter-trace-otlp-proto`?** Skip: you only need the existing packages.
**No OTel set up yet?**
```bash theme={null}
npm install @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-proto
```
### 2. Wire the OTel exporter pointing at Agnost AI
**Already exporting OTel?** Append Agnost AI as an additional span processor on your existing TracerProvider:
```typescript theme={null}
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
provider.addSpanProcessor(
new BatchSpanProcessor(
new OTLPTraceExporter({
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
}),
),
);
```
**No OTel yet?** Boot a NodeSDK at startup (e.g. Next.js `instrumentation.ts`):
```typescript theme={null}
import { NodeSDK } from '@opentelemetry/sdk-node';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
new NodeSDK({
traceExporter: new OTLPTraceExporter({
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
}),
}).start();
```
### 3. Pass userId / sessionId per call
```typescript theme={null}
import { generateText } from 'ai';
await generateText({
model: openai('gpt-4o'),
prompt: 'Hello',
experimental_telemetry: {
isEnabled: true,
metadata: {
userId: 'user-42',
sessionId: 'conv-abc123',
},
},
});
```
`experimental_telemetry.isEnabled` defaults to `false`: set it per call.
### What appears in Agnost AI
* **Conversations** grouped by `sessionId`.
* **User-level analytics** grouped by `userId`.
* **Events** for every generated span.
### Verify
Run one `generateText` call, then open **Events** in Agnost AI. If the event is present, check **Conversations** for session grouping.
### Troubleshooting
* Confirm `experimental_telemetry.isEnabled` is `true` on the call.
* Confirm the OTLP exporter is initialized before the first AI SDK call.
* Confirm `X-Agnost-Org-ID` is set on the exporter.
### References
* [Vercel AI SDK telemetry](https://ai-sdk.dev/docs/ai-sdk-core/telemetry)
* [Add custom metadata](https://ai-sdk.dev/docs/ai-sdk-core/telemetry#telemetry-metadata)
## Next steps
* [Conversations](/using-conversations): review the complete Vercel AI SDK interaction.
* [Events](/using-events): inspect generated model and tool spans.
* [Intents](/using-intents): organize production conversations by what users wanted.
# VoltAgent
Source: https://docs.agnost.ai/voltagent
Capture traces from VoltAgent agents over OTLP
## Choose a setup method
| Method | Use it when |
| ------------------- | ------------------------------------------------------------------------------ |
| **Agnost AI skill** | You want your coding agent to inspect the app, make the change, and verify it. |
| **Agnost AI SDK** | You want explicit control over the interaction boundaries and fields. |
| **OpenTelemetry** | The framework already emits useful spans or you operate an OTel pipeline. |
Start with one method for a call path. Combining SDK tracking with framework
OpenTelemetry on the same call can create duplicate interactions.
## Recommended: Agnost AI skill
**Recommended.** Choose this path when you want your coding agent to inspect the
project, select a supported transport, make the smallest instrumentation change,
send a test event, and verify that it reached Agnost AI.
Install the skill once in your coding-agent environment:
```bash theme={null}
npx skills add AgnostAI/skills --skill agnost-ai
```
Run the framework-specific prompt below from the application root. Review the
resulting diff before deploying it.
```text theme={null}
Use the agnost-ai skill to add Agnost AI analytics to this VoltAgent application.
Org ID: your-org-id
Instrument the real agent.generateText() path and verify one fresh interaction.
```
## Manual setup: Agnost AI SDK
```bash theme={null}
npm install agnostai
```
```typescript theme={null}
import * as agnost from 'agnostai';
agnost.init('your-org-id');
const interaction = agnost.begin({ userId: 'u-42', agentName: 'voltagent-support', input: prompt });
try {
const result = await agent.generateText(prompt);
interaction.end(result.text);
} catch (error) {
interaction.end(String(error), false);
throw error;
} finally {
await agnost.shutdown();
}
```
## Manual setup: OpenTelemetry
VoltAgent ships its own observability layer. Plug an OTLP exporter into `VoltAgentObservability`, then pass `userId` / `conversationId` per call.
### 1. Install
**Already have `@voltagent/core` and `@opentelemetry/exporter-trace-otlp-proto`?** Skip.
**No setup yet?**
```bash theme={null}
npm install @voltagent/core @opentelemetry/exporter-trace-otlp-proto @opentelemetry/sdk-trace-node
```
### 2. Wire the OTel exporter pointing at Agnost AI
**Already have `VoltAgentObservability`?** Append a span processor for Agnost AI to the existing `spanProcessors` array:
```typescript theme={null}
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-node';
const agnostProcessor = new BatchSpanProcessor(
new OTLPTraceExporter({
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
}),
);
// Inside your existing VoltAgentObservability:
spanProcessors: [
// ...your existing processors,
agnostProcessor,
]
```
**No Observability yet?** Full setup:
```typescript theme={null}
import { VoltAgent, VoltAgentObservability } from '@voltagent/core';
import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-proto';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-node';
const exporter = new OTLPTraceExporter({
url: 'https://otel.agnost.ai/v1/traces',
headers: { 'X-Agnost-Org-ID': process.env.AGNOST_ORG_ID! },
});
const observability = new VoltAgentObservability({
spanProcessors: [new BatchSpanProcessor(exporter)],
});
new VoltAgent({ agents: { agent }, observability });
```
### 3. Pass userId / conversationId per call
```typescript theme={null}
await agent.generateText('Hello', {
userId: 'user-42',
conversationId: 'conv-abc123',
});
```
VoltAgent maps `userId` → `user.id` and `conversationId` → `conversation.id` on every span: both read directly by Agnost AI for user / session grouping.
### What appears in Agnost AI
* **Conversations** grouped by `conversationId`.
* **User-level analytics** grouped by `userId`.
* **Events** for every exported span.
### Verify
Run one `agent.generateText` call, then open **Events** and confirm the span arrived.
### Troubleshooting
* Confirm your `BatchSpanProcessor` is attached to `VoltAgentObservability`.
* Confirm `X-Agnost-Org-ID` is set on the exporter.
* Confirm `userId` and `conversationId` are passed per call.
### References
* [Enable OpenTelemetry export](https://voltagent.dev/docs/observability/mlflow/)
* [Add custom metadata](https://voltagent.dev/observability-docs/setup/#add-metadata-to-traces)
## Next steps
* [Conversations](/using-conversations): review the complete VoltAgent interaction.
* [Events](/using-events): inspect agent, model, and tool activity.
* [Intents](/using-intents): organize production conversations by what users wanted.