## The Problem with LLM API Keys in the Enterprise
When your AI agents call OpenAI, Anthropic, or your internal model endpoints, they authenticate with API keys — and those keys are kingdom-level credentials. A single leaked key can run up $50,000 in compute charges overnight or exfiltrate sensitive company data through a rogue model prompt.
## The Okta-First Approach: Machine Identity for AI Agents
Instead of embedding API keys in environment variables or secrets managers alone, we treat every AI agent as a machine identity in Okta.
### Step 1: Create a Service Account per Agent
``
bash
# Using Okta CLI
okta apps create service-app \
--name "agentic-remediation-prod" \
--grant-type client_credentials \
--scopes "llm:infer"
`
Each agent receives a short-lived OAuth 2.0 token scoped to only the AI endpoints it needs — never a permanent key.
### Step 2: Rotate Credentials Automatically
typescript
// lib/okta-machine-token.ts
export async function getAgentToken(agentId: string): Promise {
const response = await fetch( https://${process.env.OKTA_DOMAIN}/oauth2/v1/token, {
method: "POST",
headers: { "Content-Type": "application/x-www-form-urlencoded" },
body: new URLSearchParams({
grant_type: "client_credentials",
client_id: process.env[OKTA_CLIENT_ID_${agentId.toUpperCase()}]!,
client_secret: process.env[OKTA_CLIENT_SECRET_${agentId.toUpperCase()}]!,
scope: "llm:infer",
}),
});
const data = await response.json();
return data.access_token; // expires in 15 minutes
}
`
## Datadog: Detecting Anomalous LLM Usage
Even with Okta protecting the front door, you need visibility into what's happening inside the model. Datadog's LLM Observability traces every inference call.
### Configuring LLM Tracing
python
from ddtrace.llmobs import LLMObs
LLMObs.enable(ml_app="sabalynx-agents", agentless_enabled=True)
# Now every LangChain/OpenAI call is automatically traced
import openai
client = openai.OpenAI() # traces injected automatically
`
### Anomaly Detection Rules
Set up these monitors in Datadog:
1. Token spend spike — Alert if any single agent spends >10x its hourly baseline
2. Prompt injection pattern — Alert on inputs containing patterns
3. Unusual destination — Alert if an agent calls an LLM endpoint not in the approved allowlist
## The Combined Architecture
`
AI Agent Request
→ Okta mTLS verification (is this a registered machine identity?)
→ Okta token scoping (does this agent have llm:infer permission?)
→ LLM API Gateway (rate limiting, input sanitization)
→ Datadog LLM Obs (trace, monitor, alert)
→ Model endpoint
This pattern ensures that if an AI agent is compromised, you get a security event in Okta Identity Threat Protection within seconds, and Datadog's anomaly monitor pages your on-call engineer simultaneously.
## Key Takeaways
- Never use static API keys for AI agents — use Okta machine identities with short-lived tokens
- Scope every agent to the minimum LLM permissions required
- Enable Datadog LLM Observability from day one — retrofitting tracing is painful
- Set anomaly monitors on token spend *before* your agents go to production