gilles/claude-code-best-practice
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

report added claude cli vs agent sdk

Browse Source
This commit is contained in:
Shayan Rais
2026-02-03 20:10:35 +05:00
parent 9c483e21d0
commit 57937f03b6
2 changed files with 471 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
!/sdk-vs-cli-diagram.svg
+138
View File
@@ -0,0 +1,138 @@
<svg viewBox="0 0 900 580" xmlns="http://www.w3.org/2000/svg" style="font-family: system-ui, -apple-system, sans-serif;">
<defs>
<!-- Gradients -->
<linearGradient id="cliGradient" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" style="stop-color:#667eea"/>
<stop offset="100%" style="stop-color:#764ba2"/>
</linearGradient>
<linearGradient id="sdkGradient" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" style="stop-color:#34d399"/>
<stop offset="100%" style="stop-color:#059669"/>
</linearGradient>
<linearGradient id="questionGradient" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" style="stop-color:#4facfe"/>
<stop offset="100%" style="stop-color:#00f2fe"/>
</linearGradient>
<linearGradient id="claudeGradient" x1="0%" y1="0%" x2="100%" y2="100%">
<stop offset="0%" style="stop-color:#d4a574"/>
<stop offset="100%" style="stop-color:#c4956a"/>
</linearGradient>
<!-- Drop Shadow -->
<filter id="shadow" x="-20%" y="-20%" width="140%" height="140%">
<feDropShadow dx="2" dy="4" stdDeviation="4" flood-opacity="0.15"/>
</filter>
<!-- Glow effect for hover -->
<filter id="glow">
<feGaussianBlur stdDeviation="3" result="coloredBlur"/>
<feMerge>
<feMergeNode in="coloredBlur"/>
<feMergeNode in="SourceGraphic"/>
</feMerge>
</filter>
<!-- Arrow markers -->
<marker id="arrowheadCli" markerWidth="10" markerHeight="7" refX="9" refY="3.5" orient="auto">
<polygon points="0 0, 10 3.5, 0 7" fill="#667eea"/>
</marker>
<marker id="arrowheadSdk" markerWidth="10" markerHeight="7" refX="9" refY="3.5" orient="auto">
<polygon points="0 0, 10 3.5, 0 7" fill="#059669"/>
</marker>
</defs>
<!-- Background -->
<rect width="900" height="580" fill="#f8fafc" rx="12"/>
<!-- Title -->
<text x="450" y="35" text-anchor="middle" font-size="18" font-weight="700" fill="#1e293b">Same Question, Different Paths: Why Outputs May Vary</text>
<!-- THE QUESTION - Central element at top -->
<g filter="url(#shadow)">
<rect x="275" y="55" width="350" height="70" rx="12" fill="url(#questionGradient)"/>
<text x="450" y="82" text-anchor="middle" font-size="11" fill="#fff" font-weight="600">USER QUESTION</text>
<text x="450" y="105" text-anchor="middle" font-size="15" fill="#fff" font-weight="700">"What is the capital of Norway?"</text>
</g>
<!-- Split arrows from question -->
<path d="M 370 125 L 370 145 L 200 145 L 200 165" stroke="#667eea" stroke-width="3" fill="none" marker-end="url(#arrowheadCli)"/>
<path d="M 530 125 L 530 145 L 700 145 L 700 165" stroke="#059669" stroke-width="3" fill="none" marker-end="url(#arrowheadSdk)"/>
<!-- CLI PATH (Left Side) -->
<g filter="url(#shadow)">
<rect x="50" y="170" width="300" height="50" rx="10" fill="url(#cliGradient)"/>
<text x="200" y="200" text-anchor="middle" font-size="14" fill="#fff" font-weight="700">Claude CLI (Claude Code)</text>
</g>
<!-- CLI System Prompt Box -->
<g filter="url(#shadow)">
<rect x="50" y="235" width="300" height="145" rx="10" fill="#fff" stroke="#667eea" stroke-width="2"/>
<text x="200" y="258" text-anchor="middle" font-size="12" fill="#667eea" font-weight="700">SYSTEM PROMPT (modular, ~269+ base)</text>
<line x1="70" y1="270" x2="330" y2="270" stroke="#e2e8f0" stroke-width="1"/>
<text x="75" y="290" font-size="10" fill="#475569">+ 18+ Tool Instructions (Write, Read, Bash...)</text>
<text x="75" y="308" font-size="10" fill="#475569">+ Coding Guidelines &amp; Style Rules</text>
<text x="75" y="326" font-size="10" fill="#475569">+ Security &amp; Injection Defense</text>
<text x="75" y="344" font-size="10" fill="#475569">+ CLAUDE.md Project Context</text>
<text x="75" y="362" font-size="10" fill="#475569">+ Environment (OS, Git, Directory)</text>
</g>
<!-- SDK PATH (Right Side) -->
<g filter="url(#shadow)">
<rect x="550" y="170" width="300" height="50" rx="10" fill="url(#sdkGradient)"/>
<text x="700" y="200" text-anchor="middle" font-size="14" fill="#fff" font-weight="700">Claude Agent SDK</text>
</g>
<!-- SDK System Prompt Box -->
<g filter="url(#shadow)">
<rect x="550" y="235" width="300" height="145" rx="10" fill="#fff" stroke="#059669" stroke-width="2"/>
<text x="700" y="258" text-anchor="middle" font-size="12" fill="#059669" font-weight="700">SYSTEM PROMPT (minimal default)</text>
<line x1="570" y1="270" x2="830" y2="270" stroke="#e2e8f0" stroke-width="1"/>
<text x="575" y="290" font-size="10" fill="#475569">+ Minimal Tool Instructions</text>
<text x="575" y="308" font-size="10" fill="#475569">+ Basic Safety Rules</text>
<text x="575" y="326" font-size="10" fill="#94a3b8" font-style="italic">− No Coding Guidelines</text>
<text x="575" y="344" font-size="10" fill="#94a3b8" font-style="italic">− No Project Context</text>
<text x="575" y="362" font-size="10" fill="#94a3b8" font-style="italic">− No Environment Info</text>
</g>
<!-- Arrows to Claude -->
<path d="M 200 380 L 200 400 L 450 400 L 450 420" stroke="#667eea" stroke-width="2" fill="none" marker-end="url(#arrowheadCli)"/>
<path d="M 700 380 L 700 400 L 450 400 L 450 420" stroke="#059669" stroke-width="2" fill="none" marker-end="url(#arrowheadSdk)"/>
<!-- Claude Model (Center) -->
<g filter="url(#shadow)">
<rect x="325" y="420" width="250" height="55" rx="12" fill="url(#claudeGradient)"/>
<text x="450" y="445" text-anchor="middle" font-size="11" fill="#fff" font-weight="600">CLAUDE MODEL</text>
<text x="450" y="463" text-anchor="middle" font-size="13" fill="#fff" font-weight="700">Claude Opus 4.5</text>
</g>
<!-- Warning/Inconsistency Box at bottom -->
<g filter="url(#shadow)">
<rect x="125" y="495" width="650" height="75" rx="12" fill="#fef2f2" stroke="#ef4444" stroke-width="2"/>
<text x="450" y="518" text-anchor="middle" font-size="13" fill="#dc2626" font-weight="700">⚠ NO GUARANTEE OF IDENTICAL OUTPUT</text>
<text x="450" y="538" text-anchor="middle" font-size="10" fill="#7f1d1d">Even with matching prompts + temperature=0, outputs may differ due to:</text>
<text x="450" y="556" text-anchor="middle" font-size="10" fill="#991b1b" font-weight="600">No Seed Parameter · MoE Routing · Floating-Point Variance · Infrastructure Non-Determinism</text>
</g>
<!-- Token comparison badges -->
<g>
<rect x="75" y="385" width="110" height="22" rx="11" fill="#667eea"/>
<text x="130" y="400" text-anchor="middle" font-size="9" fill="#fff" font-weight="600">~269+ base tokens</text>
</g>
<g>
<rect x="715" y="385" width="100" height="22" rx="11" fill="#059669"/>
<text x="765" y="400" text-anchor="middle" font-size="9" fill="#fff" font-weight="600">minimal default</text>
</g>
<!-- VS indicator -->
<circle cx="450" cy="305" r="22" fill="#1e293b"/>
<text x="450" y="310" text-anchor="middle" font-size="12" fill="#fff" font-weight="700">VS</text>
<!-- Legend -->
<g transform="translate(50, 495)">
<text x="0" y="0" font-size="9" fill="#64748b" font-weight="600">LEGEND:</text>
<rect x="0" y="8" width="12" height="12" rx="2" fill="url(#cliGradient)"/>
<text x="16" y="18" font-size="9" fill="#64748b">CLI Path</text>
<rect x="55" y="8" width="12" height="12" rx="2" fill="url(#sdkGradient)"/>
<text x="71" y="18" font-size="9" fill="#64748b">SDK Path</text>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 7.3 KiB

reports/claude-agent-sdk-vs-cli-system-prompts.md
+333
View File
@@ -0,0 +1,333 @@
# Claude Agent SDK vs Claude CLI: System Prompts and Output Consistency
![SDK vs CLI System Prompts Diagram](/!/sdk-vs-cli-diagram.svg)
---
## Executive Summary
When sending the same message (e.g., "What is the capital of Norway?") through the **Claude Agent SDK** versus the **Claude CLI (Claude Code)**, the system prompts accompanying these messages are fundamentally different. The CLI uses a **modular system prompt architecture** (~269 base tokens with additional context conditionally loaded based on features), while the SDK uses a minimal prompt by default. **There is no guarantee of identical output between the two**, even with matching configurations, due to the absence of a seed parameter and inherent non-determinism in Claude's architecture.
---
## 1. System Prompt Comparison
### Claude CLI (Claude Code)
The Claude CLI uses a **modular system prompt architecture** with a ~269-token base prompt, with additional context conditionally loaded:
| Component | Description | Loading |
|-----------|-------------|---------|
| **Base System Prompt** | Core instructions and behavior | Always (~269 tokens) |
| **Tool Instructions** | 18+ builtin tools (Write, Read, Edit, Bash, TodoWrite, etc.) | Always |
| **Coding Guidelines** | Code style, formatting rules, security practices | Always |
| **Safety Rules** | Refusal rules, injection defense, harm prevention | Always |
| **Response Style** | Tone, verbosity, explanation depth, emoji usage | Always |
| **Environment Context** | Working directory, git status, platform info | Always |
| **Project Context** | CLAUDE.md content, settings, hooks configuration | Conditional |
| **Subagent Prompts** | Plan mode, Explore agent, Task agent | Conditional |
| **Security Review** | Extended security instructions (~2,610 tokens) | Conditional |
**Key Characteristics:**
- **Modular architecture** with 110+ system prompt strings loaded conditionally
- Base prompt is modest (~269 tokens), total varies by features activated
- Includes extensive security and injection defense layers
- Automatically loads CLAUDE.md files in the working directory
- Session-persistent context in interactive mode
### Claude Agent SDK
The Agent SDK uses a **minimal system prompt by default** containing:
| Component | Description | Token Impact |
|-----------|-------------|--------------|
| **Essential Tool Instructions** | Only tools explicitly provided | Minimal |
| **Basic Safety** | Minimal safety instructions | Minimal |
**Key Characteristics:**
- No coding guidelines or style preferences by default
- No project context unless explicitly configured
- No extensive tool descriptions
- Requires explicit configuration to match CLI behavior
---
## 2. What Each Interface Sends
### Example: "What is the capital of Norway?"
#### Via Claude CLI
```
System Prompt: [modular, ~269+ base tokens]
├── Base system prompt (~269 tokens)
├── Tool instructions (Write, Read, Edit, Bash, Grep, Glob, etc.)
├── Git safety protocols
├── Code reference guidelines
├── Professional objectivity instructions
├── Security and injection defense rules
├── Environment context (OS, directory, date)
├── CLAUDE.md content (if present) [conditional]
├── MCP tool descriptions (if configured) [conditional]
├── Plan/Explore mode prompts [conditional]
└── Session/conversation context
User Message: "What is the capital of Norway?"
```
#### Via Claude Agent SDK (Default)
```
System Prompt: [minimal]
├── Essential tool instructions (if any tools provided)
└── Basic operational context
User Message: "What is the capital of Norway?"
```
#### Via Agent SDK (with `claude_code` preset)
```typescript
const response = await query({
prompt: "What is the capital of Norway?",
options: {
systemPrompt: {
type: "preset",
preset: "claude_code"
}
}
});
```
```
System Prompt: [modular, matches CLI]
├── Full Claude Code system prompt
├── Tool instructions
├── Coding guidelines
└── Safety rules
// NOTE: Still does NOT include CLAUDE.md unless settingSources is configured
```
---
## 3. Customization Methods
### Claude CLI Customization
| Method | Command | Effect |
|--------|---------|--------|
| **Append to prompt** | `claude -p "..." --append-system-prompt "..."` | Adds instructions while preserving defaults |
| **Replace prompt** | `claude -p "..." --system-prompt "..."` | Completely replaces the system prompt |
| **Project context** | CLAUDE.md file | Automatically loaded, persistent |
| **Output styles** | `/output-style [name]` | Apply predefined response styles |
### Agent SDK Customization
| Method | Configuration | Effect |
|--------|---------------|--------|
| **Custom prompt** | `systemPrompt: "..."` | Replaces default entirely (loses tools) |
| **Preset with append** | `systemPrompt: { type: "preset", preset: "claude_code", append: "..." }` | Preserves CLI functionality + custom instructions |
| **CLAUDE.md loading** | `settingSources: ["project"]` | Loads project-level instructions |
| **Output styles** | `settingSources: ["user"]` or `settingSources: ["project"]` | Loads saved output styles |
### Configuration Comparison Table
| Feature | CLI Default | SDK Default | SDK with Preset |
|---------|-------------|-------------|-----------------|
| Tool instructions | ✅ Full | ❌ Minimal | ✅ Full |
| Coding guidelines | ✅ Yes | ❌ No | ✅ Yes |
| Safety rules | ✅ Yes | ❌ Basic | ✅ Yes |
| CLAUDE.md auto-load | ✅ Yes | ❌ No | ❌ No* |
| Project context | ✅ Automatic | ❌ No | ❌ No* |
*Requires explicit `settingSources: ["project"]` configuration
---
## 4. Output Consistency Guarantees
### Critical Finding: NO Determinism Guaranteed
**The Claude Messages API does not provide a seed parameter for reproducibility.** This is a fundamental architectural limitation.
### Factors Preventing Identical Output
| Factor | Description | Controllable? |
|--------|-------------|---------------|
| **Different system prompts** | CLI vs SDK have different defaults | ✅ Yes (with configuration) |
| **Floating-point arithmetic** | Parallel hardware quirks | ❌ No |
| **MoE routing** | Mixture-of-Experts architecture variations | ❌ No |
| **Batching/scheduling** | Cloud infrastructure differences | ❌ No |
| **Numeric precision** | Inference engine variations | ❌ No |
| **Model snapshots** | Version updates/changes | ❌ No |
### Temperature and Sampling
Even with `temperature=0.0` (greedy decoding):
- Full determinism is **NOT guaranteed**
- Minor variations can still occur due to infrastructure factors
- Known bug: [Claude CLI produces non-deterministic output for identical inputs](https://github.com/anthropics/claude-code/issues/3370)
---
## 5. Achieving Maximum Consistency
To get the **closest possible** identical outputs between SDK and CLI:
### Agent SDK Configuration
```typescript
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
// Option 1: Use claude_code preset
const response = await client.messages.create({
model: "claude-sonnet-4-20250514",
max_tokens: 1024,
// Match CLI system prompt as closely as possible
system: "Your exact system prompt matching CLI",
messages: [
{ role: "user", content: "What is the capital of Norway?" }
],
// Use greedy decoding for maximum consistency
temperature: 0
});
// Option 2: With Agent SDK query function
import { query } from "@anthropic-ai/agent-sdk";
for await (const message of query({
prompt: "What is the capital of Norway?",
options: {
systemPrompt: {
type: "preset",
preset: "claude_code"
},
temperature: 0,
model: "claude-sonnet-4-20250514",
// Load project context like CLI does
settingSources: ["project"]
}
})) {
// Process response
}
```
### CLI Configuration
```bash
# Match the SDK configuration as closely as possible
claude -p "What is the capital of Norway?" \
--model claude-sonnet-4-20250514 \
--temperature 0
```
### Still Not Guaranteed
Even with perfectly matching configurations:
- Output may differ between runs
- Output may differ between SDK and CLI
- No seed parameter exists to force reproducibility
---
## 6. Practical Implications
### When to Use Each Interface
| Use Case | Recommended Interface | Reason |
|----------|----------------------|--------|
| Interactive development | Claude CLI | Full tool suite, project context |
| Programmatic integration | Agent SDK | Fine-grained control, embedding |
| Consistent API responses | Agent SDK + custom prompt | More control over system prompt |
| Batch processing | Agent SDK | Better for automation pipelines |
| One-off tasks | Claude CLI | Faster setup, immediate context |
### Design Recommendations
1. **Don't rely on bit-perfect reproducibility**
- Build applications robust to minor output variations
- Use structured outputs and validation
2. **For production pipelines requiring consistency:**
- Cache results when possible
- Use structured outputs with JSON schema validation
- Combine with deterministic logic and validation
- Consider multiple generations with consensus
3. **For matching CLI behavior in SDK:**
```typescript
systemPrompt: {
type: "preset",
preset: "claude_code",
append: "Your additional instructions"
},
settingSources: ["project", "user"]
```
---
## 7. System Prompt Token Impact
| Configuration | Architecture | Notes |
|---------------|-------------|-------|
| SDK (minimal) | Minimal default | Only essential tool instructions |
| SDK (claude_code preset) | Modular (~269+ base) | Matches CLI, varies by features |
| CLI (default) | Modular (~269+ base) | Additional context loaded conditionally |
| CLI (with MCP tools) | Modular + MCP | MCP tool descriptions add significant tokens |
**Note:** Claude Code uses a modular architecture with 110+ system prompt strings. The base prompt is ~269 tokens, with individual components ranging from 18 to 2,610 tokens depending on features activated.
**Implication:** The SDK's minimal default gives you more context for your actual task, but at the cost of Claude Code's full capabilities.
---
## 8. Summary Table
| Aspect | Claude CLI | Agent SDK (Default) | Agent SDK (Preset) |
|--------|------------|--------------------|--------------------|
| **System prompt** | Modular (~269+ base) | Minimal | Modular (matches CLI) |
| **Tools included** | 18+ builtin | Only if provided | 18+ builtin |
| **CLAUDE.md auto-load** | Yes | No | No (needs config) |
| **Coding guidelines** | Yes | No | Yes |
| **Safety rules** | Full | Basic | Full |
| **Temperature control** | Yes | Yes | Yes |
| **Determinism guarantee** | No | No | No |
| **Identical outputs?** | N/A | No (vs CLI) | Closer, but no |
---
## 9. Conclusion
**Q: What system prompts accompany the same message in SDK vs CLI?**
The CLI uses a **modular system prompt architecture** with a ~269-token base prompt and 110+ conditionally-loaded components (tool instructions, coding guidelines, safety rules, project context). The SDK uses a **minimal default** with only essential tool instructions, though it can be configured to match CLI behavior using the `claude_code` preset.
**Q: Is there a guarantee of identical output?**
**No.** Even with matching system prompts, identical inputs, and `temperature=0`, there is no guarantee of identical outputs due to:
- Absence of a seed parameter in Claude's API
- Floating-point arithmetic variations
- Infrastructure-level non-determinism
- Model architecture (Mixture-of-Experts) routing variations
**Recommendation:** Design systems to be robust to output variations rather than relying on deterministic behavior. For consistency-critical applications, use structured outputs, caching, and validation layers.
---
## Sources
- [Modifying System Prompts - Agent SDK](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/sdk#modifying-system-prompts)
- [Claude Code CLI Reference](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/cli)
- [Claude Code Headless Mode](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/headless)
- [Claude Code Best Practices - Anthropic Engineering](https://www.anthropic.com/engineering/claude-code-best-practices)
- [Claude Messages API Reference](https://docs.anthropic.com/en/api/messages)
- [GitHub Issue #3370: Non-deterministic output](https://github.com/anthropics/claude-code/issues/3370)
- [Claude Code System Prompts Repository](https://github.com/Piebald-AI/claude-code-system-prompts) - Analysis of modular prompt architecture
- [Why Deterministic Output from LLMs is Nearly Impossible](https://unstract.com/blog/understanding-why-deterministic-output-from-llms-is-nearly-impossible/)
---
*This report was generated by Claude Code using the Opus 4.5 model on February 3, 2026.*