diff --git a/reports/claude-in-chrome-v-chrome-devtools-mcp.md b/reports/claude-in-chrome-v-chrome-devtools-mcp.md new file mode 100644 index 0000000..24a34ed --- /dev/null +++ b/reports/claude-in-chrome-v-chrome-devtools-mcp.md @@ -0,0 +1,338 @@ +# Comprehensive Browser Automation MCP Comparison Report + +## Executive Summary + +Based on extensive research, I've analyzed the two tools from your screenshots plus a third major contender. Here's my comprehensive breakdown to help you choose the best option for automated testing of your work. + +--- + +## 1. The Three Contenders + +### **A. Chrome DevTools MCP** (Your Screenshot #1) +- **Source:** Official Google Chrome team +- **Released:** Public preview September 2025 +- **Architecture:** Built on Chrome DevTools Protocol (CDP) + Puppeteer +- **Token Usage:** ~19.0k tokens (9.5% of context) +- **Tools:** 26 specialized tools across 6 categories + +### **B. Claude in Chrome** (Your Screenshot #2) +- **Source:** Official Anthropic extension +- **Released:** Beta, rolling out to all paid plans (Pro, Max, Team, Enterprise) +- **Architecture:** Browser extension with computer-use capabilities +- **Token Usage:** ~15.4k tokens (7.7% of context) +- **Tools:** 16 tools including computer use capabilities + +### **C. Playwright MCP** (Strong Alternative) +- **Source:** Microsoft (official + community implementations) +- **Architecture:** Accessibility tree-based automation +- **Token Usage:** ~13.7k tokens (6.8% of context) +- **Tools:** 21 tools + +--- + +## 2. Detailed Feature Comparison + +| Feature | Chrome DevTools MCP | Claude in Chrome | Playwright MCP | +|---------|---------------------|------------------|----------------| +| **Primary Purpose** | Debugging & Performance | General browser automation | UI Testing & E2E | +| **Browser Support** | Chrome only | Chrome only | Chromium, Firefox, WebKit | +| **Token Efficiency** | 19.0k (9.5%) | 15.4k (7.7%) | 13.7k (6.8%) | +| **Element Selection** | CSS/XPath selectors | Visual + DOM | Accessibility tree (semantic) | +| **Performance Traces** | ✅ Excellent | ❌ No | ⚠️ Limited | +| **Network Inspection** | ✅ Deep analysis | ⚠️ Basic | ⚠️ Basic | +| **Console Logs** | ✅ Full access | ✅ Full access | ⚠️ Limited | +| **Cross-browser** | ❌ No | ❌ No | ✅ Yes | +| **CI/CD Integration** | ✅ Excellent | ❌ Poor (requires login) | ✅ Excellent | +| **Headless Mode** | ✅ Yes | ❌ No | ✅ Yes | +| **Authentication** | Requires setup | Uses your session | Requires setup | +| **Scheduled Tasks** | ❌ No | ✅ Yes | ❌ No | +| **Cost** | Free | Requires paid plan | Free | +| **Local Setup** | Node.js required | Browser extension | Node.js required | + +--- + +## 3. Tool Breakdown + +### Chrome DevTools MCP (26 Tools) + +``` +INPUT AUTOMATION (8): click, drag, fill, fill_form, handle_dialog, + hover, press_key, upload_file + +NAVIGATION (6): close_page, list_pages, navigate_page, + new_page, select_page, wait_for + +EMULATION (2): emulate, resize_page + +PERFORMANCE (3): performance_analyze_insight, + performance_start_trace, performance_stop_trace + +NETWORK (2): get_network_request, list_network_requests + +DEBUGGING (5): evaluate_script, get_console_message, + list_console_messages, take_screenshot, + take_snapshot +``` + +### Claude in Chrome (16 Tools) + +``` +BROWSER CONTROL: navigate, read_page, find, computer + (click, type, scroll) + +FORM INTERACTION: form_input, javascript_tool + +MEDIA: upload_image, get_page_text, gif_creator + +TAB MANAGEMENT: tabs_context_mcp, tabs_create_mcp + +DEVELOPMENT: read_console_messages, read_network_requests + +UTILITIES: shortcuts_list, shortcuts_execute, + resize_window, update_plan +``` + +### Playwright MCP (21 Tools) + +``` +NAVIGATION: navigate, goBack, goForward, reload + +INTERACTION: click, fill, select, hover, press, + drag, uploadFile + +ELEMENT QUERIES: getElement, getElements, waitForSelector + +ASSERTIONS: assertVisible, assertText, assertTitle + +PAGE STATE: screenshot, getAccessibilityTree, + evaluateScript + +BROWSER MGMT: newPage, closePage +``` + +--- + +## 4. Use Case Analysis for Automated Testing + +### **Chrome DevTools MCP is BEST for:** + +✅ **Performance Testing** +- Recording performance traces with Core Web Vitals +- Identifying render bottlenecks and layout shifts +- Memory leak detection and CPU profiling + +✅ **Deep Debugging** +- Network request inspection (headers, payloads, timing) +- Console error analysis and stack traces +- Real-time DOM inspection + +✅ **CI/CD Pipelines** +- Headless execution support +- Stable, script-based automation +- No authentication state dependencies + +**Ideal workflow:** "Find why this page is slow" or "Debug this API call" + +--- + +### **Claude in Chrome is BEST for:** + +✅ **Manual Testing Assistance** +- Testing while logged into your accounts +- Exploratory testing with visual context +- Recording workflows you can replay + +✅ **Quick Verification** +- Design verification (comparing Figma to output) +- Spot-checking new features +- Reading console errors during development + +✅ **Recurring Browser Tasks** +- Scheduled automated checks +- Multi-tab workflow management +- Learning from your recorded actions + +**Ideal workflow:** "Check if my changes look right" or "Test this form with my login" + +--- + +### **Playwright MCP is BEST for:** + +✅ **E2E Test Automation** +- Cross-browser testing (Chrome, Firefox, Safari) +- Generating reusable test scripts +- Page Object Model generation + +✅ **Reliable UI Testing** +- Accessibility tree = no flaky selectors +- Deterministic interactions +- Less prone to breaking from UI changes + +✅ **CI/CD Integration** +- Headless mode for pipelines +- Generate Playwright test files from natural language +- Integration with test management tools + +**Ideal workflow:** "Write E2E tests for this user flow" or "Test this across browsers" + +--- + +## 5. Token Efficiency Analysis + +| Tool | Token Usage | % of Context | Efficiency Rating | +|------|-------------|--------------|-------------------| +| Playwright MCP | ~13.7k | 6.8% | ⭐⭐⭐⭐⭐ Best | +| Claude in Chrome | ~15.4k | 7.7% | ⭐⭐⭐⭐ Good | +| Chrome DevTools MCP | ~19.0k | 9.5% | ⭐⭐⭐ Acceptable | + +**Impact:** With 200k token context: +- Playwright leaves 186.3k tokens for your work +- Claude in Chrome leaves 184.6k tokens +- Chrome DevTools leaves 181k tokens + +The ~5.3k token difference between Playwright and Chrome DevTools could matter for complex sessions with lots of code context. + +--- + +## 6. Security Considerations + +### Chrome DevTools MCP +- ✅ Isolated browser profile by default +- ✅ No cloud dependencies +- ✅ Full local control +- ⚠️ Remote debugging port security (use isolated profiles) + +### Claude in Chrome +- ⚠️ **23.6% attack success rate** without mitigations (reduced to 11.2% with defenses) +- ⚠️ Uses your actual browser session (cookie exposure risk) +- ⚠️ Blocked from financial/adult/pirated sites +- ⚠️ Still in beta with known vulnerabilities + +### Playwright MCP +- ✅ Isolated browser contexts +- ✅ No cloud dependencies +- ✅ Mature security model (Microsoft backing) +- ✅ Can handle authentication safely + +--- + +## 7. Installation Commands + +### Chrome DevTools MCP + +```bash +claude mcp add chrome-devtools npx chrome-devtools-mcp@latest +``` + +### Claude in Chrome + +``` +Install from Chrome Web Store (requires Pro/Max/Team/Enterprise plan) +``` + +### Playwright MCP (Recommended) + +```bash +# First, install browsers +npx playwright install + +# Then add to Claude Code (user scope = all projects) +claude mcp add playwright -s user -- npx @playwright/mcp@latest +``` + +--- + +## 8. Recommendations + +### **For Your Automated Testing Workflow:** + +#### 🥇 **Primary Tool: Playwright MCP** + +**Use for:** Day-to-day E2E testing, cross-browser verification, generating test scripts + +**Why:** +- Lowest token usage (more context for your code) +- Cross-browser support (Chrome, Firefox, Safari) +- Accessibility tree approach = more reliable selectors +- Excellent CI/CD integration +- Can generate actual Playwright test files +- Free, no subscription required + +#### 🥈 **Secondary Tool: Chrome DevTools MCP** + +**Use for:** Performance debugging, network analysis, Core Web Vitals + +**Why:** +- Unmatched for performance traces and debugging +- Deep network request inspection +- Official Google tooling with long-term support +- Essential when you need to answer "why is this slow?" + +#### 🥉 **Situational: Claude in Chrome** + +**Use for:** Quick manual verification while logged in, exploratory testing, design verification + +**Why:** +- Good for quick visual checks during development +- Can read your logged-in state +- Useful for "does this look right?" verification +- Skip for CI/CD or serious test automation + +--- + +## 9. Recommended Setup + +```bash +# Install both Playwright and Chrome DevTools MCP +npx playwright install +claude mcp add playwright -s user -- npx @playwright/mcp@latest +claude mcp add chrome-devtools -s user -- npx chrome-devtools-mcp@latest +``` + +### Suggested Workflow + +``` +1. DEVELOP → Claude Code (terminal) +2. TEST → Playwright MCP (E2E, cross-browser) +3. DEBUG → Chrome DevTools MCP (performance, network) +4. VERIFY → Claude in Chrome (quick visual checks) +5. CI/CD → Playwright MCP (headless, automated) +``` + +--- + +## 10. Final Verdict + +| If You Need... | Use This | +|----------------|----------| +| Cross-browser E2E tests | **Playwright MCP** | +| Performance analysis | **Chrome DevTools MCP** | +| Network debugging | **Chrome DevTools MCP** | +| Quick visual verification | **Claude in Chrome** | +| CI/CD automation | **Playwright MCP** | +| Test script generation | **Playwright MCP** | +| Lowest token usage | **Playwright MCP** | +| Logged-in session testing | **Claude in Chrome** | +| Console log debugging | **Chrome DevTools MCP** | + +### **TL;DR Recommendation:** + +**Install both Playwright MCP and Chrome DevTools MCP.** Use Playwright as your primary testing tool (it's more token-efficient, cross-browser, and better for E2E). Use Chrome DevTools when you need deep performance analysis or network debugging. Use Claude in Chrome only for quick manual verifications where you need your logged-in session. + +--- + +## Sources + +- [Chrome DevTools MCP - GitHub](https://github.com/ChromeDevTools/chrome-devtools-mcp) +- [Anthropic - Piloting Claude in Chrome](https://claude.com/blog/claude-for-chrome) +- [Claude in Chrome Help Center](https://support.claude.com/en/articles/12012173-getting-started-with-claude-in-chrome) +- [Playwright MCP - GitHub](https://github.com/microsoft/playwright-mcp) +- [Simon Willison - Using Playwright MCP with Claude Code](https://til.simonwillison.net/claude-code/playwright-mcp-claude-code) +- [Testomat.io - Playwright MCP Claude Code](https://testomat.io/blog/playwright-mcp-claude-code/) +- [MCP Integration Guide - Scrapeless](https://www.scrapeless.com/en/blog/mcp-integration-guide) +- [Chrome DevTools MCP Guide - Vladimir Siedykh](https://vladimirsiedykh.com/blog/chrome-devtools-mcp-ai-browser-debugging-complete-guide-2025) +- [Addy Osmani - Give your AI eyes](https://addyosmani.com/blog/devtools-mcp/) + +--- + +*This report was generated by Claude Code using the Opus 4.5 model on December 19, 2025.*