IIS Debug Tools: A Practical Guide for Troubleshooting Performance IssuesTroubleshooting IIS performance issues requires a mix of monitoring, diagnostics, and targeted debugging. This guide walks through the most useful IIS debug tools and techniques, how to collect and interpret data, and practical workflows to diagnose common performance problems such as high CPU, memory leaks, slow request handling, and thread pool exhaustion.
Overview: when to use IIS debug tools
Use IIS debug tools when you see symptoms like:
- High CPU usage on the web server
- Steady memory growth or unexpected recycled worker processes (w3wp.exe)
- Slow responses or timeouts for specific pages or endpoints
- Thread pool starvation, request queueing, or CLR thread blocking
- Intermittent crashes or unhandled exceptions
Start with lightweight monitoring (Performance Monitor, IIS logs, Failed Request Tracing) and escalate to heavier debugging (Debug Diagnostics, ProcDump, WinDbg) when pinpointing the root cause.
Key tools and what they do
- IIS Manager / Logging / Request Tracing — built-in visibility: access logs, request details, and status codes.
- Performance Monitor (PerfMon) — track counters for CPU, memory, ASP.NET, .NET CLR, and IIS-specific metrics like Requests Queued, Current Connections, and Worker Processes.
- Failed Request Tracing (FREB) — capture detailed traces for individual requests that meet rules (slow, status code ranges).
- Debug Diagnostics (DebugDiag) — specialized for IIS: capture memory dumps based on rules (high CPU, memory leaks, crashes); includes analysis scripts.
- ProcDump — lightweight tool to capture dumps based on triggers (CPU spike, unhandled exceptions).
- Windows Event Viewer — system and application events, IIS and WAS messages, and CLR exceptions.
- WinDbg / SOS / PSSCOR2/4 — deep post-mortem analysis of dumps; inspect managed heaps, threads, and locks.
- Application Insights / New Relic / Dynatrace (APM tools) — code-level telemetry and distributed tracing (if already integrated).
- IIS Advanced Logging / Log Parser / Log Analytics — for log analysis and custom queries across log files.
Data collection strategy
- Reproduce or wait for the issue while keeping monitoring enabled.
- Capture lightweight telemetry:
- Enable Performance Monitor counters: % Processor Time, Private Bytes, Gen 0/1/2 sizes, CLR Exceptions, Requests Current, Requests Queued, Queue Length.
- Turn on IIS Request Logging and FREB for slow or failing URLs.
- If CPU, memory, or crashes are observed, collect memory/process dumps:
- Use ProcDump for CPU-based dumps: procdump -ma -p
-s 5 -n 3 -c 90 (capture when CPU >90% for 5s). - Use DebugDiag to capture full memory dumps on memory leak growth patterns or unhandled exceptions.
- Use ProcDump for CPU-based dumps: procdump -ma -p
- Preserve application and environment context: app pool name, w3wp PID, recycle times, recent deployments, config changes.
- Always collect multiple dumps across time if the issue is intermittent.
Diagnosing high CPU
Common causes: infinite loops, hot code paths, frequent blocking/locking, or native interop issues.
Workflow:
- Identify offending w3wp.exe process (IIS Manager → Worker Processes or Task Manager).
- Capture a CPU-triggered dump with ProcDump or a series of lightweight thread dumps.
- Use DebugDiag’s CPU analysis or WinDbg with !runaway, !analyze -v, and thread inspection:
- In WinDbg: load SOS (for .NET) and use !threads, !clrstack, !dumpstack to find managed call stacks consuming CPU.
- Look for repeated stack traces in multiple threads or tight loops in application code.
- If native code appears in stacks, examine symbols and possible interop paths.
Practical tip: If CPU is caused by garbage collection, PerfMon counters like % Time in GC and Gen 2 size help determine if tuning GC or memory usage is needed.
Diagnosing memory leaks and high memory usage
Symptoms: Private Bytes steadily increasing, periodic app pool recycle due to memory limits, or OutOfMemoryException.
Workflow:
- Monitor Private Bytes and Virtual Bytes for the w3wp process.
- Capture full memory dumps (DebugDiag or procdump -ma) when memory reaches suspicious levels.
- Use DebugDiag memory leak analysis to get a summary of root causes and suspected leaking types.
- For managed leaks, use WinDbg + SOS and commands:
- !dumpheap -stat to identify dominant types by size/count
- !dumpheap -type
and !gcroot to find why objects are rooted
- Check for common leak patterns: static collections, event handlers not unsubscribed, long-lived caches, native resource handles not released.
- For native leaks, use UMDH or DebugDiag native leak analysis.
Diagnosing slow requests and request queueing
Symptoms: high Requests Queued, long request duration in logs, FREB traces showing long module handlers.
Workflow:
- Use FREB to capture detailed timing for slow requests and identify which pipeline module or handler is taking time.
- Check ASP.NET request queueing counters (Requests Queued, Queue Length) and thread pool metrics (.NET ThreadPool Queue Length, Active Threads).
- Capture thread dumps to inspect blocked threads; in WinDbg/SOS use !syncblk to find locking contention, and !threads / !clrstack on blocked threads to see where they wait.
- Investigate database, external service, or synchronous I/O calls that block request threads. Use APM traces or add timing logs around external calls.
- Consider scaling options: increase thread pool minimums, enable async patterns in code, or offload heavy work to background services/queues.
Common IIS-specific pitfalls
- Misconfigured application pool settings (rapid-fail protection, recycling thresholds) causing misleading symptoms.
- Overly aggressive requestLimits or queueLength settings causing 503s under load.
- Running in Classic pipeline mode with modules expecting Integrated mode.
- Native modules or ISAPI filters causing instability or leaks.
- Not using asynchronous I/O in ASP.NET handlers leading to thread starvation under high latency calls.
Example troubleshoot scenario (high CPU causing poor response times)
- Observe sustained high CPU on server and slow responses.
- Identify w3wp process with highest CPU.
- Run: procdump -ma -p
-s 5 -n 3 -c 85 - Run DebugDiag CPU analysis on the dumps; if using WinDbg, run !runaway then load SOS and run !threads and !clrstack for hot threads.
- Find repeated managed method X called from many threads — inspect code paths, add logging, and profile locally to reproduce.
- Fix may be a tight loop introduced in recent deployment; patch and redeploy, monitor.
Practical tips and best practices
- Reproduce issues in a staging environment before debugging in production when possible.
- Always collect context: timestamps, app pool recycles, configuration changes, and deployment history.
- Keep symbols and PDBs for your builds accessible when analyzing managed/native dumps.
- Automate recurring dump collection with ProcDump rules for known failure modes.
- Use combination of tools: PerfMon/FREB for triage, DebugDiag/ProcDump for capture, and WinDbg/SOS for deep analysis.
- When in doubt, capture multiple smaller dumps over time rather than a single huge dump.
When to escalate to developers or Microsoft support
- You find a native crash in IIS or unexpected behavior in WAS/IIS that you cannot correlate to managed code.
- Debugging points to OS-level networking or kernel drivers.
- You need Microsoft-level dump analysis for complex native bugs in IIS or HTTP.sys.
Quick reference: common commands
- ProcDump CPU trigger:
procdump -ma -p <PID> -s 5 -n 3 -c 90 - WinDbg basic analysis (examples):
.loadby sos clr !threads !dumpheap -stat !gcroot <ObjectAddr> - DebugDiag: use built-in rule templates for CPU, memory, and crash analysis.
If you want, I can:
- Produce a concise checklist to run during an incident.
- Walk through a real dump analysis step‑by‑step with sample output.
Leave a Reply