Troubleshooting IIS 7: Examining Trace Failed Request Logs
Troubleshooting is one of the most important aspects of system management that a systems administrator can learn. It can mean the difference between a system being down for 30 minutes and you being a hero for getting an issue resolved quickly, or being down for 24 hours, and management looking for replacement personnel. Learning the locations and types of information available to help resolve issues is an important step in becoming the go-to guy when problems develop.
Web servers in general are difficult to troubleshoot user issues because often things on the user side of the equation and outside of your direct control might be causing the problem. While you will still have those unknown parts of the equation, Trace Failed Request Logs allow you to follow a request from the start through internal IIS processes until its end and generate a report of any Failed Requests. This can give you immense amounts of information to work with and help you narrow down any issue that may arise.
This article will walk you through installing and configuring a Failed Request Trace Log rule to troubleshoot an error a user is having while accessing a website that requires authentication. There are more robust scenarios possible but this will give you a good start for using this feature.
You can use Trace Logs to track failed requests from the following providers by default:
- WWW Server – Trace failed requests through the server core. Use this to trace requests through IIS worker processes.
- ASP.NET – Trace failed requests through the ASP.NET managed code. Includes .aspx and other managed modules, like forms-based authentication requests.
- ASP – Use this provider to trace ASP request from start to finish.
- ISAPI Extension – Use this when tracing requests in and out of an ISAPI extension process
For this article we are going to look at the WWW Server provider more in depth. Within that provider you can trace any combination of the following:
- Authentication – Help track down authentication errors from related modules.
- Security – When you suspect a security issue is occurring because of insufficient privileges you can use this area to report those failed requests.
- Filter – Tracks requests through IsapiFilterModule, RequestFilteringModule or both. Use to determine length of time for a request to process through ISAPI filter.
- StaticFile – Trace requests to the StaticFile module, use to see how long requests are taking or how filters may be affecting those requests.
- CGI – Track requests through CGIModule, use to trace a request for a CGI file through the CGIModule.
- Compression – Use to trace requests through compression related modules, StaticCompressionModule & DynamicCompressionModule.
- Cache – Trace events through cache related modules.
- RequestNotifications – Capture all request notifications from start to finish through the RequestMonitorModule.
Note: Creating Trace Logs will degrade your server’s performance, so do NOT implement on a production server unless it is truly necessary and you understand that it could cause other problems because of the performance hit.
As with all things in IIS 7 enabling the additional functionality requires an additional module being installed and configured to the core installation. In this case the HTTP Tracing role service has to be installed and enabled on the server you want to run the tracing on.
1. Open Server Manager, in the left pane select Roles, and in the center select Add Role Services.
2. Next scroll down to the Health & Diagnostics section and place a check next to Tracing, then click Next.
3. Confirm the module you are installing by visually looking it over and then click Install.
4. After installation you will get a results screen that should give you the status of the install. Go ahead and click Close.
You can configure Failed Request Tracing Rules at either the server or site level depending on your needs. For our example I am going to configure the rules at the server level. Let’s get started:
1. Open IIS Manager and select the server you want to work with. In the center pane double click on Failed Request Tracing Rules.
2. In the right hand pane click on Add.
3. In the Add Failed Request Tracing Rule window you can specify the content to trace with several options.
The custom rule will accept wildcards also, so there is a lot of flexibility and by limiting the content monitored you can limit the performance impact on the server. Since we are using a test server I am going to select All Content, and click Next.
4. Since we are looking at authentication errors I will filter based on the 401 authentication failure error. So check the box next to Status Code(s): and fill in 401 for the code we will be filtering for, then click Next.
Note: You can find a full list of status codes at Microsoft Help and Support.
5. In the next window we can select the providers we want to monitor. In this case I am only going to choose WWW Server.
I then can choose the level of filter for information, while doing general troubleshooting I recommend Verbose. You can also filter which parts of the provider you want to audit, and since I know the problem I could choose just Authentication, but I will leave them all selected for now.
Click Finished when you are done selecting your parameters.
6. After a second you will now see your rule in the center pane.
7. Now even though you have set the rules at the server level, you have to enable the logging at the site level. Navigate to your site and then in the right pane click Failed Request Tracing.
8. Place a check next to Enable, and then you can set the location for the logfiles and also how many you want to keep. Click Ok to confirm.
Note: Unlike other log files there is an individual record kept for each instance.
That’s it; you now have configured your first Failed Request Tracing rule. Now let’s take a look at the results.
Failed Request Tracing Log Example
Let’s take a look at an example of the output from our rule. I connected to a website that was created requiring basic authentication and then used phony credentials.
Here is what the output of the error looked like:
As you can see from the summary view it shows you the trigger for the error and the module that traced the error. If you click on the Details or Compact View you will find a wealth of information about what exactly was going on with that request from start to finish and the reason it failed.
Here is part of the Compact View that traces the request event by event matched with a time code.
This has been a very simplistic look at this very in depth troubleshooting tool, but I hope it shines a little light on the immense help it can be in troubleshooting Failed Request Tracing, especially with in house code.
It also shows you exactly how IIS handles certain events and requests, so it can be used as an educational tool for those who require that kind of in depth knowledge.