Problem

When I tried to process a log file no statistical data was being collected. No error messages were generated even though many new records were processed. The data file remained very small.

Analysis

I set up a test environment in which I could process the data with the new version and debugging turned on. To turn debugging on I changed DebugMessages from 0 to 1 in the configuration file and added "-debug=10" parameter to awstats.pl command used to process the log file. This gave me all of the debug messages printed to the screen. I started processing a log file and interrupted it after it had processed a few lines by hitting Ctrl-C. In the debug output I could see that 7.0 version was having problems with the log format. Some values were parsed correctly and others looked like they were offset by one variable. In my configuration I was using a custom logformat which used a combination of IIS column names and AWSTATS keywords. I don’t remember how I created it but I suspect I found an example on the Internet. It worked for the last two years but now it seemed to have a problem with cs-username(%logname).

Solution

My solution was to convert the IIS column names to the corresponding AWSTATS keywords. Even after using the AWSTATS keywords it was still not parsing correctly. To get the parsing to work correctly, I had to change %logname to %other. In my log files, the value for cs-username(%logname) is a single dash.   Obviously it is not important or necessary. When I look at the code, line 9027, I see a comment that says the regular expression code allows spaces to support Lotus Notes. As a result it puts two fields into one variable. Naturally the parsing for %other works correctly. After I knew what the problem and answer was, I could find the answer on the Internet. Here is someone else who experienced the same problem, http://www.internetofficer.com/forum/awstats-iis-installation-and-configuration/awstats-wont-recognize-logname-cs-username-field/. It sure would have been nice if Awstats re-mapped cs-username to %other rather than %logname.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

1. I went to http://www.countryipblocks.net/ to get the IP ranges I wanted to block. Beware these ranges include bogon networks(e.g. 192.168.0.0). The first time I applied the IP ranges I blocked myself out.
2. I used the perl script below to merge the networks.
3. I used the obligatorymoniker IP Security.vbs script to load the ranges. You will have to change the "IIS://localhost/smtpsvc/1" to the site you want to add the IP restrictions to.

use Net::CIDR::Lite;
use NetAddr::IP::Lite;
my $cidr = Net::CIDR::Lite->new;
# Disallow IPs

open (IPDISALLOW, "ip_disallow.txt") || die "couldn't open the file!";

while ($record = <IPDISALLOW>) {
 if (substr($record,0,1) != '#'){
#print $record;
 $cidr->add($record);
 }
}

close(IPDISALLOW);
#print "$_\n" for $cidr->list;
foreach ($cidr->list) {
my $ip = new NetAddr::IP::Lite $_;
#print "The address is ", $ip->addr, " with mask ", $ip->mask, "\n" ;
print $ip->addr, ",", $ip->mask,"$_\n";
} 

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Over the last couple of days I was receiving complaints about slow responses from our web site. By the time I would look at the CPU utilization it would be within the normal range. I looked in the IIS log file for timeouts but could not find any. So I ran a two hour trace on Thursday afternoon. Friday morning I crunched the trace with PAL and discovered several unaccountable CPU peaks attributable to w3wp.exe. An Internet search for “w3wp.exe high cpu usage”  led me to this thread in which several people recommended using IISPeek to find the misbehaving request. So I installed a trial version of IISPeek and started watching the transactions coming in. Pretty soon I saw something I was not expecting. A shopping site was coming to our site and trying to retrieve a  product advertising feed. What was surprising was not that the shopping site was retrieving the feed but there was no log of it in the IIS log file.   This request had been consuming our CPU for several minutes and then disappearing without a trace. I knew this query had some serious performance issues but the IIS log indicated that it had been working on previous days. I did not know it was running so often. With IISPeek and Task Manager running together I could see the impact on the site. Evidently this particular shopping site would time out or cancel the query before our site either returned the data or timed out.  It was at this moment that I figured out that IIS must have a “no harm, no foul” policy about canceled queries. My reliance on the IIS log was a mistake in this case. Since this shopping site was not getting the data, the shopping site would try again at a later time. When I was watching it with IISPeek I was seeing this request about every fifteen minutes. Fortunately I had already developed some web page caching code I could implement quickly and get us over this hump. Within an hour the shopping site had its data and our web site was back to normal. I have solved a lot of web site problems by looking at the IIS log but in this case it was not the right tool for the job. On Monday I am buying a copy of IISPeek!

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.