Analog

Introduction

Analog is an "old-school" log file analyzer. It was written by Stephen Turner and released as freeware on June 21, 1995, but the license was changed in November 2004 to the GNU General Public License. Stephen wrote Analog while studying for his PhD at the University of Cambridge.

Versions existed for Windows, macOS, Linux, and most Unix-like operating systems but in December 2004, after releasing version 6 of the program, Stephen ceased work on it. The original website, www.analog.cx (Internet Archive), was last updated in 2005 and disappeared in August 2014, but there are mirrors of it still around such as the one at reverse.net and the one at web42.com.

In 2007, as a long-term user of Analog C:Amie revived it and produced an updated version of Analog, Analog CE, with the full download, along with the help files, examples and source code on GitHub. This is this version of Analog that I use.

When I checked in October 2023, the download link on the Analog C:Amie site no longer worked and simple gave a database error. The last available version was 6.0.17 created on June 20, 2021. For this reason I have made a zip file of the files from GitHub that includes the last version of analog.exe for Windows. Download (Zip, 2.15Mb). I cannot accept any responsibility for these files. They are not mine and apart from the broken link on the website, I do not know why they are no longer available. All I can say is that this is the version I use and it appears to work.

Installation

Analog is a simple program consisting of a single Windows EXE file that requires no installation along with a CSS, configuration, image, and language files that requires no installation apart from creating the folders that hold them. It can be run directly from the command line which makes it particularly easy to add to Windows Task Scheduler.

The only thing I forgot to do when setting it up was to copy the images folder, which also includes a CSS file, to the directory that contains the output statistics page.

Analog installation folder along with the configuration files
The images folder must also be copied to the folder that contains Analog's output files

Organization

This is where and how I installed the Anlog files:

Configuration Files

At some point I have to read all the documentation and configure Analog properly, but it is very easy to get a basic output from the program. All the standard configuration file needs is which Apache log file to read and where to put the output. This is part of my analog-brisray.cfg file:

LOGFORMAT COMBINED
LOGFILE "C:\Apache24\logs\brisray-access-2017*.log"
LOGFILE "C:\Apache24\logs\brisray-access-2018*.log"
LOGFILE "C:\Apache24\logs\brisray-access-2019*.log"
LOGFILE "C:\Apache24\logs\brisray-access-202*.log"
LOGFILE "C:\Apache24\logs\brisray-access.log"
OUTFILE ../htdocs/brisray/utils/alogs/index.htm
HOSTNAME "for brisray"

It looks like this because I split the Apache log file into monthly files. run-analog.bat is the batch file that actually runs Analog and tells it which configuration files to use. I have three websites, and an Analog configuration file for each of them. The batch file is simply:

CD C:\Apache24\analog
analog +ganalog-brisray.cfg -G
analog +ganalog-gambia.cfg -G
analog +ganalog-offroad.cfg -G

It simply changes to the Analog directory while the configuration files to use are between the +g and -G delimiters. Using a single batch file to run all three sites mens it is simple to set up the batch file in Windows Task Scheduler. Instructions on how to do that can be found on the mirror sites such as reverse.net or web42.com.

I split the Apache log files at just after midnight on the 1st of each month. I run Analog daily at around 1am. I include the unsplit Apache log file as Analog does not do incremental processing and runs through every record in every log file when I run it. This means log records are not counted twice. It also means I can move Analog's output files into a new folder when it starts to run out of memory.

Old Files

It may be that I have not read the help pages deeply enough yet but Analog has to read all the Apache log files a report is needed for every time it is run. It appears that it cannot, for example, run last month's log files and simply append this month's log file results to what it already has.

Low Memory

My web server, "The Server in the Cellar", is a 2016 Dell Inspiron 3847 desktop running Windows 10 with an Intel i5-4460 @ 3.2GHz CPU, a 1Tb SSD, and 16Gb RAM. I have log files going to back to January 2016. In February 2023, I noticed that for the site with the most traffic, this one, the output from Analog was not being completed properly. An easy way to tell if this is happening is to look at the reports produced by Analog. If the footer of the page, the part that contains the Analog version number and the time taken to complete the page, is missing, then it never finished processing properly and that's usually due to a memory problem.

A little bit of troubleshooting showed the reason why - the machine was running out of memory while processing the log files for it. The simplest solution was to run Analog on the log files from January 2016 to December 2021 and output that to a new folder, not forgetting that folder needed a copy of the images folder. Analog was run again in a similar way to output the statistics from 2017 to 2022.

One easy way to tell if this is happening is to look at the bottom of output page created. If the "This analysis was produced by CE 6.0.17." text, along with the running time and bottom menu does not appear, then the program did not finish properly.

Analog only exhibits this behaviour only after running through over 23Gb of files on my web server, a 2016, Dell Inspiron 3847 with an Intel i5-6200U processor @ 2.4GHz, 16Gb RAM and 1TB SSD. This is only happens after processing roughly four year's worth of my log files. As it is happens so infrequently and the program is very fast it is no great task to manually create the new folders, change the config file, and run Analog again.

Live Output

I have decided to make the statistics obtained from my web logs public. The logs are subject to referer spam, so if the referring site in these analytics pages looks suspicious, it probably is.

Unless it was changed, Analog's statistics pages always start with "Web Server Statistics for". A search of that phrase in Google or Bing brings up thousands of pages, some dating back to the original 1995 version of Analog and many using Report Magic.

Newer File Types

Analog was written in 1995, and I use Analog CE v6.0.17 which was published in June 2021. It seems the multimedia list Analog uses have has not changed much since it was first published and I added webp, webm and svg files to the list of file types to be counted as hits but not page views. To do this I added the lines to the configuration file:

TYPEOUTPUTALIAS .webp ".webp [Google Web Graphic]"
TYPEOUTPUTALIAS .svg ".svg [Scalable Vector Graphic]"
TYPEOUTPUTALIAS .webm ".webm [Google Web Movie]"

Fun Facts

Analog described itself (Internet Archive) as "The most popular logfile analyzer in the world". It's difficult to argue with that because in October and November 1998, Georgia Tech's Graphic, Visualization, & Usability Center (GVU) found that Analog was used by 24.9% of those surveyed. Web Trends came second with 20.3% of the market. If you are interested, the original datasets can be found on the Internet Archive.

Sources & Resources

Analog.cx (Internet Archive) - the first site
Analog.cx (Internet Archive) - the final site
Analog Documentation - this site has archives of the original Analog documentation from version 0.9 Beta 3 to version 6
Analog Help - The email listings from 2005 to 2009
Analog (Wikipedia)
C:Amie (not) Com! - The home of Analog CE
Comparison table of the features offered by Analog, AWStats, and Webalizer
Trucksess - How Chris Trucksess edited his Analog configuration files to simplify the analytics page

Helper Utilities

On the Analog mirrors, the helper programs are in the base directory/helpers folder. Since the pages were written some of the helper program sites have disappeared. This list gives the Internet Archive of the last version I could find for them.

A-Soft Analog Shell
Analog Crawler
analog-fiscal-year.pl
SEARCHENGINE commands http://www.science.co.il and http://mba.vanderbilt.edu/
ROBOTINCLUDE commands
psLogger
adddates.pl
dnsparse.pl
fastresolve
ip2host
LookupIP

Mirror Sites - Original Files

Not surprisingly, the list of mirror sites found on places such as reverse.net and at web42.com is getting shorter. Of the list of 40 listed on those pages, which appear to have been written in November 2004, there were just five left in April 2023. When I checked in February 2024, the mirror in Hungary had gone leaving just four, and just one outside the US. Although that is quite a drop, for a 20 year old program that is quite a feat.