Apache Log Analyzer

SpectX’ log analyzer runs queries on raw log files in their current storage, no need to import or index them anywhere. The Apache access log format is detected automatically. Simply replace the URI of the data in the following scripts and get querying.  The data can be queried from your desktop, in a log server storing text-based logs or in the cloud. SpectX is fast - querying 10GB of log files on a local machine using 2 data processing units takes about 6 seconds.

To analyze Apache log files with the free edition of SpectX:

1. Download and install SpectX Desktop (installation instructions for Windows, Linux, OSX)
2. Open Data browser, navigate to the directory containing your logs. Click on one of the files and press 'Preview' to see it in raw form: 

Screenshot of SpectX data browser

2a: If the files are located in a remote location, press New>Datastore to configure access to the remote storage. Then open the datastore.

3. Press ‘Prepare query’ and then ‘Run’.  You should see a clean table of your Apache logs:

Screenshot of parsed Apache logs

Pro-tip #1: Add a wildcard * to the path in the LIST command to query multiple files in the same directory
Pro-tip #2: If you wish to calculate country codes, AS names and -numbers from the IP addresses, configure SpectX’ access to the Maxmind GeoLite2 database

What to Ask? 5 Sample Queries

Paste these queries one at a time at the end of the script you ran in step 3 above.  Or take a look at the SpectX Apache query pack in Github for more queries..

Pro-tip #3: use Ctrl+space while typing to auto-complete queries.

1. Search for URIs that contain the string 'php'
| filter(lower(uri) CONTAINS ‘php’)
2. How many hits per hour?
| select(time:timestamp[1h], count(*))
| group(time)
| sort(timestamp)
3. Top user agents?
| select(agent, cnt:count(*))
| group(@1)
| sort(cnt DESC)
| limit(100)
4. Count successful and failed requests per IP
| select(clientIp, success:count(response<400), failure:(count(response>=400)))
| group(clientIp)
5. See successful and failed requests per country. Note: you first need to configure Maxmind access to calculate country codes from IPs.
| select(cc(clientIp), success:count(response<400), failure:(count(response>=400)))
| group(cc)
Any questions? Join our Slack channel to ask tips from the community or email us at support@spectx.com.