Apache Log Analyzer
SpectX’ log analyzer runs queries on raw log files in their current storage, no need to import or index them anywhere. The Apache access log format is detected automatically. Simply replace the URI of the data in the following scripts and get querying. The data can be queried from your desktop, in a log server storing text-based logs or in the cloud. SpectX is fast - querying 10GB of log files on a local machine using 2 data processing units takes about 6 seconds.
To analyze Apache log files with the free edition of SpectX:1. Download and install SpectX Desktop
(installation instructions for Windows
)2. Open Data browser,
navigate to the directory containing your logs. Click on one of the files and press 'Preview' to see it in raw form: 2a: If the files are located in a remote location,
to configure access to the remote storage. Then open the datastore.3. Press ‘Prepare query’ and then ‘Run’.
You should see a clean table of your Apache logs:Pro-tip #1:
Add a wildcard * to the path in the LIST command to query multiple files in the same directoryPro-tip #2:
If you wish to calculate country codes, AS names and -numbers from the IP addresses, configure SpectX’ access to the Maxmind GeoLite2 database
What to Ask? 5 Sample Queries
Paste these queries one at a time at the end of the script you ran in step 3
above. Or take a look at the SpectX Apache query pack in Github
for more queries..Pro-tip #3
: use Ctrl+space while typing to auto-complete queries.1. Search for URIs that contain the string 'php'
| filter(lower(uri) CONTAINS ‘php’)2. How many hits per hour?
| select(time:timestamp[1h], count(*)) 3. Top user agents?
| select(agent, cnt:count(*)) 4. Count successful and failed requests per IP
| sort(cnt DESC)
| select(clientIp, success:count(response<400), failure:(count(response>=400)))5. See successful and failed requests per country. Note: you first need to configure Maxmind access to calculate country codes from IPs.
| select(cc(clientIp), success:count(response<400), failure:(count(response>=400)))
Any questions? Join our Slack channel to ask tips from the community
or email us at firstname.lastname@example.org