Server Tools (part 2)

Copyright origomi

Before we get into the fun stuff it is important to become familiar with a few basic tools which can help you (i) measure performance and (ii) monitor the system over time.

Performance Measurement

The following tools will help you measure the performance of your site – helpful during development and troubleshooting. Moving from the client back to the server we have:

YSlow

A Firefox addon which gives you an overview of the request/response cycle for a single web page. It provides a summary of all requests and offers suggestions as to how things can be improved. This tool is mainly used for optimising the rendering time of a page. See Drupal Page Rendering for more details on the client side. However, in terms of server side performance it does report the download time for the actual ‘doc’ requested which indicates how fast a page is being built on the server.

YSlow

Dries Buytaert walks us through YSlow and compares scores across CMSs. Drupal scores 74, a C, and does pretty well compared to other contenders.

Page loading profiling tools

Seven distinctly different page loading profiling tools are compared. YSlow jumps out from the others.

Devel Module

The Devel module is an essential tool to have on your dev machine. Devel is your first port of call on the server side when troubleshooting a site. You can use it to discover where problems are occurring before diving in to test further and debug. If you are suspicious that things aren’t quite right it can help you out with slow queries, slow pages and RAM consumption.

In order to use it you must first enable “Devel” and “Performance Logging” on the modules page: /admin/build/modules/list. You must then configure it to report the info you need: /admin/settings/devel. Try turning on the following options to see a range of stats for each page:

• Collect query info
• Display query log
• Display page timer
• Display memory usage

MySQL Administrator

MySQL Administrator has a GUI which, amongst other things, shows the health of your server. Having a copy of it running on your dev machine can give you good feedback on:

• number of connections to the DB
• amount of traffic from the DB
• query cache hit rates
• key buffer hit rates

mysql client

The mysql command line client is invaluable for getting feedback from the server about what is going on. If a page is stuck, dive in and see what is happening.

mysql> SHOW FULL PROCESSLIST; Lists all currently running queries, how long they have been running, what they are doing and their process id. Are any queries dumping out to disk? Have any been running too long? Any queries locked waiting for another to complete? Whats the PID of that slow query?

Monitoring MySQL queries using PROCESSLIST along with system resource utilization

Simple shell script to monitor mysql.

mysql> KILL [pid]; Kill a query which will not end soon.

mysql> SHOW GLOBAL STATUS; Displays a number of status values from which it is possible to calculate query and key cache hit rates. You can do this visually with MySQL Administrator but this method is helpful when looking at remote server.

Server command line tools

$ top Dynamic list of processes on your server showing memory and CPU use.

$ mytop mytop will connect to a MySQL server and periodically run the SHOW PROCESSLIST and SHOW STATUS commands and attempt to summarize the information from them in a useful format.

$ ps aux List of processes showing user, memory and CPU.

$ vmstat Reports information about processes, memory, paging, block IO, traps, and cpu activity.

$ free -m Reports free memory in MB.

Performance Monitoring

It is important to have a handle on how things change over time. The following tools keep graphical, historic records so that you can analyze what has happened and when.

Munin

Munin is a network and system monitoring tool which provides insightful performance graphs over time. If you need to know when a problem hit then Munin will quickly be able to show you the stresses the server has been under.

Tools for Performance Tuning and Optimization

Linux system tools for monitoring a server.

Google Analytics

Google Analytics is analytic software which lets you track users and session across time amongst other things. Handy for observing traffic patterns. Implemented as JavaScript so will run for cached pages for users with JS turned on.

Woopra

Google Analytics competitor for tracking and analyzing web apps. Desktop Client GUI app rather than web interface. Variety of plans. Free plan limited to 30K page views per month.

Web analytics showdown: Woopra vs. Google Analytics

Woopra recommended only for small sites. Google integrates with Adsense and Adwords..

AWStats

AWStats is a server log file analysis tool which provides graphical stats of the requests made on your server. It is generally recommended that the Statistics Module in core be turned off and AWStats run in place. The Statistics Module doesn’t fire if Aggressive caching, Boost or Varnish are in operation and so stats will be skewed.

Most sites: Better visualisation, more accurate stats.

This article forms part of a series on Drupal performance and scalability. The first article in the series is Squeezing the last drop from Drupal: Performance and Scalability.