Performance Tuning Advice

I’ve spent a lot of time this year investigating and resolving performance issues due to high load on a PHP web application (Moodle). The load we are talking about here is up to around 2000+ visitors per hour (accordingly to Google Analytics).

The following is a list of general advice on performance tuning. Note it isn’t all technical. How you go about the process is just as important as the technical changes.

  • Implement some form of analytics to track load. Google analytics with the real time monitor is fantastic. Don’t switch this off, even during high load, otherwise you will be flying blind and it will hurt your efforts more than help (if you can’t measure it, you can’t improve it).
  • Invest time on proper load testing scripts that simulate high load behaviour so you can measure and improve. Jmeter is great for this, but there are many tools out there to help you.
  • Apache processes will creep up to maximums (e.g. 256 for prefork worker processes) when they are waiting on something (database, disk etc). Tuning at this level includes tweaking Apache config, PHP config and PHP caching (e.g. eAccelerator). Other options include proxies and caching. This will improve load handling and the speed of your site. Also ensure that your application is caching as much as possible (javascript, css, http headers etc).
  • The database (MySQL in this case) has a huge impact on performance. Two very important things to look at are thread cache size (make sure it is on) and if using InnoDB, your InnoDB buffer pool size. Be sure to check out the excellent MySQLTuner perl script to help you get this right. Monitor your database and logs to find what might be holding things up. Apache relies on MySQL being snappy, if it is slow you’ll have bottlenecks further up the chain.
  • For all layers, its important to realise that the defaults won’t do. Tuning/tweaking is required to get the most out of your web server and database server. There’s a wealth of information out there, do your research and monitor your system to find the bottlenecks.
  • Document your changes. If you don’t know what you changed, then how can you tell what helped (or didn’t)? Be systematic. This isn’t the time to go around randomly changing things and hoping for the best.
  • Don’t make assumptions. It might sound strange but this is a huge one. A lot of performance tuning efforts are hindered by people assuming where problems are rather than finding them through proper metrics (e.g. load testing and monitoring). If you can’t measure it, you can’t call it (regardless of your “experience” and so on).