Jay Pipes is an excellent speaker whose enthusiasm about MySQL is contagious. If you get a chance to hear him speak or back him into a corner for 20 questions I highly suggest it.

The thing Jay Spoke about that was most relevent to what I’m doing now was Horizontal Partitioning [article] [manual] which is taking a table with a very large amount of rows and breaking that heap into a series of smaller groups that can be intelligently loaded into memory on a as needed basis.

The obvious way for to do this for my, and I’d assume many projects is by, year. This would speed things up because my project’s users are most likely to only need to access the current year’s information. If I only ask MySQL to load up a table containing the current year then that’s a much smaller bite to chew.

Horizontal Partitioning can also be done with a hash of one or more other columns, which makes it very flexible, and very useful for medium to huge projects.

On a related topic I also realized that I’ve been asked by my supervisor to investigate the benefits of Vertical Partitioning before either of us knew what to call it. Essentially it’s breaking up tables and adding one-to-one relationships between them. This allows frequently accessed sections of the original table to be accessed in isolation without having to load a bunch of unused fields to memory. I’m lead to believe that some db’s handle this pragmatically but MySQL does not.

Another key points were that the most cost effective way to mitigate a MySQL bottleneck is to add more RAM to the server. This is essentially throwing money at the problem, but less money that the equivalent labor costs associated with other solutions.

Lastly the common yet important advice of “use the smallest data container possible” because “a BigINT is twice the size of a INT but a INT is twice as fast”.

There were many many other gems from this lecture, but I forgot my notes at work and I’m tired of writing

Update (4/1/06 9:21PM) — I found this useful list of tips & tricks also partially written by Jay Pipes http://forge.mysql.com/wiki/Top10SQLPerformanceTips