I just discovered Concrete5 CMS recently when another developer in my area launched a site with it. Always up for learning something new, I went to the website, read the sales pitch, and decided to give it a whirl. Before I spend time learning yet another CMS, for kicks I thought I would benchmark it for speed against WordPress, my current go-to solution. Here we go.
Preliminaries
Hardware: All tests will be conducted on my Desktop Computer; a custom built PC with a 3.5Ghz Core2 Duo, 4GB of Ram, and a 10,000 RPM hard drive. Not identical, but similar to many server setups on the market.
Software: I’m running Ubuntu 10.04 with Apache 2.2, PHP 5.3, and MySQL 5.1. Once again, apart from using a Desktop OS, this is almost identical to your usual LAMP server software. For benchmarking, I will be using Siege 2.68.
Step 1: Fresh Installs
I downloaded and installed the latest version of Concrete5 (5.4.1.1) and WordPress (3.03) Here are the screenshots of the home pages out of the box:
WordPress Fresh Install
Concrete5 Fresh Install
Step 2: Balancing
Okay, First thing, to be fair we need to balance the page weights. Siege will not load all the linked resources (like CSS and Javscript), I actually only care about the html page weights. Out of the box Concrete is 1771 bytes and WordPress is 2015 bytes. Pretty close. After removing several widgets from the WordPress sidebar (Those extra queries weren’t fair anyway) and adding the right amount of Lorem Ipsum, the WordPress page is now exactly 1771 bytes as well. Perfect.
Step 3: Attack!
To stress test my desktop server I am using the following siege command:
siege -c 50 -r 40 http://localhost/[siteurl]
This will attempt to make 50 concurrent requests to the website, and will repeat each request 40 times. This is a total of 2000 requests to each site.
Step 4: Results
Here is the raw data from the tests:
| WordPress | Concrete5 | |
|---|---|---|
| Total Requests: | 2000 | 2000 |
| Average Response Time (seconds): | 2.52 | 1.62 |
| Transactions per second: | 16.23 | 22.46 |
| Longest Transaction (seconds): | 4.8 | 3.13 |
| Shortest Transaction (seconds): | .10 | .07 |
| Elapsed Time | 123.21 | 89.04 |
And the corresponding screenshots:
Results for WordPress
Results for Concrete5
Step 5: Conclusions
As we can see from the data Concrete5 outperformed WordPress by 20-30% in every measure. This is a significant amount. What does this tell us? For small sites with little content, Concrete5 will scale to additional concurrent users better than WordPress. What does this not tell us? For one thing, the sites may not scale to additional content equally well. This test also ignored all the static content which will download from either CMS with equal speed. Finally, WordPress also has some excellent caching plugins that may have closed the gap.
Other Considerations
Am I suggesting you ditch WordPress and port your sites to Concrete? Not at all. WordPress has many good things going for it; it is the #1 blogging tool and is a finely tuned engine that powers millions of websites. What I CAN say for sure, is that if you can outperform that, then you’re doing something right. Hats off to the team at Concrete.
——————————————————-
Update (1 Day Later)
Okay, After posting this, I had a lot of people point out to me that this is not a fair fight. Concrete has caching enabled by default, and WordPress does not. After installing wp-super-cache, the elapsed time for 2000 requests from WordPress fell from 123 seconds to 27 seconds. Wow, crazy plugin. Either way, my original results stand when you consider out of the box performance. Wp-super-cache is neither bundled with WordPress nor do it’s version numbers suggest it is stable. It’s technical options are overwhelming to all but advanced users. Kudos still go to Concrete5 for integrating a simple, stable caching system into the framework.
Thank you thank you!
Performance has long been a complaint about concrete5 with past versions, we’re doing so much overhead with in-context editing that optimizing it has been a challenge.
With the most recent version 5.4.1 we added a lot of thought behind creating our own static page cache do deal with performance. It’s awesome to hear that it’s working well for you!
-frz
ceo, concrete5.
Woohoo! awesome!
Nice. Thanks. Install the mobile theme pack for me on here?
great post. thanks for taking the time to do this. One more reason to use concrete5 over wordpress & co.
Great work C5. Have been using the system for 18 months and love it. Whats better is that all my clients love it 😉
Keep up the good work
I’d like to see that test repeated without any caching whatsoever, on both systems. My guess is WordPress would win. With static page caching, almost any CMS will behave the same; like a static HTML page. The difference without caching is how many queries is done to show a page, and how hard the database is hit.