Testing the Impacts of Website Caching Tools

This article was originally published on Sucuri blog (read here).

Try to remember what you ate for lunch yesterday.

It took you about 3-5 seconds, right?

Ok. Now recall that memory once more.

Took you less than a second this time, for sure.

You remembered much faster the second time around because you didn’t have to “query” that information again from your brain’s “storage”. The information was cached. The same principle applies to a website cache, but at a very basic level. Of course with multiple data centers (i.e. brains) storing the content (i.e. memory), the results are exponentially better.

There are several tools available for caching website content, including popular plugins and content delivery networks (CDN). The Sucuri Firewall runs on top of a built-in, globally-distributed Anycast CDN. After it processes a specific content request for the first time, it does most of the heavy lifting when serving the same content for every visit thereafter.

We decided to run a series of experiments (including simulating DoS attacks) to observe the impacts of caching tools on website performance.

Cache Testing Methodology

A small WordPress server located in Los Angeles (USA) was deployed for this article. Only one plugin named FakerPress was installed in order to generate dummy content and populate the WordPress install so that its behavior would be consistent with real sites.

Numbers and images do most of the talking in this article, and although it sounds complex, the results are clear enough to give you a good understanding of how caching affects overall website performance – and security.

Note: All tests in this article were performed three times and the best results reported.

A professional load-test tool could have been used, but to simplify things, another server at Roubaix (France) and a software called Siege (version 3.0.8) was used instead to measure the performance with the following command.

timeout -sHUP 1m siege -c1 test-domain.com

For those of you interested, here’s a breakdown of how this command works.

To run for 1 minute and then kill the command:

timeout -sHUP 1m

To run 1 concurrent user on “test-domain.com”

siege -c1 test-domain.com

Test #1 – No Caching – One Connection

First, we wanted to get a baseline so we started with no caching tools on the site and ran a speed test with only one connection (i.e. visitor) making a request to the site.

To see what happened with the WordPress server, an awesome tool called NetData provided the graphs needed (with the peak moment selected):

So far so good. The Roubaix test server IP was whitelisted on the firewall to ensure that security filters didn’t interfere with the benchmark. The ‘hosts’ file of the Roubaix test server had been edited to reach the website through the Firewall.

Test #2 – CDN Caching – One Connection

Here’s the result of the same command, but now using Sucuri Firewall to handle the traffic and content delivery:

The Sucuri Firewall runs on top of a built-in CDN, so although the WordPress site was hosted in LA, most of the content came from Sucuri Frankfurt data center (nearest to France), which improved the response time.

The resource usage looks very different too:

As mentioned, the numbers and images would do most of the talking here. The Firewall completed twice as many transactions in a simple 1 concurrent test.

It wasn’t only faster. Oh no. This is only the tip of the iceberg.

Facing a DoS Attack

We continued to use Siege but this time increased the amount of stress on the WordPress server.

In essence, we simulated a very small Denial-of-Service (DoS) attack. The next four tests simulated 20 concurrent users hitting the site in a single minute. 

Due to our Distributed Denial of Service (DDoS) protection, we had to whitelist the server IP. This made sure that Sucuri Firewall did not block the stress test. In a normal configuration, the Sucuri Firewall detects bot-like behavior and blocks it quickly to avoid any damage on the website server.

Don’t forget that because of the low cost and popularity of DDoS attacks today, it’s more likely that you would face bigger and more sophisticated attacks against your website.

Test #3 – No Caching – 20 Concurrent Connections

Let’s spice things up a bit. Instead of 1 concurrent user, 20 concurrent connections will go to battle.

As usual, the WordPress server (at Los Angeles) will be hit directly first:

The resource usage was so high that NetData couldn’t get the exact metric in a few intervals:

Test #4 – Firewall Caching – 20 Concurrent Connections

Are you curious to know the results when using the firewall? Me too!

The test server’s ‘hosts’ file was edited to reach the firewall first, and game on:

Think the WordPress server is exhausted? Not at all, take a look:

Response time went from 3.69 seconds to 0.03 seconds with Sucuri Firewall in place.

Not counting the resource usage (which was pretty low) the test shows an 800% increase in the amount of data transferred.

Note: It is not only by using cache that Sucuri Firewall handles DDoS attacks. Sucuri has sophisticated algorithms that can detect and block all sorts of DDoS. When you are under Sucuri protection, this is done a million times everyday. You will only realize this when you read the Report section of the Firewall dashboard.

Caching Plugins

You may be asking yourself,  “Could I use a cache plugin to get the same result?”

For the purpose of this test, we installed the popular WP Super Cache plugin, but you can get similar results using other cache plugins.

Test #5 – Plugin Caching – 20 Concurrent Connections

With the recommended WP Super Cache settings enabled, the WordPress server was hit again:

Great results comparing to the non-cached WordPress test. On the other hand, the resource usage is still high for 20 concurrent users:

After the dust has settled a bit and the cache was flushed, we tried again.

Test #6 – Firewall and Plugin Caching – 20 Concurrent Connections

This time, we combined the firewall and cache plugin.

The WordPress server is just chilling. No significant resource usage:

All right, 20 concurrent users is a piece of cake. It’s time to try something more exciting.

Test #7 – Firewall and Plugin Caching – 100 Concurrent Connections

With 100 concurrent users (still within a single minute), here are the server results with a cache plugin in place:

The WordPress server had a lot more work to do:

With the same concurrency, it’s the firewall’s turn to do all the work:

You can imagine how calm the WordPress server was when the Firewall handled the traffic:

Even with the best cache configuration and all the sysadmin magic – Sucuri Firewall always handled the web traffic better than a single server.

Security Benefits of CDN Caching

DoS/DDoS attacks are usually a pipe battle when they are volumetric. On the 100 concurrent test (without the firewall) the test reached 2.5 MB/sec of IPv4 traffic. This means that the test was basically a 20 Mbps attack.

Given the fact that most hosting providers wouldn’t allow you to use that kind of network bandwidth, a single test server could seriously harm your website. For customers of the Sucuri Firewall, you can rest assured that you are protected against suspensions and high bandwidth charges from your hosting company.

What to Do When Caching Causes Problems

Sometimes caching can lead to trouble or isn’t ideal. We know that.

For example, some websites need to avoid caching sensitive content. Some people prefer not to use caching because they want to be sure their content is updated instantly.

The good news is that there are solutions to these problems.

For one – don’t set the cache mode to “Disabled” or exclude important pages under “Non-cache URL’s” – we offer several modes of caching to fit the needs of any website configuration. With the Sucuri Firewall, we offer a unique link you can bookmark (Clear Cache – API) and use anytime to clear the cache.

Get in touch with us and one of our analysts will gladly help you.

We also prepared two great articles in our knowledge base that can help you handle any cache headaches:

• https://kb.sucuri.net/cloudproxy/Troubleshooting/troubleshoot-cache-issues
• https://kb.sucuri.net/cloudproxy/Troubleshooting/investigating-cache-headers

Conclusion

Keep in mind that it was a one-minute HTTP flood with just one IP against a small NGINX VPS.

Picture that with 50 thousand IPs in a 24-hour timeframe against a shared hosting environment.

Do you want that headache? If you don’t, you should consider using the Sucuri Firewall.

When the cache is correctly set up, it could save your business and your reputation. Here’s some advantages you have when using the Firewall cache:

• Faster responses;
• More stability during peaks or attacks;
• Less CPU and memory usage;
• Better search engine rank;
• Less bandwidth usage.

Are you willing to throw those advantages away by choosing not to use caching?