WebPagetest is a fantastic tool for measuring web page performance. But sometimes you have to be careful when using the public test agents. In particular the agents are often run on small machines such as the Amazon EC2 VMs. so if the page has a lot of JavaScript code running this might give a distorted picture of the actual performance.

The best way to assess the overall JavaScript performance of a browser is to run Google’s Octane 2.0 benchmark suite. It consists of 17 individual benchmarks, which are run and then the geometric average is calculated as the Octane score.

Instead of scripting the browser to click on the Start link, the easiest way to automatically start the test is to use a query string parameter like this:

http://chromium.github.io/octane/?auto=1

The only problem is that WebPagetest will stop the test before it has completed, but you can work around that by setting the “Mininum test duration” under the Advanced tab. A value of 80 seconds seems suitable for most agents, but if you get incomplete Octane results for some test locations you might have to increase this.

Here are some example measurements for European locations, collected by making 9 test runs (first+repeat view) and collecting the lowest and the highest scoring runs:

Test location

Browser

Lowest score

Highest score

Ireland - EC2

Chrome

7,623

8,182

Manchester, UK

Chrome

10,515

11,223

London, UK

Chrome

7,741

11,663

Amsterdam, NL – Go Daddy

Chrome

7,065

8,775

Amsterdam, NL – IISpeed

Chrome

12,957

17,260

Frankfurt, Germany

Chrome

7,555

8,260

Frankfurt, Germany – Dynatrace

Chrome

7,647

7,959

Germany - EC2

Chrome

7,604

8,154

Prague, Czech Republic

Chrome

19,172

22,183

Warsaw, Poland

Chrome

4,802

6,466

 

As you can see, the Prague location is actually the best performing one, followed by the IISpeed agent in Amsterdam.