Browser Benchmark Wars: How does Firefox 4 Beta 9 stand up?

Posted by Bradley Wint on 17/01/2011

With Firefox 4 beta 9 impressing the crowds, we decided to do a benchmark test of all the major browsers on the market to see who comes out on top.

The contenders in this event include Firefox 4.0 B9, Firefox 3.6.13, Opera 11.00, Chrome 8.0.552.237, Safari 5.0.2 and Internet Explorer Beta 9. We did many tests on the JavaScript and memory management of each browser. We also initially decided to test CSS load times and site load times, but found the differences to be insignificant with too high a margin of error. At the end of the day JavaScript is responsible for a number of operations on any site including display of ads, call functions from other sites, some graphical management and much more. Programmed right, it can do pretty much anything these days. JavaScript can also be very heavy on a browser if the engine cannot properly manage it.

The system used to conduct these tests on was a Dell XPS 164 with a 2.8Ghz Duo-Core processor, 6GB DDR3 RAM and a 512MB ATI Radeon HD 3670 mobility card. The benchmark results will obviously vary from computer to computer depending on its overall computing and graphical power, but that does not take away from the validity of these tests. All non-essential plugins were turned off during the test. Finally no one test can can be compared with the other, since they test different aspects and may have opposing results.

With regard to IE beta 9, it still seems to be rather buggy with some functions just not working all together, so the results may need a bit of deviation (not sure how much) to compensate for the lack of processing of some JS functions.

SunSpider JavaScript Benchmark

This is the industry’s best JS benchmark tool because it tests a wide variety of JavaScript functions rather than focusing on one particular area. Also it tests on the JS platform and not other elements such as DOM, showing the browser’s true JavsScript potential. The lower the score, the better.

Firefox 4 beta 9 wins this category, showing the true potential of its JägerMonkey JavaScript engine. Not surprisingly, Opera and Chrome are just behind with a score of 307 and 309ms respectively. Firefox 3.6.13 seems to be the only browser that does not do well on this test, which explains why it sometimes can be laggy in everyday browsing scenarios.

Kraken JavaScript Benchmark

This test is a modified version of the SunSpider tool with enhancements. It is much more intensive as well. Once again, the lower the better.

Firefox 4 beta 9 was the clear winner here. All browsers seems to function well during the test but IE slightly froze a number of times, hence its poor results.

V8 JavaScript Benchmark, Version 6

This script comprises of a number of various JS test modules developed by various coders. The separate scores are then compiled and averaged (on a proportion basis) to give a final overall score. This script also focuses only on the JS aspect of the browser. The higher the score is, the better.

Chrome beats the competition coming out on top with score of 4594, while Firefox 4 beta 9 comes in second much lower at 3304. Shamefully, Firefox 3.6.13 comes in at a very low 493. Even Internet Explorer beats it at 1218. During this test, Firefox 3 basically became unresponsive at one point and displayed the JS force close option, but I continued with it till the test was done, which may explain why it crashes so often in real life. IE9 also displayed some sticky behaviour throughout the test.

Peacekeeper

Peacekeeper measures your browser’s performance by testing its JavaScript functionality. By measuring a browser’s ability to handle commonly used JavaScript functions Peacekeeper can evaluate its performance. Peacekeeper focuses more on the visual rendering aspect of JS, so the scores may go in a completely different direction here. The higher the score, the better.

Chrome comes out on top here followed behind by Opera 11. Not surprising. On the site, Opera 11 seemed to be the highest performer among all the aggregated scores. Note that this test also depends on the tester’s GPU and CPU capabilities, so the scores can go any way depending on the machine. IE9 unfortunately just didn’t work, I am guessing because of incompatibility with the test. Firefox 4 does not do too well, even though Mozilla pretty much boasted about its 3D and graphical capabilities.

Memory – 1 Tab Open

Moving on to a basic memory test, we wanted to see how much active memory it used after a 5 minute period with one tab open. We left it for 5 minutes so it could get past that cold start period and settle down. We loaded up our site for the test.

Disappointingly, Firefox 4 does not do well. It used 136MB of active RAM while Opera surprisingly came in at 111.2MB. IE9 used the lowest but some of the functions seem to just not work so we are not giving it full credit just yet. Otherwise, Safari seems to be the winner here based on it being a stable release. Chrome is not far behind.

Memory – 5 Tabs Open

We opened 5 tabs at let it sit for 5 minutes to stabilize. The sites we used were BlogTechnical, TechSpy, Newgrounds, BBC and Facebook (signed in home page).

Once again, Firefox 4 pretty much lost. Firefox 3 did pretty well in comparison to Chrome though. Browsers like Chrome and IE9 cannot be compared directly to Firefox and others because they split up their processes on a per tab basis. Each tab has its own process, so if one tab happens to crash, it doesn’t affect the others. In total, it may use a lot of RAM but by itself, it is very light.

Final Thoughts

Working backwards, Firefox 4 definitely doesn’t solve its RAM problems and can pose a problem for those who leave their browsers open for very extended periods of time. If it keeps adding more active memory over time like Firefox 3, then it may get very laggy at some point. Chrome and IE9 may use a lot of memory, but because of the split processes, the load per tab is significantly less thus preventing the chance of a crash and if it does crash, it protects the other tabs from fall prey to force closing. Firefox also needs to work on the 3D JS management because Chrome and Opera are still miles ahead.

Talking about the overall JS functions though, Firefox 4′s new engine definitely has shown its potential by winning the first 2 tests and coming in a good spot for the 3rd.

To say there is a clear winner here would be very difficult but the main battle stood between Chrome, Firefox and Opera with each of them excelling in various arenas. Firefox may be a very popular browser but if they don’t watch their backs, Chrome will easily be knocking on their doorstep. Firefox has always been popular because of its extensive list of add-ons but with most of them now being ported to Chrome it may be at risk of staggering on the market.

JavaScript has become quite an important tool over the past year with many sites using different forms of it, especially AJAX to improve users’ browsing experience. CSS is obviously important too but because the results fluctuated so much, it was hard to say how well they performed side by side.

The sad thing though is that users may get completely different experiences on their own systems, so it’s about finding which browser works best for you. All the browsers tested here seem to have a very similar visual interface, and the times have come now where the most popular browser will come down to which performs best over which offers more features.

Hope you all enjoyed.