articles

Benchmark of web-applications testing tools

  Benchmark of web-applications testing tools
Author Alhusaine NEMER – test’n’dev
Date 24/05/2023
Version 0.5
Licence CC BY-NC-ND 4.0


Introduction

In this article, we will compare some of popular tools used for web testing automation.

We will focus on tools having those characteristics:

We will begin with 4 tools (in alphabetical order):

  1. Cypress.io
  2. Nightwatch.js
  3. Playwright
  4. Selenium WebDriver

Indeed, in this article, and to be fair in the comparison, we will take JavaScript/TypeScript versions of Playwright and Selenium.

Quick presentation of the compared tools

⚠️ to be completed

Popularity of the 4 tools

TL;DR:

To have an overview of “popularity” of each tool, we will use 4 indicators retrieved on NPM, GitHub, Twitter and Google.

As per end of may 2023:

indicator Cypress Selenium Playwright Nightwatch
Number of weekly downloads on NPM [^1] 4908680 2088040 1304640 180000
Number of stars on GitHub 43434 26651 51509 11370
Number of followers on Twitter 21000 23000 9000 3000
Interest over time on Google Trends 84,71 2,65 26,29 1,71

this graph shows the relative popularity of each tool, according to the previous indicators (NPM downloads, Github stars, Twitter followers, Google Trends).

See more details in “popularity” sub-page.

Technical comparison

aspect Cypress Nighwatch Playwright Selenium
Other than JS/TS? No No Yes (C#, Java, Python) Yes, many
Protocol Inside Browser WebDriver CDP (debug protocols) WebDriver
Open-source & supported/developped by… Cypress BrowserStack Microsoft Community

⚠️ to be completed

Installation 🚧

⚠️ to be completed

Performance 🚧

We will use our code hosted on testndev/web-app-testing-tools-benchmark repository for our technical benchmark.

⚠️ to be completed

See also

There is many articles that compare web-app test automation frameworks (especially Selenium vs Cypress vs Playwright since 2022), presenting their features, trade-offs, etc.

You can read for example:

Conclusion

⚠️ to be completed



Licence

Licence Creative Commons

This article is licensed under the terms of the Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License


Credits :


Photo Credits :