Benchmark of web-applications testing tools | |
---|---|
Author | Alhusaine NEMER – test’n’dev |
Date | 24/05/2023 |
Version | 0.5 |
Licence | CC BY-NC-ND 4.0 |
In this article, we will compare some of popular tools used for web testing automation.
We will focus on tools having those characteristics:
We will begin with 4 tools (in alphabetical order):
Indeed, in this article, and to be fair in the comparison, we will take JavaScript/TypeScript versions of Playwright and Selenium.
⚠️ to be completed
TL;DR:
To have an overview of “popularity” of each tool, we will use 4 indicators retrieved on NPM, GitHub, Twitter and Google.
As per end of may 2023:
indicator | Cypress | Selenium | Playwright | Nightwatch |
---|---|---|---|---|
Number of weekly downloads on NPM [^1] | 4908680 | 2088040 | 1304640 | 180000 |
Number of stars on GitHub | 43434 | 26651 | 51509 | 11370 |
Number of followers on Twitter | 21000 | 23000 | 9000 | 3000 |
Interest over time on Google Trends | 84,71 | 2,65 | 26,29 | 1,71 |
this graph shows the relative popularity of each tool, according to the previous indicators (NPM downloads, Github stars, Twitter followers, Google Trends).
See more details in “popularity” sub-page.
aspect | Cypress | Nighwatch | Playwright | Selenium |
---|---|---|---|---|
Other than JS/TS? | No | No | Yes (C#, Java, Python) | Yes, many |
Protocol | Inside Browser | WebDriver | CDP (debug protocols) | WebDriver |
Open-source & supported/developped by… | Cypress | BrowserStack | Microsoft | Community |
⚠️ to be completed
⚠️ to be completed
We will use our code hosted on testndev/web-app-testing-tools-benchmark
repository for our technical benchmark.
⚠️ to be completed
There is many articles that compare web-app test automation frameworks (especially Selenium vs Cypress vs Playwright since 2022), presenting their features, trade-offs, etc.
You can read for example:
(in english, 24/04/2023, by Andrew Knight)
(in french, 13/03/2023, by Mr Suricate)
(in french, 31/01/2023, by Jean-François Greffier)
(in english, 04/03/2021 by Giovanni Rago)
⚠️ to be completed
Licence
This article is licensed under the terms of the Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License
Credits :
…
Photo Credits :