What are the best web scraping libraries with client-side JavaScript support?

Question

DevelopmentFrontend DevelopmentWeb

What are the best web scraping libraries with client-side JavaScript support?

11

Options
Considered

18

User
Recs.

Aug 21, 2022

Last
Updated

7 Options Considered

Best web scraping libraries with client-side JavaScript support	Price	Platforms	License
-- CasperJS	-	-	-
-- Puppeteer	Free	-	-
-- PhantomJS	-	-	-
-- Power Query Library (in Microsoft Excel)	-	-	-
-- Playwright	Free	Windows, Linux, Mac	-

See Full List

CasperJS

Built By the Slant team

Find the best product instantly.

4.7 star rating

Try it now - it's free

score 7 · Answer 1

Puppeteer

Top Pro

Actively developed and promoted by Google

Top Con

Need to use with javascript

No library to common from other language like python See More

Top Pro

Runs a real browser

Unlike the other options here, this is a real browser, just without the GUI parts. This means the quality of the test is much higher, and lets you do things like save to PDF or images. See More

Top Con

Not good for cross-browser tests

Since it only tests via Chromium, it won't help you test inconsistencies between browsers like Edge or Safari. There are projects to get around this, but they aren't mature. See More

Top Pro

Works great with modern node.js features

See All

score 1 · Answer 2

PhantomJS

Top Con

Not built for web scraping

Top Pro

Active development

Development activity and number of contributors is healthy. See More

Top Pro

Built on WebKit

WebKit is becoming the gold standard for browser compatibility, making it a good starting point for native headless browser testing. See More

See All

score 1 · Answer 3

Power Query Library (in Microsoft Excel)

Top Pro

Handles various data formats

Native support for JSON, XML, HTML, CSV, XLSX, and other formats. See More

Top Pro

Easy to use without programming experience

Power Query makes it easy to filter, combine, convert, and otherwise transform your data, no programming experience needed. See More

score 3 · Answer 4

Playwright

Top Pro

Choice of IDE

Work with Eclipse, WebStorm, IntelliJ, VSCode, or any other IDE with a choice of language (JavaScript, TypeScript, Java, Python, and .Net). See More

Top Con

New framework (small community)

Fabian Maume's Experience

New webscraping framework similar to puppetter See More

Specs

Platforms:Windows, Linux, Mac

Top Pro

Easy to learn coming from Puppeteer

See All

score 1 · Answer 5

Headless Chrome

Answer 6

node.js + jsdom + htmlparser2

Answer 7

7 Options Considered

My Recommendation for CasperJS

My Recommendation for CasperJS

My Recommendation for Puppeteer

My Recommendation for Puppeteer

Actively developed and promoted by Google

Need to use with javascript

Runs a real browser

Not good for cross-browser tests

Works great with modern node.js features

My Recommendation for PhantomJS

My Recommendation for PhantomJS

Not built for web scraping

Active development

Built on WebKit

My Recommendation for Power Query Library (in Microsoft Excel)

My Recommendation for Power Query Library (in Microsoft Excel)

Handles various data formats

Easy to use without programming experience

My Recommendation for Playwright

My Recommendation for Playwright

Choice of IDE

New framework (small community)

Easy to learn coming from Puppeteer

My Recommendation for Headless Chrome

My Recommendation for Headless Chrome

My Recommendation for node.js + jsdom + htmlparser2

My Recommendation for node.js + jsdom + htmlparser2

Don't see your favorite option? Add it.

My Recommendation for slim.js

My Recommendation for slim.js

My Recommendation for System.Web.UI.WebControls.WebControl

My Recommendation for System.Web.UI.WebControls.WebControl

My Recommendation for Slimer.JS

My Recommendation for Slimer.JS

My Recommendation for ProxyCrawl

My Recommendation for ProxyCrawl

One sec!