Archives
-
grab-site ( ArchiveTeam/grab-site )
The archivist’s web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns Created by ArchiveTeam on Feb 05, 2015.
-
browser-fingerprinting ( niespodd/browser-fingerprinting )
Analysis of Bot Protection systems with available countermeasures 🚿. How to defeat anti-bot system 👻 and get around browser fingerprinting scripts 🕵️♂️ when scraping the web? Created by niespodd on Jan 23, 2021.
-
glyphhanger ( filamentgroup/glyphhanger )
Your web font utility belt. It can subset web fonts. It can find unicode-ranges for you automatically. It makes julienne fries. Created by filamentgroup on Jan 06, 2017.
-
Crawler-Detect ( JayBizzle/Crawler-Detect )
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent Created by JayBizzle on Mar 23, 2015.
-
awesome-web-scraping ( lorien/awesome-web-scraping )
List of libraries, tools and APIs for web scraping and data processing. Created by lorien on Aug 12, 2015.