#scraping · Louche Cannon

AI's 'Creepy' Crawlers Criticized by Linux Foundation's IT Infrastructure Director - Slashdot

Wed, 02 Sep 2026 08:48:20 +0200

Bookmarked https://linux.slashdot.org/story/26/08/30/2235224/ais-creepy-crawlers-criticized-by-linux-foundations-it-infrastructure-director

GitHub - 0xchasercat/draco: Draco is a faster, cheaper and lighter firecrawl drop-in replacement that you own entirely, running on your machine, with a built in MCP, free websearch, and browserless JavaScript console. · GitHub

Mon, 03 Aug 2026 06:54:09 +0200

Bookmarked https://github.com/0xchasercat/draco/

Cloudflare blocks AI agents by default | Blog

Sat, 11 Jul 2026 15:17:30 +0200

Bookmarked https://tilion.dev/blog/cloudflare-blocks-agents

Pulpie: Pareto-Optimal Models for Cleaning the Web | Feyn

Mon, 06 Jul 2026 22:09:05 +0200

Bookmarked https://usefeyn.com/blog/pulpie-pareto-optimal-models-for-cleaning-the-web/

Firecrawl - The context API to search, scrape, and interact with the web at scale. 🔥

Sat, 04 Jul 2026 08:48:26 +0200

Bookmarked https://www.firecrawl.dev/

kage

Sat, 20 Jun 2026 22:27:18 +0200

Bookmarked https://kage.tamnd.com/

GitHub - saffron-health/libretto: The AI toolkit for building and maintaining browser automations · GitHub

Wed, 15 Apr 2026 19:08:35 +0200

Bookmarked https://github.com/saffron-health/libretto

magnolia1234/bypass-paywalls-chrome-clean

Tue, 14 Apr 2026 09:17:50 +0200

Bookmarked https://gitflic.ru/project/magnolia1234/bypass-paywalls-chrome-clean#list-of-supported-websites

ChatGPT Won't Let You Type Until Cloudflare Reads Your React State. I Decrypted the Program That Does It.

Mon, 30 Mar 2026 07:49:23 +0200

Bookmarked https://www.buchodi.com/chatgpt-wont-let-you-type-until-cloudflare-reads-your-react-state-i-decrypted-the-program-that-does-it/

D4Vinci/Scrapling: 🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

Thu, 26 Feb 2026 08:42:02 +0200

Bookmarked https://github.com/D4Vinci/Scrapling

internetarchive/brozzler: brozzler - distributed browser-based web crawler

Sun, 21 Dec 2025 13:19:36 +0200

Bookmarked https://github.com/internetarchive/brozzler

Common Crawl Is Doing the AI Industry’s Dirty Work - The Atlantic

Tue, 04 Nov 2025 17:40:38 +0200

Bookmarked https://www.theatlantic.com/technology/2025/11/common-crawl-ai-training-data/684567/

AI scrapers request commented scripts

Sat, 01 Nov 2025 12:46:54 +0200

Bookmarked https://cryptography.dog/blog/AI-scrapers-request-commented-scripts/

Aggressive bots ruined my weekend | Herman's blog

Wed, 29 Oct 2025 16:50:14 +0200

Bookmarked https://herman.bearblog.dev/agressive-bots/

Fastly warns AI bots can hit sites 39K times per minute • The Register

Thu, 21 Aug 2025 21:39:23 +0200

Bookmarked https://www.theregister.com/2025/08/21/ai_crawler_traffic/

Agents or Bots? Making Sense of AI on the Open Web

Wed, 06 Aug 2025 12:50:44 +0200

Bookmarked https://www.perplexity.ai/hub/blog/agents-or-bots-making-sense-of-ai-on-the-open-web

Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives

Mon, 04 Aug 2025 18:43:09 +0200

Bookmarked https://blog.cloudflare.com/perplexity-is-using-stealth-undeclared-crawlers-to-evade-website-no-crawl-directives/

trymeka/agent: state of the art browsing agent (WebArena 72.7%)

Thu, 31 Jul 2025 07:51:34 +0200

Bookmarked https://github.com/trymeka/agent

iocaine - the deadliest poison known to AI

Fri, 25 Jul 2025 14:34:50 +0200

Bookmarked https://iocaine.madhouse-project.org/

iocaine/iocaine: The deadliest poison known to AI. - MadHouse Git Repositories

Sat, 12 Jul 2025 09:31:17 +0200

Bookmarked https://git.madhouse-project.org/iocaine/iocaine