Trust Boundaries
← Latest writing
1 min readSecurity · Notes

Fun fact: the two teams of scraping

Fun fact: the interesting world of scraping.

Most big companies have some part of the business that depends on scraping data from other websites. At the same time, they invest in making sure their own data isn't scraped.

The first team's job is to find ways to defeat anti-bot measures like CAPTCHAs. The second team's job is to add friction that defeats bot traffic.

It's not hard to guess which team is better funded, and why the second team is wishful thinking when the goal is for the first one to succeed.