Meta has revised its commercial terms to remove controversial language that legal experts warned allowed the company to scrape news websites for AI training while ignoring robots.txt blocking instructions. The update follows criticism from digital law experts and media outlets who viewed the previous phrasing as an effort to claim broad scraping rights.
Meta spokesperson Daniel Roberts clarified that the consolidated crawling terms applied exclusively to ads and business products rather than AI model scraping. Meta reiterated that content owners can continue using standard robots.txt files to signal crawling preferences.
Why it matters
Reaffirms standard web protocols like robots.txt as valid signals for blocking commercial AI scrapers.
Reduces immediate legal exposure for big tech firms surrounding unauthorized publisher data scraping.
Ensures publishers retain leverage in negotiating formal content licensing deals with frontier AI companies.
Source: theverge.com



