Cloudflare wants to help stomp out AI scraper bots for good

Date:
Thu, 04 Jul 2024 15:18:00 +0000

Description:
Cloudflare wants to make it easier (and more effective) to prevent AI bots from scraping your site with new, free tool.

FULL STORY ======================================================================

Cloudflare has announced a new feature that allows web hosting customers to block AI bots from scraping their website content without permission.

Companies responsible for AI tools , particularly those involved with training, have come under fire in recent months over their use of content published online.

Cloudflare says its new tool responds to widespread discontent among website owners and publishers about AI bots harvesting data and aims to protect content creators on the internet. Cloudflare tool to stop AI bot scraping

In its announcement, Cloudflare wrote: We hear clearly that customers dont want AI bots visiting their websites, and especially those that do so dishonestly. To help, weve added a brand new one-click to block all AI bots.

The tool has become available for all customers, including those on Cloudflares free tier.

Previously, website owners have been able to use the robots.txt file to present bots from crawling their sites, but to varying degrees of success.
The new tools offers a more robust defence against content scraping by using machine learning to identify and block bots.

Cloudflares system relies on digital fingerprinting to differentiate bots
from legitimate users.

The company said that the enhanced protection comes as a response to significant AI bot activity across its network. Bytespider, GPTBot and ClaudeBot were among the most commonly observed, but despite previous
blocking attempts, AI bots accessed more than one-third (39%) of the top one million sites served by Cloudflare in June.

Thankfully, the new feature is available to all Cloudflare customers, including those on the free tier. By navigating to the Security > Bots menu and blocking AI Scrapers and Crawlers, users can protect their website from unwanted AI bot activity. More from TechRadar Pro Microsoft AI CEO claims any content published online is "freeware" to help train AI and people are not happy Weve rounded up a list of all the best cloud hosting on offer today Check out the best AI writers around today



======================================================================
Link to news story: https://www.techradar.com/pro/cloudflare-wants-to-help-stomp-out-ai-scraper-bo ts-for-good


--- Mystic BBS v1.12 A47 (Linux/64)
* Origin: tqwNet Technology News (1337:1/100)