Conversation

Notices

1. nixCraft 🐧 (nixcraft@mastodon.social)'s status on Saturday, 30-Sep-2023 05:44:51 JST nixCraft 🐧

   Add OpenAI, Google, and Common Crawl to your robots.txt to block generative AI from stealing content and profiting from it. See https://www.cyberciti.biz/web-developer/block-openai-bard-bing-ai-crawler-bots-using-robots-txt-file/ for more info and firewall to block those and other AI bots too.

   • Roadskater, Ph.D. (roadskater@mastodon.social)'s status on Saturday, 30-Sep-2023 05:50:31 JST Roadskater, Ph.D.

     in reply to

     @nixCraft And if one has control of the server, look into installing a port-blocking tool such as fail2ban that kicks in if it spots unwanted activity. I've seen too many bots that ignore the robots.txt file

   • YamakaziTaiga (yamakazitaiga@mastodon.social)'s status on Saturday, 30-Sep-2023 08:18:43 JST YamakaziTaiga

     in reply to

     @nixCraft So 🤔 not adding your two cents?