Conversation

Notices

1. Perfect Devil Anna (amalthea@pl.noob.quest)'s status on Tuesday, 18-Feb-2025 02:49:20 JST Perfect Devil Anna
   does ryona agency fba respect robots.txt? has anyone looked into this? 🤔
   • Protoss (cyrillic@lab.nyanide.com)'s status on Tuesday, 18-Feb-2025 02:49:19 JST Protoss

     in reply to

     • 
     @amalthea I doubt it, @mint was also notorious for making private posts that federated visible
   • Johnny Peligro (mischievoustomato@tsundere.love)'s status on Tuesday, 18-Feb-2025 02:49:19 JST Johnny Peligro

     in reply to

     • 
     • Protoss
     @Cyrillic @amalthea @mint disrespecting robots.txt is kino
      likes this.
   •  (mint@ryona.agency)'s status on Tuesday, 18-Feb-2025 02:51:24 JST 

     in reply to

     • Protoss
     • Johnny Peligro
     @mischievoustomato @Cyrillic @amalthea It's only there to control crawling (recursively following links), FBA simply looks up the predefined API endpoints. Same reason why web.archive.org respects robots.txt but archive.fo (which only saves single pages upon user request) doesn't.
   • pomstan (pomstan@xn--p1abe3d.xn--80asehdb)'s status on Tuesday, 18-Feb-2025 13:33:34 JST pomstan

     in reply to

     • 
     • Protoss

     @Cyrillic @amalthea @mint lockfags deserve hanging

      likes this.