Conversation

Notices

1. Natalie Misskey (natalie@nya.social)'s status on Wednesday, 06-Dec-2023 20:17:41 JST Natalie Misskey

   • Dave Rand

   there is currently a bot inside MIT IP space, address 18[.]4[.]38[.]176, scanning fedi at large. i have confirmed this with 5+ unrelated instance admins, large and small instances, across mastodon/misskey/pleroma/akkoma.
   
   the bot is poorly behaved. i have observed it making repeated requests, multiple times per second, for the exact same paths (the paths being, generally: user profiles, specific posts, and sometimes following links in posts). returning 403s does not stop this activity. one of my domains received hundreds of additional requests despite replying with 403 to all of them. i have also seen it make requests for paths containing html tags - seems like a badly written parser. the purpose of these requests and what data is being gathered is unclear.
   
   PTR on the ip returns sts-drand03.mit.edu. a quick web search for "mit drand" brings back https://mitsloan.mit.edu/faculty/directory/david-g-rand and his personal website: https://davidrand-cooperation.com/ (note: other IPs in the /24 also have names in the PTR which match up with names of MIT faculty, but only the .176 IP appears to be involved in this activity).
   seems he's doing research into "misinformation" and "fake news" on social media. he also appears to be on fedi! so @Drand@techhub.social, given this activity is sourced from an IP with your name on it, could you share the purpose of this traffic? what data is being collected and how is it being used? do you plan to respect robots.txt or identify yourself in your useragent? is there a process for instance admins to opt out of this activity other than blocking the source IP?

   • cool_boy_mew likes this.
   • Machismo, cool_boy_mew and Token repeated this.
   • Moon (moon@shitposter.club)'s status on Wednesday, 06-Dec-2023 21:09:07 JST Moon

     in reply to

     • cool_boy_mew
     • Dave Rand
     @natalie @Drand this bot is remarkably poorly-behaved, it has scraped my user @coolboymew over 800 times in a day, among other users.
     cool_boy_mew likes this.
     Machismo and cool_boy_mew repeated this.
   • istván (istvan@noagendasocial.com)'s status on Wednesday, 06-Dec-2023 21:09:07 JST istván

     in reply to

     • cool_boy_mew
     • Moon
     • Dave Rand

     @Moon @natalie @coolboymew @Drand > it has scraped my user @coolboymew over 800 times in a day,

     He is DETERMINED not to miss a single waifu.

     cool_boy_mew and Token like this.
   • pistolero :thispersondoesnotexist: (p@freespeechextremist.com)'s status on Wednesday, 06-Dec-2023 23:27:36 JST pistolero :thispersondoesnotexist:

     in reply to

     • Dave Rand
     • boody
     @boody @Drand @natalie (USER WAS BANNED FOR THIS POST)
     userwascannedforthisroast.png
     Machismo likes this.
   • boody (boody@techhub.social)'s status on Wednesday, 06-Dec-2023 23:27:39 JST boody

     in reply to

     • Dave Rand

     @Drand @natalie I can't reply to you because, although you are happy to scrape my instance, you have chosen an instance that blocks me.

     I am skeptical that "researching content moderation policies" is an accurate way to describe what you are doing: I have not been asked about my content moderation policies. And how are you going to determine whether a given piece of content is in violation of an instance's policy if you don't even know whether the moderators have seen it? If you are interested in misinformation, you may wish to join or operate an instance that ensures that you can interact with the people you are exploiting for your career.

     Can you explain whether or not you told the IRB that you were going to reimburse me for my bandwidth overages? Also, when I am done going through my logs to finish figuring out whose posts you have scraped and I notify them, where should I send them to request the removal of their posts from your dataset without compromising their anonymity?

   • Dave Rand (drand@techhub.social)'s status on Wednesday, 06-Dec-2023 23:27:40 JST Dave Rand

     in reply to

     @natalie Hi all, apologies for this, I didn't realize the bot was being poorly behaved - we've now stopped it. In terms of why we were doing the scraping, we are doing research on how content moderation policies vary across servers, and how this can help inform the Fediverse more broadly about effective approaches to content moderation. You can get more of a sense of the kind of research we do here: https://docs.google.com/document/d/1k2D4zVqkSHB1M9wpXtAe3UzbeE0RPpD_E2UpaPf6Lds/edit?usp=sharing

     Sorry again about causing problems for folks! (And thanks to a couple of people for emailing me to let me know about this)

     Machismo repeated this.
   • ottpossum 🦦 :verified_paw:​ (dashie@nya.otter.sh)'s status on Wednesday, 06-Dec-2023 23:28:26 JST ottpossum 🦦 :verified_paw:​

     in reply to

     @natalie first access I have from this IP on my public instance is
     [24/Nov/2023:19:39:06 +0100] "GET /api/v1/instance HTTP/1.1"
     at ~4 minutes interval 3 times, then seems to have loaded some js/css/fonts the 25, scrapped the public timeline on the 28 multiple times and started scrapping users the 29
     more or less the same behavior on this instance

   • Natalie Misskey (natalie@nya.social)'s status on Wednesday, 06-Dec-2023 23:28:27 JST Natalie Misskey

     in reply to

     for those who have checked logs on their instances, could you share the dates when the activity started? on this instance, the first request i have is from 2023/11/29, steadily ramping up since then

     Machismo repeated this.
   • ~vala :verifiedsabakan: (gaige@neue.city)'s status on Wednesday, 06-Dec-2023 23:28:48 JST ~vala :verifiedsabakan:

     in reply to
     @natalie started here, according to my logs
     
     /var/log/nginx/access.log.1:18.4.38.176 - - [05/Dec/2023:16:09:56 +0100] "GET /users/gaige HTTP/1.1" 200 2573 "-" "Python-urllib/3.9"
     
     for some reason only scraped my account, and nothing else :02wut:
     Machismo repeated this.
   • ~vala :verifiedsabakan: (gaige@neue.city)'s status on Wednesday, 06-Dec-2023 23:28:53 JST ~vala :verifiedsabakan:

     in reply to

     • ~vala :verifiedsabakan:
     @natalie also, it seems to have completely ignored my peertube and funkwhale instances
   • alyx (alyx@frogs.lgbt)'s status on Wednesday, 06-Dec-2023 23:28:54 JST alyx

     in reply to

     @natalie I see as early as Nov 25th, but it's been mostly quiet other than a sizable burst in requests for the public timeline on that day

   • vaartis looking at the big sky (vaartis@pl.kotobank.ch)'s status on Wednesday, 06-Dec-2023 23:28:59 JST vaartis looking at the big sky

     in reply to
     @natalie earliest one for me is 02/Dec/2023:18:45:36 and it seemed to have stopped on 3rd after 3344 requests to my profile and the profile of the only other active user on my instance, the user agent was "Python-urllib/3.9". Actually, now it seems like they're trying to connect via IPv6 which I have explicitly disabled so nginx is giving them a connection refused error, clearly that does not stop them because the last request was about 2 hours ago
   • Dushman (dushman@den.raccoon.quest)'s status on Wednesday, 06-Dec-2023 23:29:09 JST Dushman

     in reply to

     • Dave Rand

     @natalie@nya.social @Drand@techhub.social
     Yeah I'm blocking this IP

   • Dushman (dushman@den.raccoon.quest)'s status on Wednesday, 06-Dec-2023 23:29:19 JST Dushman

     in reply to

     • Hyolobrika
     • Dave Rand

     @Hyolobrika@social.fbxl.net
     Oh that is not the point. Bitch is loading the server with thousands of requests. @Drand@techhub.social Choke on a fat cock.

   • Hyolobrika (hyolobrika@social.fbxl.net)'s status on Wednesday, 06-Dec-2023 23:29:21 JST Hyolobrika

     in reply to

     • Dushman
     • Dave Rand
     It probably won't help much. He can get your posts etc from other servers you federate to.
     Machismo repeated this.
   • pistolero :thispersondoesnotexist: (p@freespeechextremist.com)'s status on Wednesday, 06-Dec-2023 23:55:09 JST pistolero :thispersondoesnotexist:

     in reply to

     • cool_boy_mew
     • Moon
     • Piss pot shit in my eye
     • Dave Rand
     @Moon @natalie @Drand @coolboymew Same here. Incidentally, he has apparently screwed up some of his links, so his bot is fetching URLs like "GET /signin%3C/a%3E%3Cbr/%3E%3Cbr/%3EYang HTTP/1.1", "GET /about%3C/a%3E%3Cbr/%3ELooks HTTP/1.1", "/notice/ACV7evq9u0cd7bBo1Y%3C/a%3E".
     
     Incidentally, he was scraping some Spinster user's profile from FSE, @Piss_Ant.
     cool_boy_mew likes this.
   • cool_boy_mew (coolboymew@shitposter.club)'s status on Thursday, 07-Dec-2023 00:02:30 JST cool_boy_mew

     in reply to

     • Moon
     • Dave Rand
     @Moon @natalie @Drand bruh
     
     This reminds me I'm not posting enough futas for the nice people at the MIT
   • Tsuki (tsuki@amami.paradigm-x.tokyo)'s status on Thursday, 07-Dec-2023 00:08:04 JST Tsuki

     in reply to

     • cool_boy_mew
     • Moon
     • Dave Rand
     clearly he is a love live fan
     cool_boy_mew likes this.
   • Psychotic Militant Bitch (silverwolf@nicecrew.digital)'s status on Thursday, 07-Dec-2023 00:08:11 JST Psychotic Militant Bitch

     in reply to

     • cool_boy_mew
     • Moon
     • I am just normal /k/orean :verified:
     • istván
     • Dave Rand
     I wonder how often it goes after @justnormalkorean
     cool_boy_mew likes this.
   • :btrfly: anime graf mays 🛰️🪐 (graf@poa.st)'s status on Thursday, 07-Dec-2023 00:08:16 JST :btrfly: anime graf mays 🛰️🪐

     in reply to

     • cool_boy_mew
     • Moon
     • pistolero :thispersondoesnotexist:
     • Piss pot shit in my eye
     • Dave Rand
     @p @Moon @natalie @coolboymew @Piss_Ant @Drand the only links here was /users/username. no posts or anything
     cool_boy_mew likes this.
   • istván (istvan@noagendasocial.com)'s status on Thursday, 07-Dec-2023 00:13:07 JST istván

     in reply to

     • cool_boy_mew
     • Moon
     • I am just normal /k/orean :verified:
     • Psychotic Militant Bitch
     • Dave Rand

     @Silverwolf @natalie @justnormalkorean @coolboymew @Moon @Drand I can't imagine what bae.st is spending on bandwidth if he's going after the waifu bots.

     cool_boy_mew likes this.
   • laurel (laurel@freespeechextremist.com)'s status on Thursday, 07-Dec-2023 00:29:27 JST laurel

     in reply to

     • Dave Rand
     @Drand @natalie
     
     Carefull with that link.
     go*gle docs links leak your g*ail account if you logged in (in the case you are naive enough to have one)
     
     niggerniggerniggerniggernigger
     Meemoo likes this.
   • pistolero :thispersondoesnotexist: (p@freespeechextremist.com)'s status on Thursday, 07-Dec-2023 00:39:32 JST pistolero :thispersondoesnotexist:

     in reply to

     • :btrfly: anime graf mays 🛰️🪐
     • Dave Rand
     @natalie @Drand @graf He's been hammering the shit out of FSE, also.
     
     aaronsw killed himself because he bulk-scraped some academic papers from MIT and MIT had him arrested and the federal prosecutor wanted 35 years for exceeding authorized access under the CFAA. So, now, MIT faculty member David Rand is exceeding authorized access in order to produce more academic papers for MIT. I wonder: if he produces a paper by scraping and then someone scrapes that paper, what happens?
     
     > seems he's doing research into "misinformation" and "fake news" on social media.
     
     He's a Professor of Marketing in the Brain and Cognitive Sciences Department at Sloan. It is safe to reason that (1) he is a psychopath, (2) he has no idea how any of the technical shit works and has a grad student doing it, the grad student in turn having hired an engineer undergrad.
     
     > is there a process for instance admins to opt out of this activity
     
     I guess, per his faculty page, you could just call him, or perhaps email him. Despite scraping FSE, he has chosen an instance that blocks FSE.
     Token and † top dog :pedomustdie: like this.
   • anonymous (anonymous@freespeechextremist.com)'s status on Thursday, 07-Dec-2023 13:37:42 JST anonymous

     in reply to

     • pistolero :thispersondoesnotexist:
     • p

     @p @p I’d be willing to bet money that

     1. it’s going to be used as an excuse to expand the fediblock
     2. it’s going to be used as an excuse for more internet censorship
     Token and † top dog :pedomustdie: like this.
   • p (p@raru.re)'s status on Thursday, 07-Dec-2023 13:37:43 JST p

     in reply to

     • pistolero :thispersondoesnotexist:

     what's the bet he's mapping out the "threat landscape" in actuality?
     @p@freespeechextremist.com

   • CatSoc :nv: (catlord@poa.st)'s status on Friday, 08-Dec-2023 00:06:23 JST CatSoc :nv:

     in reply to

     • cool_boy_mew
     • Moon
     • istván
     • Dave Rand
     @istvan @Moon @natalie @coolboymew @Drand which is your favourite anime girl?
     cool_boy_mew likes this.
   • istván (istvan@noagendasocial.com)'s status on Friday, 08-Dec-2023 00:39:23 JST istván

     in reply to

     • cool_boy_mew
     • Moon
     • CatSoc :nv:
     • Dave Rand

     @CatLord @natalie @coolboymew @Moon @Drand

     cool_boy_mew likes this.