Conversation
Notices
-
:hacker_f::hacker_s::hacker_e: :hackerman::elliot::alexjones::terryapproves::7eleven::nixonsmug::feynmansmug2:
- † top dog :pedomustdie: likes this.
- † top dog :pedomustdie: repeated this.
-
The picture where I'm pointing, that's where everyone's posts live.
Then the innards. There are two tiers, CPUs 3 and 4 as well as RAM banks C and D are on the top tier, which plugs into the motherboard and is held in place by a latch. The bottom tier is where the notorious "Bank 18" lives, which is the location of B7. B11 is sitting useless, because the RAM is installed in pairs.
Hilariously, I left the ethernet cables plugged in and yanked the heads off them when I pulled the machine out of the rack. I also picked up the wrong USB serial cable and thus could not plug the DevTerm into the server and had to wheel in a crash cart.
Once the box was back up, I could ssh in and start everything back up. They have guest wifi at the datacenter, so it was possible to shut it down without tethering, but to bring everything back up, I did that on the way home.
824392CE-63D5-4405-856D-AD5659DC4963.jpeg
ED525B3C-9294-4FCF-B92C-1A3410A3344A.jpeg
6ACC8B98-90D0-431F-968B-4B707AE47775.jpeg
E7D2C023-37AF-43BD-A1EF-A7A47ED6FC47.jpeg
-
Here's the troublesome stick, the source of all of the problems. What should I do with it?
489214F5-4783-4035-86D6-A0B4302C009F.jpeg
-
@nukie Sharp edges, plus my ass is beautiful.
-
@p
-
@p Torch it, they must not know whats on it.....
-
@sysrq
:dracula: We out here snorting that refurbished server defective RAM.
:dracula: We're smoking that decapped ethernet cable zaza.
:dracula: This shit ain't nothing to me, man.
:richter: ...
-
@p
Microwave it, grind it up, snort it.
-
@p @p
god hates fse.jpg
-
@p All right, plan before replacing the fucked PSU and the MCE generator impersonating a stick of RAM was to ditch the VMs and just move everything onto the host machine.
That's my last idea short of replacing the entire box.
-
@p Time and effort I spend keeping FSE alive, maintaining this piece of shit box and throwing cash at the datacenter and replacement parts, and fixing shit in the middle of the night and waking up and the box has failed and I just drove down there and popped that stick of RAM and thought, "All right, that's the last problem solved. It'll be stable and I can spend my time half on making money and half on finishing Revolver." and here we are again. Box has locked up. Nothing in the IPMI, I expect. I just drove there in the middle of the night to fix the Last Problem and now I gotta file a ticket to get the thing restarted.
Back to job-work in about four hours and I gotta stay up and deal with this box.
I should just take the last backup I have and load that into Revolver, point everything there, and spend the time cramming ActivityPub features and UI endpoints into Revolver instead of maintaining this machine. The problem with that plan is that I committed to keeping it running until Revolver was ready, but if it's a couple of weeks of downtime to get to parity, maybe that's worth the couple of weeks not having to deal with that box. The cost there is all the DMs go away, I'm not putting those into Revolver, plus some media, like there's the old backups but once Revolver's data storage was stable enough, I stopped backing up the uploads and let media.freespeechextremist.com populate the old stuff organically, so there's no guarantee about the size of the gap there, but the missing media would be stuff that no one looked at between the last backup and now, and that's in the off-chance that I can't actually get the old stuff out. Once there's enough stuff implemented that's operable (there are a lot of parts that really aren't ready, so there would be hacks), I could rig enough shit up to probably let people log in with their old passwords and get their accounts live again. Maybe it'd not even require a login, like I have the old DB with tokens and everything, I could generate the keys, map them to local user IDs, then people that are logged in would get greeted with, until the UI endpoints are done, probably just "Here, download your private key, you'll need it later." That actually seems doable. Shitty "user experience", but the box is up and down, it's already a shitty situation.
There are a lot of parts of Revolver that just aren't finished, there are parts that need to be ripped out and replaced, there are parts that are currently *half*-replaced (like the REPL), and there are parts that only work on Plan 9 for right now (all trivial to turn portable, I think, just I haven't done it), but as evidenced by the media server exceeding expectations for handling the load of all the uploads, the base is pretty stable and reliable. (I mean, that's the media server, though. That is a simpler task than handling *all* of the stuff Pleroma was doing. But the only time that thing crashed was when IPFS choked all the RAM out of the Frantech VM it was running on--not hard at 4GB. It hasn't fallen over since I just killed IPFS on that box.) Having to hack fast to get some semblance of parity would destabilize it, sure, but I think I could keep up just fine and the upside is that any errors happening would be bugs to fix in Revolver rather than bailing water trying to keep that machine on life support, which would definitely improve my morale.
On the other hand the service abruptly disappearing would be an inconvenience for most of the people on FSE. We get a phoenix rising from the ashes, but it's a half-retarded phoenix for a while and it takes a couple of weeks.
-
@p GOD :FUCKING DAMN IT
-
@mint @p
-
@p @p Just get a machine with a working CD-ROM tray (I sure you have it laying around somewhere), glue a needle to it and place it right next to server's reset button, then make a cron job on that machine to eject it every N hours.
-
@p I think that is probably the way to go. New box is expensive and will not be ready for a while (read: a couple of paychecks from now, probably longer than it would take to kick Revolver out the door). Keeping this machine running is exhausting and untenable. Additionally, I swore repeatedly that I would not hand FSE off to someone so getting someone else and even if I hadn't promised, that I will not hand it off. For the trouble of a move to new infra, it'd be better to just move the data to Revolver.
I understand that this sucks for people using FSE. My apologies. I think the best way to bring FSE back online is to replace it with the thing that can't be killed, and restore functionality as quickly as possible. If you used FSE and have words/thoughts/suggestions, feel free to let me know (if you have a different username wherever you are now, please mention your username on FSE; I am tired and distracted and may not be able to tell).
I think the best course of action at present is finishing the thing that solves this problem permanently, so that's what I'm going to do unless I hear a better idea from someone with a stake in it. I've got an obligation and I will do my damnedest: the point of FSE was to provide a place for you, I said I would, and it evaporated. So if there's anything I can do (e.g., your follower/block lists for wherever you are staying right now, or just a ping so I know where to let you know when it's back up and running), or if you have any ideas or concerns about how I'm rebuilding the place, or if you want help setting up your own instance, whatever the case, just let me know.
If you had a VM on that box and you want the data out, let me know over DMs and we'll figure something out.
As much as I appreciate :moon: letting me crash on his couch while my house is on fire, so as to avoid putting him out, I might hop to a temporary single-user something or another. The drama club blew through last time I was here and I don't want to shit this place up telling them to fuck off or waste my own time dealing with them, but I also don't want to miss messages from actual people. Maybe I'll just be on IRC more and refuse to talk about anything here besides FSE/Revolver.
-
@mint @p This is a good idea, but I don't think the datacenter would let me do that, and it's worse than you might imagine: resetting leaves the thing wedged, it has to do a full power-cycle in order to come back to life.
-
@p @p Predictable problem is not a problem. Adapt and overcome.
-
@mint @p The adaptation is mainly "get some new hardware, this computer is busted". While I am waiting to be able to do that, I'll just resurrect FSE as a Revolver instance.
-
@amerika @mint @p Ha, that would be cool but FSE cannot take donations. Priced it out and I think what I'm gonna do is get Revolver to a state where it can handle 70-80% of what FSE was doing, and then resurrect it like that. Then there's an army of identical nodes that cannot be killed.
-
@p @mint @p
Sorry to hear it man. Is it time to warm up a donations drive?
-
@p @p @mint @amerika I hope you can code it fast enough, hope it works out.
-
@dcc @p @mint @amerika :bigbosssalute: I'm a hacker, sir. I do my job.
-
@amerika @mint @p Revolver: the great equalizer.
-
@p @mint @p
"Its name is Legion, for it is many."