Conversation

Notices

1. Lake Emotion (lakeemotion@freespeechextremist.com)'s status on Friday, 03-Nov-2023 12:11:19 JST Lake Emotion

   • 
   • pistolero :thispersondoesnotexist:
   • 7666
   • pwm
   Hello! I have much experience in desktop Linux and a little with servers, but I really feel very enthusiastic about system administration.
   
   The best way to learn is by homelabbing, so that's what I'm doing. But I don't like the way I do it. Currently, I have one hardware machine without any containers, VMs, firewalls (except iptables) and so on. And most important, I don't have any documentation or configuration management (like Ansible or Puppet). And the more services I deploy on my machine, the harder it becomes to maintain.
   
   So, I want to ask experienced system administrators of fedi: how do you do that stuff?
   Do you use any configuration management engines, like Ansible? Why not? I feel it's not an "old school" way, and my inner retrograde says it is bad.
   Should I use a separate VM for each service? Do containers make any sense?
   What are the best practices to make the system more maintainable?
   
   I know it's a very complex and extensive question... So, maybe there's some good literature on system administration?
   
   Pinging people with AFAIK home-hosted instances: @mint @p @7666 @pwm
   
   Sorry if I bother you.
   •  likes this.
   • † top dog :pedomustdie: (dcc@annihilation.social)'s status on Friday, 03-Nov-2023 12:11:14 JST † top dog :pedomustdie:

     in reply to

     • 
     • pistolero :thispersondoesnotexist:
     • 7666
     • pwm
     @LakeEmotion @7666 @pwm @p @mint I host my servers at home, you dont need containers, ansible, etc. Set things up right and also have backups.
      likes this.
   • 7666 (7666@comp.lain.la)'s status on Friday, 03-Nov-2023 12:11:26 JST 7666

     in reply to

     • 
     • pistolero :thispersondoesnotexist:
     • pwm
     @LakeEmotion @pwm @p @mint No, yes, no.
     
     Maintainable? Just set it up right the first time.
      and † top dog :pedomustdie: like this.
   •  (mint@ryona.agency)'s status on Friday, 03-Nov-2023 18:43:36 JST 

     in reply to

     • pistolero :thispersondoesnotexist:
     • 7666
     • pwm
     @LakeEmotion @7666 @pwm @p Nothing wrong with setting everything up by hand, especially if it's a hobby project. The pet/cattle analogy above works well.
     I'd recommend getting a cheap VPS and using it as a reverse proxy for the homeserver. This will hide your home IP (as long as you make it into a proper VPN, routing outgoing traffic over this VPS as well) and might also speed things up if you cache and serve static content (images, emojis, frontend) directly from it.
     † top dog :pedomustdie: likes this.
   • pwm (pwm@crlf.ninja)'s status on Friday, 03-Nov-2023 18:44:27 JST pwm

     in reply to

     • 
     • pistolero :thispersondoesnotexist:
     • 7666
     @LakeEmotion @7666 @p @mint I am home hosted with a vps edge.
     I use proxmox with a bunch of lxc containers.
     None of it is hard buy it is probably all silly.
      likes this.
   •  (mint@ryona.agency)'s status on Friday, 03-Nov-2023 21:20:33 JST 

     in reply to

     • pistolero :thispersondoesnotexist:
     • 7666
     • pwm
     @LakeEmotion @7666 @pwm @p This hasn't happened yet to agency, and I've been hosting it for year and a half. Same with FSE which has been around for thrice as long. I won't worry about it as long as I'm not trying to present some commercial, enterprise-grade service.
   • Lake Emotion (lakeemotion@freespeechextremist.com)'s status on Friday, 03-Nov-2023 21:20:34 JST Lake Emotion

     in reply to

     • 
     • pistolero :thispersondoesnotexist:
     • 7666
     • pwm
     @mint @7666 @p @pwm but what if with time my pets will become a cattle? I will need to rework everything, then.
   • pistolero :thispersondoesnotexist: (p@freespeechextremist.com)'s status on Saturday, 04-Nov-2023 07:26:24 JST pistolero :thispersondoesnotexist:

     in reply to

     • 
     • 7666
     • pwm
     @LakeEmotion @mint @7666 @pwm
     
     > but what if with time my pets will become a cattle? I will need to rework everything, then.
     
     You don't have cattle yet, you a pet. Don't neglect real pets for hypothetical cattle. If you wanna learn how to do Ansible stuff and you use a fedi instance as a practical project, that's fine, but it's not something you need now with one machine and might not ever be something you need. You need to know what the load is gonna look like.
     
     A lot of things were built into Pleroma around some anticipated usage patterns that didn't turn out to happen, and some other usage patterns that ended up real didn't get accounted for. For example, hellthreads. Nobody expected the naturally occurring hellthreads to have as outsized an effect on the network as they did, let alone that there would be people that make hellthreads for fun. So there was an accidental DDoS: https://git.pleroma.social/pleroma/pleroma/-/issues/2765 . The problem was that a message gets rejected by the HellthreadMRF, but that message is a reply and has replies, so Pleroma queued those up to fetch. Since it dropped the original post, and it's a reply to the parent post, the parent post gets rejected and the first post is queued up again. So it walks up the chain and back down, repeatedly fetching the same objects, then rejecting them, and every instance that tried fetching that post did the same thing.
     
     It's very difficult to anticipate a problem like that: it's difficult enough to detect it. All I saw was that any time I clicked "Like" on a post in the hellthread, FSE started getting hammered, I realized it was my friends' instances that were doing it, this took a while to figure out that it was due to the length of the thread and the mismatch between FSE's HellthreadMRF threshold and the other instances', and then it took a while to verify this.
     
     Because instances tend towards idiosyncrasy, you're going to run into weird shit on a frequent basis and it's going to be difficult to generalize or anticipate. You want to just keep an eye on what is happening and whether or not the thing that is happening is causing a problem.
     † top dog :pedomustdie: likes this.