Notices by ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com), page 7

@Soy_Magnus@detroitriotcity.com @burner@norwoodzero.net true

@mischievoustomato@rebased.taihou.website I want 128GBs of RAM but I still think I will run out somehow

@mischievoustomato@rebased.taihou.website once you go 32GBs+ RAM you can't go back

@mischievoustomato@rebased.taihou.website Well I have tried loading models even too big for my computer

@mischievoustomato@rebased.taihou.website No lol

@mischievoustomato@rebased.taihou.website Uhhh quit being poor I guess

@mischievoustomato@rebased.taihou.website I have 64 so I mostly just let it auto unload when it wants lol

You can copy my script I should likely just put in it's own file instead of writing it inside Nix lol

https://github.com/nonetrix/nixdots/blob/master/misc/shell-scripts/llm-actions.nix

@mischievoustomato@rebased.taihou.website If you have like exactly 32GBs of RAM you might want to unload it in the background, just make a dmenu script like I have and add something like this

curl http://localhost:11434/api/generate -d "{\"model\": \"''${model}\", \"keep_alive\": 0}"

Ignore the dumb escape sequence

@mischievoustomato@rebased.taihou.website I have autism please forgive

@mischievoustomato@rebased.taihou.website Oh never mind

@mischievoustomato@rebased.taihou.website I got it on a key binding even I can just bring it up with meta + a, also I got this stupid script

@mischievoustomato@rebased.taihou.website Yes it is a MoE model so it is super efficient on CPU, just be sure you have above 32 GBs RAM

@mischievoustomato@rebased.taihou.website No, use this https://ollama.com/ and https://ollama.com/library/mixtral

@burner@norwoodzero.net excuse

@burner@norwoodzero.net He disappeared into the void

@burner@norwoodzero.net You're definitely latinx

@burner@norwoodzero.net ESL

@burner@norwoodzero.net Only you implied that which tells me you actually do care more than anything, I didn't need to know that

@burner@norwoodzero.net I am laughing so hard right now (right)

@burner@norwoodzero.net Well I won't judge you for not know internet lingo really I guess