Conversation

Notices

1. ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Saturday, 09-Sep-2023 09:22:11 JST ✡️ ((fuggy)) :swastika:

   llama cpp or kobold cpp seem to be the best way to run LLM shit on AMD GPUs, this shit just werks and is so fast and 4bit works too!

   • ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Saturday, 09-Sep-2023 09:23:58 JST ✡️ ((fuggy)) :swastika:

     in reply to

     If you didn't know, they ported to ROCm recently. They also have CUDA support now, but I didn't ever try it