Conversation

Notices

1. ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Monday, 18-Mar-2024 11:22:54 JST ✡️ ((fuggy)) :swastika:

   With how closed http://x.ai has been with the release of Grok and how they have treated past open sourcing efforts, I can confidently say that it's just a virtue signal for their court battle. They don't give two shits about open source

   • ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Monday, 18-Mar-2024 11:23:44 JST ✡️ ((fuggy)) :swastika:

     in reply to

     I'm betting the repo ceases activity after like a few months or weeks, it's a trend. Just look at their Github orgs

   • ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Monday, 18-Mar-2024 11:24:11 JST ✡️ ((fuggy)) :swastika:

     in reply to

     You cannot convince me for one second they are running code THIS old in production on Twitter lmao it's just used as a haha we better cuz algorithm open source (it's not)

   • Your New Kemono Waifu :verified: :cornbread_the_cat: (sjw@bae.st)'s status on Tuesday, 19-Mar-2024 07:26:34 JST Your New Kemono Waifu :verified: :cornbread_the_cat:

     in reply to
     @fuggy can I get an abridged version of what's going on?
   • ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Tuesday, 19-Mar-2024 07:27:17 JST ✡️ ((fuggy)) :swastika:

     in reply to

     • Your New Kemono Waifu :verified: :cornbread_the_cat:

     @sjw@bae.st
     
     UhhhElon Sues OpenAI claiming their not open
     People call Elon out for not having his models as open source (Grok)
     He says he will open source it in a week
     Everyone expects it to be 33B because on the website it said it was model based on Grok-0 which is 33B parameters
     Does
     Code is complete garbage and pretty much useless
     Code depends on having 8 GPUs even if you can run it on less :top_kek:
     Turns out It's 300B MoE model based on experts fine tuned from Grok-0
     Barley beats GPT-3 (175B) :pepe_lol:
     No one can run it because of that
     The weights are int8 instead of fp16 so that throws wrench in everything like fine tuning etc. Pretty sure
     Be me
     Open issue about releasing Grok-0 weights which would be 33B in theory
     It gets closed as not planned
     Literally no explanation
     Imb4 they just stop developing the related code just like they did with the "open source algorithm" and literally every other open source project under Twitter/X/x.ai GitHub org
     I ask them for at least explanation as to why they can't release Grok-0 weights
     They reopen the issue without explanation as wellHuffing hopium for now for Grok-0 but I still feel like it's virtue signal for their court case with OpenAI, I doubt they will release any other models after this

   • ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Tuesday, 19-Mar-2024 07:29:28 JST ✡️ ((fuggy)) :swastika:

     in reply to

     • Your New Kemono Waifu :verified: :cornbread_the_cat:

     @sjw@bae.st oh yeah and no instruct tuned model good luck fine tuning 300B MoE model with no paper or anything or instructions

   • ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Tuesday, 19-Mar-2024 07:31:24 JST ✡️ ((fuggy)) :swastika:

     in reply to

     • Your New Kemono Waifu :verified: :cornbread_the_cat:

     @sjw@bae.st idk maybe I am being a Karen but seems like a complete botched release

   • ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Tuesday, 19-Mar-2024 07:34:07 JST ✡️ ((fuggy)) :swastika:

     in reply to

     • Your New Kemono Waifu :verified: :cornbread_the_cat:

     @sjw@bae.st also I mentioned how their previous open source efforts where shit, they release them then never update them unless they are running year old code in production. Meanwhile Elon still uses it as a gotcha against his competitors even though we know nothing of the current algorithm in production clearly

   • ✡️ ((fuggy)) :swastika: (fuggy@skippers-bin.com)'s status on Tuesday, 19-Mar-2024 08:06:12 JST ✡️ ((fuggy)) :swastika:

     in reply to

     • Your New Kemono Waifu :verified: :cornbread_the_cat:

     @sjw@bae.st fp16 is more useful since we could then turn it back into int8 and lower etc. Also what is used for training I think

   • Your New Kemono Waifu :verified: :cornbread_the_cat: (sjw@bae.st)'s status on Tuesday, 19-Mar-2024 08:06:13 JST Your New Kemono Waifu :verified: :cornbread_the_cat:

     in reply to
     @fuggy Wouldn't using int8 if you can increase performance?
     
     The thing about the 8 GPUs is just bad programming. I understand how it happened. They just built software for their hardware platform. However if they were to ever change hardware they'd have to rewrite large chunks of their code. Better to just do it correctly from the start.