Conversation

Notices

1. nixCraft 🐧 (nixcraft@mastodon.social)'s status on Saturday, 25-May-2024 22:48:08 JST nixCraft 🐧

   Is anyone surprised? By definition LLM can’t be 100% correct and LLM hallucination poses significant challenges in generating accurate and reliable responses. ChatGPT Answers Programming Questions Incorrectly 52% of the Time: Study. To make matters worse, programmers in the study would often overlook the misinformation. https://gizmodo.com/chatgpt-answers-wrong-programming-openai-52-study-1851499417

   • Pascal Leinert (pasci_lei@mastodon.social)'s status on Saturday, 25-May-2024 23:08:12 JST Pascal Leinert

     in reply to

     @nixCraft I just asked ChatGPT to create a Pong clone in Python, and it actually works. But I guess it's Pong, shouldn't be too complicated.

   • tqwhite (tqwhite@mastodon.social)'s status on Saturday, 25-May-2024 23:11:32 JST tqwhite

     in reply to

     @nixCraft 52%? No. That's completely wrong. I use GPT ten times a day for programming questions.

     Its accuracy is comparable to that of coworkers and my efforts using Google. IE, not perfect but usually eventually workable.

   • Mustardon 🇺🇸 (bitsnpieces@mastodon.social)'s status on Saturday, 25-May-2024 23:16:46 JST Mustardon 🇺🇸

     in reply to

     @nixCraft It's just the latest version of ELIZA.

   • Wilhelm (wbud@mastodon.social)'s status on Saturday, 25-May-2024 23:41:44 JST Wilhelm

     in reply to

     @nixCraft GitHub has started adding recommended fixes to security issues found by CodeQL and they're apparently generated by GitHub Copilot.

     The one it found recently in my repo was an unsanitized string being logged with NLog. The recommendation was to do String.Replace("\n", "") which is fine, but it didn't really explain why that specific fix was appropriate or sufficient.

     Don't like it.

   • Helpro (helpro@mastodon.social)'s status on Sunday, 26-May-2024 00:29:32 JST Helpro

     in reply to

     @nixCraft I heard a speech where the guy told us chatgpt invented an article when answering...

   • Corporation 9592 (eldubuu@mastodon.social)'s status on Sunday, 26-May-2024 01:32:16 JST Corporation 9592

     in reply to

     @nixCraft

     Thats about the same percentage as #StackOverflow & #Reddit, so yeah, that tracks….

   • SpaceJellyfish (spacejellyfish@mastodon.social)'s status on Sunday, 26-May-2024 02:10:13 JST SpaceJellyfish

     in reply to

     @nixCraft I am just a self-taught hobby tinkerer, nevertheless let me as blunt as possible:

     If any programmer has to revert to an algorithm to help with their task they should consider another waste of time than programming.

     Specifically when they are not even able to state their problem clearly and to the point.

   • Crafty (craftycat@mastodon.social)'s status on Sunday, 26-May-2024 03:43:41 JST Crafty

     in reply to

     @nixCraft Finally some numbers to back up what I've been feeling all along. I genuinely find very little use for LLMs in my work as a software dev, it often spits out complete bullshit, and/or sends my own unaltered code back at me with "i made these changes".

   • bimmer (bimmer@mastodon.social)'s status on Sunday, 26-May-2024 04:09:44 JST bimmer

     in reply to

     @nixCraft getting answers to programming questions is as simple as reading documentation :clippy:

   • nulll (nulll@mastodon.social)'s status on Sunday, 26-May-2024 14:29:53 JST nulll

     in reply to

     @nixCraft It's sad to see so many people trying to justify the "toy" by blaming the people who asked the question for the wrong answer provided by the AI.