Notices by David Chisnall (*Now with 50% more sarcasm!*) (david_chisnall@infosec.exchange)

@wonderofscience What’s caring the shadows with soft edges? Most shadows in space have hard edges (which most SciFi CGI gets wrong) because there’s no air to diffuse the light. You can see that with the shadows the astronaut casts on the side of the station. But the ones from the station have soft edges, which I would expect to happen only if they are cast by something translucent. Maybe the edges of a solar panel?

@aral Coincidentally, I wrote to my MP the day before Starmer said this nonsense. I was writing specifically about the consultation on weakening UK copyright law to allow AI grifters to launder the commons with no repercussions, which is framed as 'we obviously want to give everything away and kill the British creative industry to make some Americans richer, help us find the most efficient way of doing that'.

The relevant part of my letter was:

Unfortunately, there seems to be a lack of understanding in government of how current 'AI' systems work. Announcements by members of the cabinet could easily be press releases from companies trying to sell things on the current hype wave. The only ray of sunshine has been the skepticism from the MOD. Much of the current hype wave surrounding generative AI is from companies run by the same people behind the Bitcoin / blockchain / web3 hype (which consumed a lot of energy, made the climate disaster worse, and failed to produce a single useful product).

There are a few places where machine learning techniques have huge value. Anomaly detection can be very useful for at-scale early diagnosis of various medical conditions, but this alone will not fix the NHS. Most of the hype has failed to create any products of real value. For example:

• 77% of employees report that using AI tools makes them less productive[1].

• A study on Google's own workers found that using AI tools made them less productive[2].

• OpenAI, the flagship company driving the hype wave is still making massive losses[3], including losing money on the $200/month subscription plan[4].

• Software written using AI has more security vulnerabilities[5]

It is not worth throwing the UK's creative sector under a bus to provide more money for these companies and their investors.

If you want a good overview of these problems, I'd recommend Pivot-to-AI[6] as a starting point. Beyond this, I'd also point out that OpenAI has been caught harvesting data from sites whose terms of use specifically prohibit it[7] (see also the LibGen article on Pivot-to-AI). Breaking the law should not be rewarded and no opt-out can work with people who do not follow the law. Opt in is the only viable solution.

The footnotes were:

[1] https://www.forbes.com/sites/torconstantino/2024/09/12/77-of-surveyed-employees-say-ai-tools-make-them-less-productive/
[2] https://redmonk.com/rstephens/2024/11/26/dora2024/
[3] https://www.cnbc.com/2024/09/27/openai-sees-5-billion-loss-this-year-on-3point7-billion-in-revenue.html
[4] https://9meters.com/technology/ai/openai-loses-money-on-200-month-pro-plan-because-people-are-using-it-too-much
[5] https://arxiv.org/html/2404.03823v1
[6] https://pivot-to-ai.com
[7] https://aoir.social/@aram/113811386580314915

I don't encourage people to send the same text to their MPs, because that gets ignored, but a set of citations like this may help push back.

A lot of the current hype around LLMs revolves around one core idea, which I blame on Star Trek:

Wouldn't it be cool if we could use natural language to control things?

The problem is that this is, at the fundamental level, a terrible idea.

There's a reason that mathematics doesn't use English. There's a reason that every professional field comes with its own flavour of jargon. There's a reason that contracts are written in legalese, not plain natural language. Natural language is really bad at being unambiguous.

When I was a small child, I thought that a mature civilisation would evolve two languages. A language of poetry, that was rich in metaphor and delighted in ambiguity, and a language of science that required more detail and actively avoided ambiguity. The latter would have no homophones, no homonyms, unambiguous grammar, and so on.

Programming languages, including the ad-hoc programming languages that we refer to as 'user interfaces' are all attempts to build languages like the latter. They allow the user to unambiguously express intent so that it can be carried out. Natural languages are not designed and end up being examples of the former.

When I interact with a tool, I want it to do what I tell it. If I am willing to restrict my use of natural language to a clear and unambiguous subset, I have defined a language that is easy for deterministic parsers to understand with a fraction of the energy requirement of a language model. If I am not, then I am expressing myself ambiguously and no amount of processing can possibly remove the ambiguity that is intrinsic in the source, except a complete, fully synchronised, model of my own mind that knows what I meant (and not what some other person saying the same thing at the same time might have meant).

The hard part of programming is not writing things in some language's syntax, it's expressing the problem in a way that lacks ambiguity. LLMs don't help here, they pick an arbitrary, nondeterministic, option for the ambiguous cases. In C, compilers do this for undefined behaviour and it is widely regarded as a disaster. LLMs are built entirely out of undefined behaviour.

There are use cases where getting it wrong is fine. Choosing a radio station or album to listen to while driving, for example. It is far better to sometimes listen to the wrong thing than to take your attention away from the road and interact with a richer UI for ten seconds. In situations where your hands are unavailable (for example, controlling non-critical equipment while performing surgery, or cooking), a natural-language interface is better than no interface. It's rarely, if ever, the best.

@foo @hacks4pancakes Are they the same recipe? Tim Tams are designed not to melt in Australian heat and so have a surprisingly high quality chocolate. You can tell the real ones because you can bite off diagonally opposite corners and then drink coffee using it as a straw. The biscuit will eventually become sodden and it will implode (forming a very tasty smaller chocolate snack) but the chocolate will not all melt off.

In the UK, we have Penguins, which look the same. Trying the same thing with a Penguin will not end well (an Aussie friend made me try this for science).

It’s not surprising that a society that values confidence more than competence would become excited by machines that generate text that sounds confident and is entirely wrong.

@aral @frechdachs It’s not a complex concept, it’s a complex thing to enshrine in legalese. And when you do, you are giving power to lawyers and people who can afford to pay lawyers, not to individuals.

If you want to peg a small tech thing to copyleft, that’s your choice. As someone who has written and released a few hundred thousand lines of code under permissive license, I won’t be joining in. I’ve seen people adopt the position that you are taking (that copyleft is the only true way) and it almost never achieves the goals that are stated.

You’re starting by saying that your licensing ideology is more pure than that of many successful projects. I don’t know how you expect that to build an inclusive environment.

@frechdachs @aral It’s also the point at which I realise that it’s not a crusade that I will be contributing to, which is a shame because I agree with all of the other points.

I’ve written a lot about why I don’t contribute to copyleft projects over the years, so I won’t repeat it all here, but at a high level, it boils down to two points:

Copyleft project licenses always include some conditions that are easy to accidentally violate. For example, if I compile a GPLv2 project and give a friend a binary so that they don’t have to build it themselves, I’ve violated the license. Will anyone take me to court? Almost certainly not, but now we’re in a land of selective enforcement and everyone needs to talk to lawyers to understand when they are compliant, not compliant but probably fine, or not complaint and at risk. I’ve spent a depressing amount of time talking to lawyers about the GPL and the number of things where the conclusion was that it was a technical violation but it would not be in the interests of anyone with standing to sue to take it to court was depressing.

I also contributed to a GPL’d FSF project where we had strong evidence of a company taking the code and distributing modified versions, but the FSF decided that it was too expensive to enforce the license. This is the worst situation because all of the legal restrictions apply to people acting in good faith but not to others because the cost of enforcement is too high.

More importantly, forcing people to cooperate via legal threat never works. People contribute changes to permissively licensed projects because they understand the value of the commons. I’ve seen far more proprietary software written as a result of the GPL than Free Software. Even small companies would rather reimplement something in house than understand the GPL in all of its intricate corner cases and so we end up with them providing no benefit to the commons. In contrast, the evolution I’ve seen many times with permissive software is as follows:

1. They take the code and maintain their own fork. Yay, the commons gives us free stuff!
2. They implement a new feature in their own fork. Yay, building on the commons lets us differentiate!
3. Upstream implements the same feature but a different way. Oh, private stuff on top of the commons is costing us,
4. They reevaluate how much of a competitive advantage they actually get. Hmm, maybe out 2% addition is not actually as valuable as we thought it was.
5. They developed their next new feature upstream and strive to have as small a diff as possible in house.

After step five, they are eager and willing contributors to the open ecosystem. With GPL’d projects, this doesn’t happen. They either do an in-house rewrite or they take GPL’d code and have compliance people deal with upstream and regard the license as a cost centre.

There are exceptions to this, but they tend to be projects that already grew to the size where a proprietary reimplementation is not feasible, such as Linux (and, even then, the level of grudging compliance from phone vendors, for example, shows you how little the GPL is actually helping).