Julia Evans
today I'm thinking about how floating point numbers can be treacherous -- what are specific examples of when they've betrayed you?
so far I have:
* money calculations where you need to get the exact correct answer (you have to use integers, not floats)
* twitter's tweet IDs are bigger than 2^53 and can't be represented as a Javascript number, you have to use a string
I'm looking for specific examples of what code you were writing that broke, not general facts about floats
Nathaniel J. Smith
@b0rk The reason is, x86 has two different sets of floating point instructions: the old x87 fpu ones, and the newer SSE ones. The x87 instructions by default use 80 bit precision for internal calculations and values stored in registers, or you can set a magic config register to switch to 64 bit precision instead. This setting is per-thread, but global within that thread. SSE instructions are always 64 bit precision.
Nathaniel J. Smith
@b0rk As You Know, WSL 1 was an implementation of the Linux kernel abi on top of windows, to allow running Linux binaries. There's nothing special happening at the CPU level; you're still running native x86-64 code on the real x86-64 processor. It's just syscalls for io and stuff that are being emulated.
Nonetheless, musl's string->double code can give wrong results when running under wsl1.
０７６萌ＳＮＳ is a social network, courtesy of ０７６. It runs on GNU social, version 2.0.2-beta0, available under the GNU Affero General Public License.
All ０７６萌ＳＮＳ content and data are available under the Creative Commons Attribution 3.0 license.