The part I don't get is, is this here:
..where generateIEEE754 yields a (very, very small) floating point number. Why / how does this leak memory?
EDIT: got it, apparently it is a "CSS Font Face Parsing Type Confusion". That's wild. But this is only a read primitive, what then?
There are 2 webkit exploits in PS3HEN.
The one you mention which is a read primitive used to leak browser memory data. It's a powerful read primitive allowing you to leak the amount of bytes of data you specify from a memory offset you specify.
The other one is based on a vulnerability in the NaN code & it is used to trigger ROP execution via a parseFloat call..
When you provide a specifically crafted NaN value to parseFloat, webkit interpreter execution jumps to the offset described in that NaN value. It's as straightforward a code execution exploit as you can get.
So roughly speaking we first use the memory leak exploit to determine the offset of the custom ROP stack frames we planted in memory.
Once we have that offset, we craft an appropriate NaN value with it, pass it to parseFloat as an argument & the custom ROP stack frames get executed..
The last custom ROP stack frame takes care of making the execution jump back to the original execution line as if the parseFloat call execution had not been diverted & returned a valid value, you could call that a "graceful return" from ROP.
The vulnerabilities exploited in PS3HEN were already used in Vita's henkaku years ago. When esc0rtd3w & I started ps3xploit, xerpi had already ported the memory leak exploit to ps3 but nobody ever tried to leverage it. So we did & I ended up porting the NaN/parseFloat exploit implementation.
Technically speaking the old Vita henkaku webkit exploits & the PS3HEN webkit exploits are pretty much identical, it's mostly the memory layout & the ROP implementation that change.
It's the same thing with the kernel exploit, the ioctl exploit is an henkaku exploit developed for Vita which habib ported to ps3. At the time I wrote kernel ROP for that port.
Iirc there should be some write up available for all 3 vulnerabilities in henkaku documentation.
There is also a series of videos & posts on ps3 webkit exploitation in this forum, check the
tutorial thread started by esc0rtd3w, you should find the information useful. The series of videos was left unfinished due to the absence of interest.
For once someone was ready to teach basic console exploitation skills to anyone interested, providing write ups, videos & software to test with, not a single person took advantage of it afaik... Disappointing to say the least, especially when you consider how rarely such an opportunity arises.
I don't work on the PS3HEN releases, those releases still rely on code I wrote over 3 years ago for the old ps3xploit tools, code that I don't use anymore.
On the other hand, as I wanted full OFW support for my PS3 Toolset project, I couldn't use the parseFloat exploit so I replaced it with a Flash 9 exploit I wrote that's working on OFW.
You may wonder why Flash 9, why not stick to webkit? Because Flash 9 offers an ideal read/write primitive candidate with the bytearray actionscript object whereas the ps3 webkit doesn't offer any decent candidate objects, strings are immutable, standard arrays impractical, it doesn't have typed arrays (or any vector-like objects) like the Vita webkit has. Also, a Flash 9 exploit is less "restricted" than a webkit exploit, it can be used in both silk & silk webkit browsers as well as any other OS or game/app feature using Flash 9.
For the moment the webkit memory leak exploit still works great on OFW so I kept it in the PS3 Toolset project so as not to burn an extra exploit for no reason however I already crafted an all-in-one replacement exploit that I really like actually. It leverages a Flash 9 type confusion vulnerability enabling you to customise an actionscript object internal data. Once you got control over its internals, the object public methods & properties can be used in the actionscript code both for memory leaking purposes & for ROP triggering. It's ready to deploy if ever s#ny decides to patch the vulnerabilities currently exploited in the PS3 Toolset.