PS2 [Testers Needed] Open PS2 Loader with ZSO support (compressed ISO)

A

Acid_Snake

Developer
One of the upcoming features for OPL 1.2.0 will be support for ZSO format (compressed ISO images).
Those of you from the PSP scene might remember this format, it's the good old CSO but using LZ4 instead of DEFLATE (much faster and lighter).

I've been working on this feature for about a week now and the results are incredible: many games now fit and run 100% perfect with zero lag on USB devices (great examples being MGS3, Shadow of the Colossus, GTA 3 and Vice City, Resident Evil 4 and more).

I've tested the following games on a Fat PS2 with USB:

https://docs.google.com/spreadsheets/d/1K7szQS_9VG8es22eORw3xXTXxp5bZRpHOL6YJDH7S6c/edit?usp=sharing

Comparison of ISO and ZSO performance in the same USB device (an external SSD):
ISO: https://drive.google.com/file/d/1O7EGH1cO09SexxgyfadjHJrUvIR0kkAu/view
ZSO: https://drive.google.com/file/d/15E_9sWO_eNHl52XDjOpBIueADcZ9_ZwY/view

Other devices such as HDD and SMB are also known to work well, with HDD having no lag at all on any game.

Latest release can be found here:
https://github.com/ps2homebrew/Open-PS2-Loader/releases/tag/latest

Always Download the latest OPL builds from the Official OPL Github Release Page.

- Latest changes:
-- DVD9 games now work on all devices!
-- PS2 Logo now works on all devices.

To install ZSO files to internal HDD, use the version of hdl-dump attached in the release page.
 
Last edited:
Can be possible to not compress all sectors? In example: app could detect (based on ISO9660 fs table or just user chose via switch like i.e "--ignore *.abc BIG.VOL someanotherfile.bin") if there is video (easy in case of native formats like i.e *.pss) and ignoring it? That's increasing complexity of the code and format, but freeing users from FMV stuttering problems and it is best to do it now, while project is still fresh, instead support later both sub-formats. Just my three cents.

Besides that it is of course splendid improvement and space saver!
 
Berion said:
Can be possible to not compress all sectors? In example: app could detect (based on ISO9660 fs table or just user chose via switch like i.e "--ignore *.abc BIG.VOL someanotherfile.bin") if there is video (easy in case of native formats like i.e *.pss) and ignoring it? That's increasing complexity of the code and format, but freeing users from FMV stuttering problems and it is best to do it now, while project is still fresh, instead support later both sub-formats. Just my three cents.

Besides that it is of course splendid improvement and space saver!
Click to expand...
The CSO/ZSO format specification already has a mechanism to skip compressing blocks if it detects that the resulting compression doesn't have much of a gain.
It is actually a bad idea to NOT compress audio and video data as opposed to what people used to believe.

Back in the early days of PSP CSO format the FMVs would stutter like crazy for two big reasons: slow memory sticks and absolutely no optimizations on the reader algorithms that would yeild EXTREMELY high IO operations.

It was thought that the issue was the decompression speed, which is why JSO and (later) ZSO were created: with the idea in mind that a faster decompression would result in no stutter or high loading times. The results were clear: barely any changes.
This is because reading a file from an ISO (let's say a 520KB file) requires only 1 constant IO call no matter the size or where in the file it is, but the same read on a compressed format (whichever it is) results in a linearly growing amount of calls since you have to read arbitrary offsets withing the file and read block by block.

Most recent readers like the latest Inferno Drivers found in PRO, ARK and Adrenaline solve this issue by introducing caches that minimize IO and an algorithm that tries to read all compressed data at once rather than block by block to also reduce IO.
The results was that all compressed formats, including CSO, were now performing very well, very close (if not equal) to ISO speeds.

For PS2 of course I implemented an optimized reader algorithm like Inferno has, and the ZSO format uses the extremely fast LZ4 decompression algorithm that doesn't require any extra RAM and works just fine on the PS1 CPU dressed as a co-processor that Sony calls IOP.

To finish it off: I highly recommend compressing media files as that actually helps reduce stutter and even aliminates it on many games via USB. Back then the compression algorithms they used were heavy on CPU and RAM, so they would often use weaker algorithms with weaker compression, or even no compression at all, so that FMVs would go smooth. Nowadays we have LZ4, that achieves very good compression ratio with extremely fast decompression, so you get more ISO sectors per device block out of the USB 1.1 port of the PS2 and thus eliminating or greatly reducing stuttering.
 
Sounds reasonable but it is against human brain intuition. ^^ Media like PSS or ADPC are already compressed, so isn't decompression of twice compressed data is a lot longer? Isn't decompressor will decompressing faster blocks which have less patterns (i.e isn't faster is decompress compressed i.e 1234000012340000 than 1234567890ABCDEF)?

Forgot to mention: thank You for Your work. PS2 environment more than any other Sony consoles (right after PSP) needs compressed disc images. And now it became reality.
 
Berion said:
Sounds reasonable but it is against human brain intuition. ^^ Media like PSS or ADPC are already compressed, so isn't decompression of twice compressed data is a lot longer? Isn't decompressor will decompressing faster blocks which have less patterns (i.e isn't faster is decompress compressed i.e 1234000012340000 than 1234567890ABCDEF)?

Forgot to mention: thank You for Your work. PS2 environment more than any other Sony consoles (right after PSP) needs compressed disc images. And now it became reality.
Click to expand...

There's a catch though: ZSO decompression is done by the IOP, while the FMVs are processed by the EE and other hardware decoders, not the IOP itself (the PS1 CPU can't possibly work with the FMVs that the PS2 uses).
Also LZ4 decompression is equivalent to doing 3 memcpy, that's EXTREMELY fast.

I have links to a video I made comparing the EA Sports Big FMV intro for ISO and ZSO on the same exact USB device and the difference is night and day. ZSO actually makes games playable on USB, many of them perfect with zero lag.

ISO: https://drive.google.com/file/d/1O7EGH1cO09SexxgyfadjHJrUvIR0kkAu/view
ZSO: https://drive.google.com/file/d/15E_9sWO_eNHl52XDjOpBIueADcZ9_ZwY/view

Edit:
Now that I mention it, I need help testing this new experimental build with DVD9 support:
https://github.com/JoseAaronLopezGarcia/Open-PS2-Loader-ZSO/releases/tag/r111
 
Last edited:
You could wrap it to PE (Windows executable in CLI) so the users can just drag and drop one or more *.iso to get one or more *.zso. This will increase the audience as not all having set up Python on Windows or using Linux.

If I could suggest, also move it to separate repository as people would think (like @Carloss) it is part of OPL.

I don't want to be annoying with my "golden tips" but the more software is published in friendly to most none technical people, the best for getting larger qa testers involved.
 
Berion said:
You could wrap it to PE (Windows executable in CLI) so the users can just drag and drop one or more *.iso to get one or more *.zso. This will increase the audience as not all having set up Python on Windows or using Linux.
Click to expand...
We tried that, the problem is that python scripts wrapped into PE are very often detected as malware by many antivirus software, so we just leave the python script as is to not confuse users.

In Windows 10 if you try to use python in the command line without having it installed it will open the Windows Store so you can install it from there, so it isn't really that big of a deal.
 
  • Like
Reactions: TnA
Can't wait to try this. Awesome work!

bucanero said:
there are many CSO projects on github, some already support LZ4 as a compression method.

this project is actively maintained, has LZ4 as compression option and has build options for windows, linux and macOS:
https://github.com/unknownbrackets/maxcso
Click to expand...
@bucanero

It gives me this:

Error while processing GAME.ISO: LZ4 format not supported within CSO v1 file
Error while processing CONVERTED_GAME.ZSO: Unable to update buffer size to match
block size

The ISO is not CSO. What am I doing wrong here?

@Acid_Snake

The provided python file is not working with me. I have Windows 7 and I am guessing it requires a newer python that Windows 7 cannot use.
 
Last edited by a moderator:
Error has told You. ;) You must use CSO v2, because LZ4 compression is not supported. Maybe this way?
Code: 
maxcso --format=2 --use-lz4 game.iso

But this OPL fork not supporting *.cso, so why You want to try it? ;) Bucanero just point the idea to fork maxcso project to fit this specific ZISO needs (not using existing one implemented yet experimental in CSO).

- - -

ziso.py is written in Python 2. Python 3 is around few years with us. For sure Windows 7 have Python 2 interpreter (of course it is not part of the system as on Linuxes).
 
Last edited:
Berion said:
Error has told You. ;) You must use CSO v2, because LZ4 compression is not supported. Maybe this way?
Code: 
maxcso --format=2 --use-lz4 magic.iso

But this OPL fork not supporting *.cso, so why You want to try it? ;) Bucanero just point the idea to fork maxcso project to fit this specific ZISO needs (not using existing one implemented yet experimental in CSO).

- - -

ziso.py is written in Python 2. Python 3 is around few years with us. For sure Windows 7 have Python 2 interpreter (of course it is not part of the system as on Linuxes).
Click to expand...

Thanks.

"Error has told You. ;) You must use CSO v2, because LZ4 compression is not supported. Maybe this way?
Code:
maxcso --format=2 --use-lz4 magic.iso"

It gives me an error.
 
Last edited:
The provided python file is not working with me. I have Windows 7 and I am guessing it requires a newer python that Windows 7 cannot use.
Click to expand...

Run any Linux (i.e Mint), mount partition where You have stored disc images, put there script and run it. Both Python2 and 3 are already included in this distribution.

Or try this:
https://www.python.org/downloads/windows/
 
  • Like
Reactions: TnA
Berion said:
Run any Linux (i.e Mint), mount partition where You have stored disc images, put there script and run it. Both Python2 and 3 are already included in this distribution.

Or try this:
https://www.python.org/downloads/windows/
Click to expand...

I re-installed 2.7.

It seem I already had it but I think I also have 3.8 so maybe it is trying to run with the newer version and that maybe is causing the problem.
 
I don't using Python on Windows, so I don't know, maybe uninstall both and after that reinstall 2.7? It is the same pain in the ass as Java on this system; while everything just works on Linux (in those specific matters of course ;p).
 
Berion said:
I don't using Python on Windows, so I don't know, maybe uninstall both and after that reinstall 2.7? It is the same pain in the ass as Java on this system; while everything just works on Linux (in those specific matters of course ;p).
Click to expand...

I get this either way:

"ziso.py game.iso game.zso
Traceback (most recent call last):
File "C:\Users\User\Desktop\maxcso_v1.13.0_windows-32bit\ziso.py", line 29, in
<module>
import lz4.block
ModuleNotFoundError: No module named 'lz4'

It seems I did not have Python 3. I guess I can try it and see what happens.
 
You must log in or register to reply here.

Similar threads

Back
Top