PSP pop-fe.py: POP-FE: A Tool for linux to automate the process to create ps1 EBOOTs

Thanksgiving update for pop-fe.

It will now automatically inject CDDA tracks into the EBOOT.PBP for PSP and PS3 when possible.
To do this you need to install https://github.com/dcherednik/atracdenc
See README for instructions on how to do this for Linux and Windows.

It only works right now for single disk games and I have tested it with RidgeRacerRevolution.
I don't think I have any multi-disc games that uses CDDA tracks. Are there any multi-disc games that uses audio tracks
and if so which ones are there so I can get them and add support for them.


So, you like PSX games and want to play them on PSP or PS3? Download and try out pop-fe.
It supports EBOOT.PBPs > 2GB in size and it now automatically takes care of CDDA for you too.
 
Last edited:
Devildwarf said:
Thanks for this, this looks like a great update.
The only multi disc CDDA game that comes to mind is "Lunar Silver Star Story Complete" but there probably is more, there is a list of CDDA games at
https://tme.miraheze.org/wiki/List_of_PlayStation_games_with_CD_audio_tracks
but im not sure how acurate it is (you can check against redump's list to see how many cd's and audio tracks a game has if that helps too)
Click to expand...

I have added support for dealing with multi-disc games and CDDA tracks.
Should work now. Thanks for the pointers.
 
Here is a question about PSP/PS3 EBOOT.PBPs and multi-track discs.
I made some experiments with Loaded, which is ~60MB data and the rest of the disk are audiotracks.
Basically, right now I encoded the entire disc in the EBOOT.PBP, all the tracks including any AUDIO tracks.

Which means that for a game like Loaded I first encode about ~500MByte of audio tracks as part of the full disc image
and then I inject ~50MB of the audio tracks encoded as ADTRAC3 LP2.
This is a bit wasteful IFF teh game itself never accesses the audio data tracks directly and only via system calls like "play cd audio track #n on the speaker", which the emulator can intercept and just redirect to play the ADTRAC3 LP2 data instead.

I tested this by truncating the image that is stored inside the EBOOT.PBP to only store the first track, which is always data.
For LOADED this resulted in an EBOOT.PBP that is ~100MB in size instead of ~600MB.


Now the question is, is this safe? It does not sound completely safe to me to do this unless some restrictions are guaranteed.
So,
1, can I assume that the first track is the only track that ever contains data, i.e. that the game will never read data off track 2 or later?
2, can I assume that games will never actually check how many tracks there are on the disk and thus they will not notice that all tracks except track 1 are missing?
3, can I assume that games will never try to read the audio tracks directly?


It sounds risky to me to do this hack but it would save a lot of space for games that contain a lot of large audio tracks.
On the other hand, there are not that many games that use CDDA in the first place and possible not that many of them use a lot of large tracks? So maybe it is not worth the risk that it will cause some games to fail?
Or I could hide this behind a command line argument and then "use at own risk, it can either save a lot of storage or it can break your EBOOT.PBP"
 
A little story indirectly related...
In psdevwiki we have a compatibility list for PS1 games running in PS3
In the last couple of months i been doing some serious manteinance of the compatibility lists (has been a pain)... and one of the things i had in mind was to add a explicit note in that list for the games that "Contains CDDA audio tracks", or "Requires libcrypt patch" beause both things affects the compatibility, and was "features" of the PS1

As you can see the list contains some mentions of some games using CDDA (but i guess is very incomplete)... and there is almost no mentions of the libcrypt protections
As far i know the libcrypt protections was used only in games published in PAL-EU regions... and the games published in the official PSN store under the label "PSone Classic" already have an official libcrypt patch applyed to the ISO
My guess is a lot of the PAL-EU games that appears in that list as "unplayable" probably is because have a libcrypt protection

So... my plan was to create a new page in wiki named "PS1 game patches" dedicated to them, and create links in between the games that requires that patches and the new page... then document in the new page a full list with the info required to apply that patches
But... i dont know how many games uses CDDA and libcrypt... so the plan is freezed by now, i dont know what to do yet

*Also, we have the problem that some of that libcrypt patches was released together with a "trainer" (the kind of modifyed main screen where you can enable or disable cheats or stuff)... for some reason that trainers doesnt seems to work in PS3, are problematic, so is needed to separate the real patch (that disables libcrypt) from the other stuff that is not really a requirement for the game to run
 
sandungas said:
A little story indirectly related...
In psdevwiki we have a compatibility list for PS1 games running in PS3
In the last couple of months i been doing some serious manteinance of the compatibility lists (has been a pain)... and one of the things i had in mind was to add a explicit note in that list for the games that "Contains CDDA audio tracks", or "Requires libcrypt patch" beause both things affects the compatibility, and was "features" of the PS1

As you can see the list contains some mentions of some games using CDDA (but i guess is very incomplete)... and there is almost no mentions of the libcrypt protections
As far i know the libcrypt protections was used only in games published in PAL-EU regions... and the games published in the official PSN store under the label "PSone Classic" already have an official libcrypt patch applyed to the ISO
My guess is a lot of the PAL-EU games that appears in that list as "unplayable" probably is because have a libcrypt protection

So... my plan was to create a new page in wiki named "PS1 game patches" dedicated to them, and create links in between the games that requires that patches and the new page... then document in the new page a full list with the info required to apply that patches
But... i dont know how many games uses CDDA and libcrypt... so the plan is freezed by now, i dont know what to do yet

*Also, we have the problem that some of that libcrypt patches was released together with a "trainer" (the kind of modifyed main screen where you can enable or disable cheats or stuff)... for some reason that trainers doesnt seems to work in PS3, are problematic, so is needed to separate the real patch (that disables libcrypt) from the other stuff that is not really a requirement for the game to run
Click to expand...

Very interesting. I have no PAL-EU disks myself but I have heard about libcrypt and my naive understanding is that this is basically a type of copy protection. So even if you convert one of those games to EBOOT.PBP it would fail to run since the copy protection would trigger?
This affects playing the EBOOT on both PSP and PS3?
And the way to resolve this is to apply a binary patch to the software on track 1 to disable this check?

If so it might be a useful thing to add a feature to pop-fe to, based on game-id, try to download and apply patches for this automagically
IF it detects a game image where there is a corresponding patch.

So how could that work? I could imagine something like like a website where I could download/wget a patch based on game-id.
The patch would contain perhaps the md5sum of track1 before the patch is applied, the md5sum of track1 after the patch is applied and the actual patch itself.
Then pop-fe could work like this:
1, go to website and try to download the two md5sum hashes and the patch.
2, if patch is not found we just continue like normal, if there is no patch there is nothing we can do about it.
3, IF a patch is found but the md5sum of track1 matches the "after patch applied checksum" we just print, patch is already applied, skipping merging patch. And we continue like normal.
4, IF a patch is found but the md5sum of "before patch is applied" does not match, then we print a warning "patch found but md5sum mismatch" and we skip applying the patch and continue as normal.
5, We apply the patch to track 1, verify that the new file matches the "after patch applied" checksum and uses this patched image to generate the eboot.pbp.

I.e. we get some kind of "automatically download and apply a patch for the game from psdevwiki or other site"
That would be neat to add.
 
sandungas said:
A little story indirectly related...
In psdevwiki we have a compatibility list for PS1 games running in PS3
...
Click to expand...

Looking at that page I see the section "Fix to Audio and Framerate Slowdown..."
I already do this unconditionally and set RESOLUTION=1 (== only option ticked is "480") for all PS3 packages.
Are there any disadvantages to always set RESOLUTION=1 ? It seems fine on all (not that many honestly) PKGs I have tried.

Second question, Should I set RESOLUTION=1 for PSP EBOOTs as well or does this framerate fix only apply to PS3? A genuine EBOOT I looked at did not contain a 'RESOLUTION' field at all, which is why I don't generate one for PSP either.


BTW, as part of popstation.py I have parsers and encoders for SFO, like such in case someone finds it useful:
$ python
Python 3.8.10 (default, May 4 2021, 00:00:00)
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a=open('PARAM.SFO','rb')
>>> b=a.read()
>>> from popstation import ParseSFO
>>> print(ParseSFO(b)['parameters']['RESOLUTION'])
{'data_fmt': 1028, 'data': 1}
 
Last edited:
See the section at most bottom of the compatibility list named "SCE boot screen restoration", this was added by other wiki editor, not me, i never experienced that problem, but i think the way how is explained is misleading
The point is... we need to apply a patch to remove the libcrypt protection, and some of this patches was usually distributed as a "trainer", where the patch includes the data to disable the libcrypt + some kind of menu that appears at the time we boot the game
As example in legacy of kain, soul reaver
I guess what really "breaks" the emulator is the visual effects that appears at the back, this was the kind of thing the programmers was doing for fun playing around with the audio or the graphics chips at low level

This is why the section in the wiki section pretends to explain that is needed to "restore" the start sequence of the game... but is wrong in concept because the solution is to apply the libcrypt patch data ONLY (without that trainer menus)
I was thinking if there is some way to collect all that libcrypt patches and cleanup them and publish them in wiki as something legacy... are the kind of patches you could include in your program, not sure how many games needs them though but the patches are going to be tiny

For the RESOLUTION in the PARAM.SFO... it works mostly as a restriction, the PS3 is going to select one of the available modes, by keeping only 1 mode you are forcing the PS3 to use that one, but in this case im not sure if the result would be the same by enabling 480 and 576 together

The PARAM.SFO format used in PS3 is like an update of the format used in PSP... some stuff was added specific for the PS3, right now i dont remember if the RESOLUTION is specific for PS3, but check a few PSP games, if none of them have it it means the PSP doesnt uses it
 
Just found a pretty cool list of the libcrypt protected games, and the patches released for them, so... you could make your app to download them from here
...and i need to take a look at them and imagine how to document them in psdevwiki
https://psxdatacenter.com/sbifiles.html

And is needed some feedback too, to know if some of them "breaks" the PS1 emulator on PS3 as i explained above, the ones that does some replacements in the game boot sequence (the attrac animation and/or main menu) are candidate to break the emu and requires to be "rebuilt" to use them in PS3

*And i found a technical writing introduction to libcrypt... is in spanish though
https://www.elotrolado.net/hilo_introduccion-al-libcrypt-1-2_91953
 
sandungas said:
Just found a pretty cool list of the libcrypt protected games, and the patches released for them, so... you could make your app to download them from here
...and i need to take a look at them and imagine how to document them in psdevwiki
https://psxdatacenter.com/sbifiles.html

And is needed some feedback too, to know if some of them "breaks" the PS1 emulator on PS3 as i explained above, the ones that does some replacements in the game boot sequence (the attrac animation and/or main menu) are candidate to break the emu and requires to be "rebuilt" to use them in PS3

*And i found a technical writing introduction to libcrypt... is in spanish though
https://www.elotrolado.net/hilo_introduccion-al-libcrypt-1-2_91953
Click to expand...


I think the SBI files are a representation of the actual subchannel data itself and not an actual patch so they would be useful for people that are burning discs using burners that can write the subchannel, or in emulators that also emulate the capability of reading the subchannel. I.e. providing/emulating the subchannel data but leaving the libcrypt code itself intact.

I think what I would need would be actual patches that disables the libcrypt code.
Here is a place that appears to keep patches for libcrypt: (that site seems pretty dodgy so I should probably not link to it)
but the structure seems very ad-hoc and unlikely possible to be able to be managed automatically.
Ideally there would be a repository of patches that could be access by game-id.
 
Yes, SBI is subchannel data, is because the libcrypt protection is stored there, btw there is another protection named "securom" i guess it works in a similar way. As far i understand securom only works in PC, not in the PS1 games

This wiki page is marked as "old" but is a good resume of what they did, it seems the SBI format is custom (it seems someone related with redump.org made it), and they are using other format named LSD too
http://wiki.redump.org/index.php?title=PlayStation_1:_LibCrypt_protection_(Old)

I dont know the details to be honest, but as far i understand the SBI patches from redump are going to be more compatible because they are only patching the game data related with the protection (without any trainer, menu, PAL/NTSC selector, etc...)


Edit: this talks are related:
https://www.psx-place.com/threads/psx-libcrypt-protected-games-on-multiman.16598/
https://gbatemp.net/threads/psx-protected-games-on-multiman.498237/
 
Last edited:
Im collecting some links at bottom of this page while trying to understand how it works https://www.psdevwiki.com/ps3/Talk:PS1_Classics_Emulator_Compatibility_List#LibCrypt_Protected_games

Special mention to the thread in cybdyn-systems.com forum... and the .ZIP file hosted in redump.org (includes all the SBI patches), download it and open some SBI files in a hexeditor, all them have the same size and are tiny (452 bytes each) because are like a descriptor of a CD. I felt in love of the SBI format instantly, it seems to be the most efficient way to defat that protection :D

Now im wondering if there is some way to apply a SBI patch to the CD image (BIN/CUE) permanently. The official PS1 emulator in the PS3 firmware doesnt supports SBI files or any subchannel descriptor file, actually as far you can read in one of the talks in gbatemp sony is "patching" his games distributed in the PSN store to defeat his own libcrypt protection, this is a clear indication that the official PS1 emulator is not able to do the patching "on the fly"

Btw @Zar have you thought in implementing SBI support for PS1 games in ManaGunZ ? :rolleyes:
 
Ok, from what I can tell, the SBI files are not actually patches but they contain subchannel data, which is data on a cdrom that can not normally be read using the normal SCSI commands and can usually not be written either.
They are out of band data which the PSX cdrom drive CAN read and return to the application and such that the game/application have special code, libcrypt, that reads this subchannel from some specific part of the disc and verifies that what the cdrom drive returned matches what libcrypt knows that sony deliberately had written to the subchannel.
Since originally, customer cdrom readers/writers could not read/write the subchannel it meant that if you copied a disk the subchannel on the disc would suddenly be some semi-random garbage and NOT that specific string of bytes that sony wrote in their factory.
Thus libcrypt could detect "this is not a genuine disc", and error out.

Later drives apparently makes it possible to read the subchannel and then if you modify an emulator to take the file containing the subchannel data/the SBI file and ehnance the emulation of the virtual cdrom drive to be "if the games this call to the cdrom drive to ask for data, give the game the content of the SBI file" which results in the game working.
https://documentation.help/ePSXe/subchannelreading.htm

So, I think technically speaking, SBI files are not a patch and they do NOT remove the libcrypt protection code, the SBI files is just a means to "make the emulator return the expected subchannel data when the libcrypt code checks the disk".
I.e. the libcrypt code is still there and operational, it is just with the SBI file the emulator can more accurately emulate the disc content and thus tricking libcrypt to think there is a genuine disk there.


That would not work for EBOOT.PBPs because there is no place in the file to store the subchannel, and the actual emulator itself does not have any means to emulate the subchannel either.
So for this case we would need an actual code patch that modifies the game code and removes the actual libcrypt code/checks from the game itself.
It looks like these types of patches are called PPF patches.
From what I can tell these PPF files are actual code patches that can be patches in either to the ISO itself or into ram by the emulator.
I think a possible solution for EBOOT.PBPs would be to "find the PPF that removes libcrypt for <game-id>", apply the patch to the .bin file, and then recompute all the data checksums that will no longer match as the data has been modified.

These kind of binary PPF patches can probably not only be used to "de-activate the libcrypt code" but also "translate all strings into language xyz" and similar.
Now, to find documentation of the PPF format and to find if there is a repository somewhere where there are PPFs searchable by game-id.../

(I would not care about trainers or similar myself, I think those things ruin the fun of games, but I would like to make it possible to 'rip my game disk into bin/enc' then pop-fe will automatically apply the "deactivate libcrypt" patch so that the resulting EBOOT will work.)

EDIT: oh, where possible, the SBI solution is probably the easiest way to fix this for emulators. It is just a matter of "emulate one more SCSI command for the CDROM drive and have it return the content of the SBI file" and as there seems to be comphrehensive datasets of all SBI files for all games ... so "emualte this SCSI command" and "provide this set of all SBI files" would fix all games in one go.
But unfortunately that is not viable for EBOOT.PBP based game conversions :-(

EDIT2: Turns out I have two games with libcrypt. Need for speed porche 2000 and vagrant story, so now I just need to track down if I can find a PPF file for them and try how this will work.
 
Last edited:
So, PPF files exists in 3 versions, PPF1/2/3 of which I think PPF1/2 are possibly the only ones for PSX disks?
Anyway, the format of the three versions are fairly straightforward to implement in python, and there is even source code for C implementations, like this:
https://github.com/meunierd/ppf/blob/master/ppfdev/applyppf_src/applyppf3_linux.c

Applying a PPF patch does nothing at all that is computationally intensive so this should be easily fast enough even in python3.
I will write a python implementation of this during the new-years holidays and add it to pop-fe.


The sad thing is that there does not seem to be any structured repository for all these PPF patches, and the places that do contain these patches seems pretty dodgy and contain at a quick glance other things too that I don't want to get involved with.
So, maybe a mode for pop-fe where "you find and provide a PPF file to pop-fe and it can automatically apply it for you, but pop-fe will not even attempt to go search for any PPF files, that is your job" could be a way to work.

Optionally, the PPF1/2 patches ONLY contain information such as "offset, length, new bytes for these offsets" which means they do NOT
contain any of the code from the game itself, it should be ok from a copyright perspective to host the PPF files on a site.
The PPF3 format can be used to UNDO a patch too so that tells me it contains the actual content from the original image which means PPF3 files are likely to come with copyright issues.

@sandungas , maybe the PPF1/2 files could be hosted on your wiki-page? It would really be nice with a single repository for all list of all these games with both the SBI files, for those that want an SBI file, as well as the PPF patch to disable libcrypt, where everything is in one place and indexed by game-id.
 
In the page specific for every PSX game in redump.org they are using a template where appears the names of 3 different protections: "libcrypt", "EDC", and "anti-modchip"
As far i understand we only need to worry about the libcrypt, because the anti-modchip protection is specifically designed to detect the modchips (the emulators doesnt triggers the protection because are not a modchip, neither the PSIO hardware), and most of the people i found talking about all this are saying the EDC protection doesnt matters

So... we just need to worry about the games listed as "libcrypt=yes" in redump.org it seems can be viewed under this link: http://redump.org/discs/libcrypt/2/
I have not checked it yet, but i guess are going to match this other list https://psxdatacenter.com/sbifiles.html
We can also use the files in this collection as a reference. There are 229 files for 229 game IDs... is just the file doesnt mentions the game ID http://redump.org/sbi/psx/
We need to crosscheck this 3 lists to verify that are really 229 games
And after that is needed to find the PPF patches for them in this database: https://consolecopyworld.com/psx/psx_patches.shtml

The problem is by doing this we are going to fall in the problem i was mentioning before, a lot of them probably have a PAL/NTSC selector, a trainer with menus, etc... and this breaks the game for the official PS1 shipped with the PS3 firmwares (and maybe the popstation PS1 emu of the PSP too, i dont know)
The only solution in short term i see is to create a list with the PPF patches of the 229 games and indicate in everyone of them if they are "compaible with PS1 emu" ...or... "incompatible with PS1 emu"
But i cant add that info by myself, we will need to wait for the feedback of other wiki editors... this will take many time and there are big probabilities that the intiative will be completly ignored and we will have zero feedback... so... for the interests of your program you cant relly on this feedback

I would love to find some way to host them in psdevwiki but there is no filehosting support, what we use to do when the file is small is to publish a hexeditor view of it... this is one of the reasons why i liked the SBI files too much, because are small and easy to "copypaste" as text by other wiki readers into a hexeditor and save it as a .bin file
But i guess we cant do that with the PPF files because are going to be huge when represented as a hexeditor view, so dunno yet how to do it, but as an initial approach i can add download links to the PPF files in the "libcrypt protected games list" with the MD5 of them

--------------
Btw, there is a easy way to "port" the PPF format to any other, as example xdelta by doing this:
1) make a backup of the disc named game_original.iso
2) make a copy of the game_original.iso renamed to game_patched.iso
3) apply the PPF patch to the game_patched.iso
4) use xdelta with the 2 files (game_original.iso and game_patched.iso) to create the xdelta patch

This kind of conversions from PPF to other patch formats could be handy incase you prefer to add support to your program to other formats instead of PPF
 
sandungas said:
The only solution in short term i see is to create a list with the PPF patches of the 229 games and indicate in everyone of them if they are "compaible with PS1 emu" ...or... "incompatible with PS1 emu"
But i cant add that info by myself, we will need to wait for the feedback of other wiki editors... this will take many time and there are big probabilities that the intiative will be completly ignored and we will have zero feedback... so... for the interests of your program you cant relly on this feedback
Click to expand...

Yes, a list/table like https://www.psdevwiki.com/ps3/PS1_Classics_Emulator_Compatibility_List
but just for the 229 libcrypt games.

And for each one there would be a column with a link to the corresponding SBI file which should be timeconsuming but easy to make as all these SBI files are all readily available.

And an additional column with a hyperlink to where a PPF file can be found. This would be a lot harder to do as these PPF files seems to be spread around many different sites and also they, as you say, come in different formats and we are only interested libcrypt-only PPF files. These links could be added manually one game at a time as people contribute to the effort. I would volunteer to try to add as many such links as my time permits to seek them out.

Having a curated list of these games, the SBI and the PPF to deal with libcrypt at a single central place would be very helpful. Even if the PPF section would be incomplete.
 
Added a utility called pop-fe-ps3.py
This is a basic pythonb pygubu application with a GUI for creating PS3 packages.
It is basically a simple graphical frontend to "./pop-fe.py --ps3-pkg"

You select .cue file, up to 5 files.
The first cue file is used to auto-populate the disc cover image and the background images as well as the disc-id and title.
You can override/change the images by clicking on them and select a different file to use.
Then there is a file-picker where you select the output directory and a text box where you enter the file name for the package.
Then click Create PKG and wait until it is finished.

It will automatically handle libcrypt. It will automatically convert any audio tracs into ATRAC3 format and inject them into the package.

It is very primitive UI for now but maybe there are people that prefers this instead of using the command line.
I am not good at UI design so anyone that can tweak and make it better are super-welcome to help and send patches.
pop-fe-ps3.png
[/IMG]

Go here to see what it looks like: https://github.com/sahlberg/pop-fe/blob/master/pop-fe-ps3.png
 
Last edited by a moderator:
nice work, a UI will help a lot for people who don't get on with command line/terminal,
I wonder if a PYinstaller build could be made ? so that users don't need to download python and all the requirements (or PyPI or any other way of making a stand alone exe or program)
 
You must log in or register to reply here.

Similar threads

Back
Top