PSP pop-fe.py: POP-FE: A Tool for linux to automate the process to create ps1 EBOOTs

R

Ronnie Sahlberg

Developer
Hi guys and gals,

This is my first thread, just joined. If I broke protocol, please correct me kindly and gently.

I Love PSP, I have five of them, and I love playing PS1 games on them. Right now playing WildArms.

I just uploaded a small tool to github that I am using to create nice EBOOT.PBP of PS1 games.
It automates them process to go find the boxart and find some screenshot as the background and creating a EBOOT for my PSP.
It works on both Linux and Windows, though some features are not yet implemented on Windows..

The way I use it is that I connect my PSP to USB on my linux box, then just run something like:
./pop-fe.py --psp-dir=auto /mnt/MetalGearSolidVRMissions.cue

I.e. I provide the path to where the SD card for my PSP showed up and also the path to where the .cue file for the game is.
Then the tool will (using bchunk re-written in python3) convert it into an ISO, to find the game-id, which is held in the system.cnf file on the iso.
Then it will go to psxdatacentre and download the box art for the game and also pick, randomly, one of the screenshots for the game to use as a background.
The it will use popstation (re-written in python3) and write this onto the PSP.


It works well for me, but can came be made better. Patches welcome, praise welcome, please no hate.
Oh, where is it ? It is here:
https://github.com/sahlberg/pop-fe

If someone want to help make this work under windows that would be awesone.

I hope some of you might find this useful. It is useful to me and I want to share.

best regards and keep PSP-ing
ronnie sahlberg
 
Last edited:
Oh,

Q: POP-FE, that is a weird name for a tool.
A: Yes. I actually don't remember where it came from. I have been using this tool for a while and never put much thought into the name.
Let me make a guess: POPstation-and-boxart-FEtcher ?
That is as good as any explanation for the name.
 
I do not own a PSP anymore but as a main linux user i appreciate this type of stuff and wish more lools for vita, ps2 and ps3 were on linux.

NIce one
 
TheDemon said:
I do not own a PSP anymore but as a main linux user i appreciate this type of stuff and wish more lools for vita, ps2 and ps3 were on linux.

NIce one
Click to expand...

Cool, let me derail my own thread.
I don't do VITA or PS3 yet. But do you have examples of PS2 related utils that you want ported to linux (and where source is available)
and I can give it a shot. No guarantees.
It could turn out like the project above, where you have a new bchunk.py rewrite that works for cue files with > 1 one file directive,
ot a version of popstation.py that will work on big-endian platforms.
(just portability bugs I fixed when re-writing them).
((i.e. I will rewrite them and fix a bunch of bugs at the same time))

Tell me about a tool for ps2 you want ported to linux and I can give it a try. Linux is great, ps2 is great and together they are even greater:)

regards
ronnie sahlbrg
 
I'm using Linux Mint (64bit) v20.1 with Cinnamon (in live mode). In this distro, "python" variable is for "python2", so I changed every "#!/usr/bin/env python" to "#!/usr/bin/env python3" and it start trying execute scripts.

But unfortunately:
Code: 
pop-fe.py --psp-dir /home/mint/psp_test/ms0 "/home/mint/psp_test/gry/Quake II (USA).cue" /home/mint/psp_test/vmc/multiplayer.vm1 --game-id=SLUS00757

Traceback (most recent call last):
  File "/home/mint/psp_test/pop-fe-master/pop-fe.py", line 18, in <module>
  import iso9660  # python-pycdio
ModuleNotFoundError: No module named 'iso9660'
Something missing in my Linux? apt doesn't find anything like "python-pycdio".
 
This can be one of the "Ubuntu things" which means, changed some package names (the same with Mint). Sometimes it is really annoying to find new name. Yet I'm using it because it is IMO the best idiot-proof Linux in the wild, perfectly fit to me. ^^

Is it even needed for something? I providing gameid as parram so I don't care of converting bin to iso. Can I comment it? Didn't tried yet, just asking.
 
Code: 
$ pip install pycdio
should do the job

If it doesn't work you can try to compile it yourself following instructions here: https://pypi.org/project/pycdio/
To build on Debian (e.g. Ubuntu):
Code: 
$ apt-get install python-dev
$ apt-get install libcdio-dev
$ apt-get install libiso9660-dev
$ apt-get install swig pkg-config
$ pip install -e .
$ make check
Click to expand...
 
Berion said:
I'm using Linux Mint (64bit) v20.1 with Cinnamon (in live mode). In this distro, "python" variable is for "python2", so I changed every "#!/usr/bin/env python" to "#!/usr/bin/env python3" and it start trying execute scripts.

But unfortunately:
Code: 
pop-fe.py --psp-dir /home/mint/psp_test/ms0 "/home/mint/psp_test/gry/Quake II (USA).cue" /home/mint/psp_test/vmc/multiplayer.vm1 --game-id=SLUS00757

Traceback (most recent call last):
  File "/home/mint/psp_test/pop-fe-master/pop-fe.py", line 18, in <module>
  import iso9660  # python-pycdio
ModuleNotFoundError: No module named 'iso9660'
Something missing in my Linux? apt doesn't find anything like "python-pycdio".
Click to expand...


Looks like that package is not available on mint/debian. Odd.
I have added support for a different iso-parsing library, so try to install pycdlib instead.
I.e.:
pip3 install pycdlib
and (which will fail on debian)
pip3 install pycdio

If at least one of them are installed then pop-fe will use it and it should work.
The only reason for parseing the iso is in order to read SYSTEM.CNF to find the game-id.
You can also avoid this by forcing the game-id via the --game_id=... command line argument.


Notes, run pop-fe.py from the directory where it is installed. It needs to be able to create some temporary files
while it is preparing the EBOOT.

Also, multi-track disks:
Pop-fe can only handle multi-track disks that comes in a single BIN file.
If you have a multi-track disk that comes with a separate file for each track you need to manually merge it into a single
bin first.
Next weekend when I get time to work on this again I will fix this.
I will basically add a dependency to binmerge (a python program that merges tracks into a single bin):
https://github.com/putnam/binmerge
and automatically call it from pop-fe to create a temporary single-bin file.
Until I do that you will have to do the merging manually.

Fortunately it seems very few PS1 games are multi-track afaict.

To handle multi-track files you MUST have a .CCD file. This is suboptimal since it should be possible to create
a CCD file from the CUE file. This requirement comes from popstation and is used to build the TOC. If anyone knows a tool that can read a CUE and create a TOC I am interested to hear. Otherwise I will write one.
I will look into fixing this next weekend to make it easier to use.
The idea is that I need to remove as many, or possible, all, manual steps that are involved in creating a nice EBOOT.
The tool should do everything automatically.

We will get there. Small incremental improvements. One at a time.
 
Last edited:
New update,

Both cue2cu2.py and binmerge are required now but README file and the python Exception that is triggered if they are missing
will provide info about how to install them.

cue2cu2 is nice to have because it has good cue parser that can parse all(?) cue's for PSX games (it is the tool used for PSIO on linux)
and, more importantly, it produces output that is completely trivial to parse and turn into a TOC for multi-track disks.
This means that there is no longer a need for a CCD file.
pop-fe can now automatically create a proper TOC for the conversion by just using the cue (and converting it to a temporary cu2 file)

binmerge provides support to handle multi-bin disks. If the cue indicates that there are tracks using different bin files, pop-fe will
now automatically merge these tracks into a new temporary single-bin file with accompanying cue file.
The original files are not modified. Only temporary files are created in the pop-fe.py directory.

But it means, multi-bin disks can now be converted into an EBOOT. And a TOC is automatically generated and no need for a CCD file.

Aside from this, the tool, as far as I can test, generates correct and bootable EBOOTS also for large multi-disk games such as FF8.
(You mentioned other tools can not handle large multidisc games? I bet it could be something as simple as the tools using an 'int' where they would really need to use an 'unsigned int'. Three and four disk games could become larger than 2GB which is where a signed 'int' would wrap and become negative. If you have the sourcecode for those tools, you could try changing all int's to be unsigned int's, it might fix them)


Please have a try. It is starting to work really well for me now. Wondering what I should add next.

EDIT: Oh, README also updated to describe how to use it to install EBOOTs onto a Vita. Is trivial and just some path mangling.
 
Update, Features:

pop-fe.py is a python script to automatically download art and create and install EBOOT.PBP games onto your PSP/VITA.
It supports:
* Automatically detecting GameId and Title
* Automatically downloading ICON0 and PIC1
* It supports creating multi-disk EBOOTS. Tested with up to 4 disks (ff8)
* It supports converting all known memory card formats into VMP and installing them.

Example:
Installing a single disk game:
./pop-fe.py --psp-dir=auto /mnt/MetalGearSolidVRMissions.cue

Installing a multi-disk game:
./pop-fe.py --psp-dir=auto /mnt/Grandia1of2.img /mnt/Grandia2of2.img

Installing a game and also a memory card image:
./pop-fe.py --psp-dir /run/media/sahlberg/disk/ Crash.cue Crash.srm


Many many more use-cases described in the README file. Please have fun and play PSX games!
 
Last edited:
I'm still stuck. ;)

Code: 
apt install python3-pip

pip3 install pycdio

ERROR: Failed building wheel for pycdio
  Running setup.py clean for pycdio
Failed to build pycdio
Installing collected packages: pycdio
  Running setup.py install for pycdio ... error

ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-pjg7ufi2/pycdio/setup.py'"'"'; __file__='"'"'/tmp/pip-install-pjg7ufi2/pycdio/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-pqiers4l/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/mint/.local/include/python3.8/pycdio Check the logs for full command output.

pip3 install pyclib
ERROR: Could not find a version that satisfies the requirement pyclib (from versions: none)
ERROR: No matching distribution found for pyclib

Code: 
/home/mint/psp_test/pop-fe/pop-fe.py --psp-dir /home/mint/psp_test/ms0 "/home/mint/psp_test/Quake II (USA)/Quake II (USA).cue" /home/mint/psp_test/vmc/multiplayerowe.mcr --game-id=SLUS00757

Traceback (most recent call last):
  File "/home/mint/psp_test/pop-fe/pop-fe.py", line 31, in <module>
  import requests_cache
ModuleNotFoundError: No module named 'requests_cache'

pip3 install request_cache
ERROR: Could not find a version that satisfies the requirement request_cache (from versions: none)
ERROR: No matching distribution found for request_cache
 
Berion said:
I'm still stuck. ;)

Code: 
apt install python3-pip

pip3 install pycdio

ERROR: Failed building wheel for pycdio
  Running setup.py clean for pycdio
Failed to build pycdio
Installing collected packages: pycdio
  Running setup.py install for pycdio ... error

ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-pjg7ufi2/pycdio/setup.py'"'"'; __file__='"'"'/tmp/pip-install-pjg7ufi2/pycdio/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-pqiers4l/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/mint/.local/include/python3.8/pycdio Check the logs for full command output.

pip3 install pyclib
ERROR: Could not find a version that satisfies the requirement pyclib (from versions: none)
ERROR: No matching distribution found for pyclib

Code: 
/home/mint/psp_test/pop-fe/pop-fe.py --psp-dir /home/mint/psp_test/ms0 "/home/mint/psp_test/Quake II (USA)/Quake II (USA).cue" /home/mint/psp_test/vmc/multiplayerowe.mcr --game-id=SLUS00757

Traceback (most recent call last):
  File "/home/mint/psp_test/pop-fe/pop-fe.py", line 31, in <module>
  import requests_cache
ModuleNotFoundError: No module named 'requests_cache'

pip3 install request_cache
ERROR: Could not find a version that satisfies the requirement request_cache (from versions: none)
ERROR: No matching distribution found for request_cache
Click to expand...

That is odd.
I have tested on a mint 20.1 system and pycdlib can be installed:
sahlberg@mint20:~/pop-fe$ cat /etc/issue
Linux Mint 20.1 Ulyssa \n \l

sahlberg@mint20:~/pop-fe$ pip3 install pycdlib
Requirement already satisfied: pycdlib in /usr/local/lib/python3.8/dist-packages (1.11.0)

Also requests_cache also installs on mint 20:
sahlberg@mint20:~/pop-fe$ pip3 install requests_cache
Collecting requests_cache
Downloading requests_cache-0.6.3-py2.py3-none-any.whl (28 kB)
Collecting itsdangerous
Downloading itsdangerous-2.0.1-py3-none-any.whl (18 kB)
Requirement already satisfied: requests>=2.0.0 in /usr/lib/python3/dist-packages (from requests_cache) (2.22.0)
Collecting url-normalize>=1.4
Downloading url_normalize-1.4.3-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from url-normalize>=1.4->requests_cache) (1.14.0)
Installing collected packages: itsdangerous, url-normalize, requests-cache
Successfully installed itsdangerous-2.0.1 requests-cache-0.6.3 url-normalize-1.4.3

What version of mint are you using?
Have you switched mint to set python3 to be the default? A lot of packages are not available on python2.
This command should set python3 to be the default systemwide

sudo apt install python-is-python3


Try making sure your mint is up to date and you have run the command above to set python3 to be default.
I have changed pop-fe so that requests_cache is now optional, i.e. used if it is available otherwise skipped in case you can not get it installed. I could install that module though on mint 20.1

And with these things, the EBOOT build process works on my mint vm:
ahlberg@mint20:~/pop-fe$ ./pop-fe.py --psp-dir=tmp /mnt/CrashTeamRacing.cue
Processing /mnt/CrashTeamRacing.cue ...
Id: SCUS94426
Title: CRASH TEAM RACING
Create EBOOT.PBP at tmp/PSP/GAME/SCUS94426/EBOOT.PBP
 
Last edited:
I'm using live, so no installation. Just doing apt update and after that installing that I need for the moment.

I was thinking that default Python in Mint 20.1 is Python3. I'll try Your method tomorrow.
 
I booted a mint 20.1 live disc (in a VM) and opened a shell.
These are the commands I used to get to a functioning pop-fe install :

sudo apt install python-is-python3
sudo apt install python3-pip
pip3 install requests_cache
pip3 install pycdlib
sudo apt install git
git clone https://github.com/sahlberg/pop-fe.git
git clone https://github.com/NRGDEAD/Cue2cu2.git
git clone https://github.com/putnam/binmerge.git
cd pop-fe
cp ../Cue2cu2/cue2cu2.py .
chmod +x cue2cu2.py
cp ../binmerge/binmerge .
chmod +x binmerge

After this I could use pop-fe to create and install an eboot onto a connected psp.
I updated it too so you no longer need to specify the path to the PSP/VITA memory card.
You can now just do --psp-dir=auto and it will try to find where the card is mounted.
 
Added support and instructions how to run it from windows (only windows 10 tested).
Auto does not work, so you have to specify the drive-letter as --psp-dir=/e/
would not be hard to add if someone wants it.
 
Berion said:
*.vcd is standard PSX disc image but with glued TOC on start. PS2VCD can making them (win32).
Click to expand...

I can make VCD files. And they work with OPL, but I make one VCD for each disk image.
Some time ago I read something that mentioned multi-disc VCDs. Which I guess would be a single VCD file that contains multiple disk images.
I don't know if that is actually a thing or not, but if it is and someone knows how they are created then I want to be able to make them too.
 
Oh, so by multidisc You have in mind two or more discs combined into one. Those are discs with broken standard of course because with LBA beyond red book. But, it is not sufficient to just make large CD image with data from all discs because:
  • some files can be different, yet have the same filenames, so executable modifications is needed to change seeking filenames,
  • executable must be modified in a way to NOP disc changing prompt, especially in games like Chrono Chross or Metal Gear Solid.
It is not worth the effort as there is not possible automate solution and POPS allow to change disc.

BTW: OPL doesn't support PSX games! OPL DB does by POPStarter. ;)
 
You must log in or register to reply here.

Similar threads

Back
Top