PSP SCUMMVM port for PSPonPS4

[jacobsson]

Do you two need 2.6.0? I just built it on Ubuntu 22.04. I used the latest precompiled psp toolchain, so that I could quickly check if I could build it. It requires glibc 2.34 iirc, so I used 22.04. It built fine. It didn't like the jobs command though, so it took a while. Built, no errors, no needing to change the code. Works fine, but I have no games, only tested if it booted.

Hi, just to keep you in the loop. I manage to build all versions of scummvm, the issue right now is to determine why we get the file system lock (when running scummvm-2.5.1+ on the PS4). We have two theories right now:
A) The problem could be that scummvm-2.5.1+ got bumped to be built with C++11 and the PSPonPS4 doesn't like that for some reason
B) The modifications I've successfully done up 2.5.1 is no longer compatible with scummvm-2.5.1+, causing the files system to act locked
C) scummvm-2.5.1+ has got changes to how the file system protects/acts and reads/writing files/folders for the emulated UMD-drive "disc0:/".

 

[Charles_n_town]

Hi, just to keep you in the loop. I manage to build all versions of scummvm, the issue right now is to determine why we get the file system lock (when running scummvm-2.5.1+ on the PS4). We have two theories right now:
A) The problem could be that scummvm-2.5.1+ got bumped to be built with C++11 and the PSPonPS4 doesn't like that for some reason
B) The modifications I've successfully done up 2.5.1 is no longer compatible with scummvm-2.5.1+, causing the files system to act locked
C) scummvm-2.5.1+ has got changes to how the file system protects/acts and reads/writing files/folders for the emulated UMD-drive "disc0:/".

Yeah, I didn't try 2.5.1, but I had to update the psp toolchain for 2.6.0 due to c++11. Then, that forced me to use Ubuntu 22.04 due to glibc 2.34+ beinga requirement. I usually use 20.04. I hope you all can figure it out though. I don't own a ps4 anymore to test.

 

[jacobsson]

i wonder about that too......
on the plus side i finally got it compiled (the gcc9 toolchain), the libraries have compiled but for some reason dont show in the configuration of scummvm causing it to fail massively, i probably messed up somewhere :\ anyway it has the exact same problems as the gcc8 build (fails on C++11 or Endianness check), So i kinda feel i wasted a whole day to compile this steaming pile of poo.... oh well
Good luck on trying to compile on the gcc-8 build, let me know how it goes.

Hi! I just managed to make a build autoboot a game that relies on C++11:
Commit msg: PSP: Remove standard libraries
Hash: 652846cac0aafa05bfe03956098a9813bec07b6a
I redid this build and made it fail at file reading too, back to searching!

This one fails during file reading:
Commit msg: PSP: Remove legacy PSP_HEAP_SIZE_KB macro
Hash: df48fa786197711ce458680c2c6905bb847dff2c

Both commit May 2, 2022

So that is good news! Now to narrow down the offending commit while using the latest toolchain

 

[Devildwarf]

Hi! I just managed to make a build autoboot a game that relies on C++11:
Commit msg: "Removed standard libraries"
Hash: 652846cac0aafa05bfe03956098a9813bec07b6a

So that is good news! Now to narrow down the offending commit while using the latest toolchain

Nice thats great progress, it looks like minimal changes there too, wouldnt be hard to revert, hopefully you can now build the latest version if you change that info back.

And yeah Charles im in the same boat, im just trying to help out here where i can, my PS4 isn't modded so i cant help on that side either.

 

[Charles_n_town]

Nice thats great progress, it looks like minimal changes there too, wouldnt be hard to revert, hopefully you can now build the latest version if you change that info back.
View attachment 42521
And yeah Charles im in the same boat, im just trying to help out here where i can, my PS4 isn't modded so i cant help on that side either.

Yeah, I was in the scenes for years, but I left due to them being too toxic. I'm 100% legit now. I've been selling all my hacked consoles as a result, should be selling my cfw ps3 today in fact. I'm mostly interested in the challenge of solving problems, especially compiling ones, for any console. That's about all I do now. I can't test my stuff, but there are plenty of people willing.

 

[jacobsson]

I recently mentioned that I might've found the breaking commit above, but it seems like I didn't cleanly rebuild that code cleanly by running make distclean first, redid it and it turned out to be borked. Besides backtracking to find narrow it down I'm also looking into if building with plugins causes the PSPonPS4 emu to give me the infamous (due to files visible in file browser):

Could not find any engine capable of running the selected game

EDIT: Nope, did they same build without plugins, same problem. Let's backtrack some more:

EDIT2;
These commit works (pre C+11):

BACKENDS: Use nullptr
https://github.com/scummvm/scummvm/commit/675695c7088b5d6b2b3c03a779134fab6f7f52e8

ALL: Cleanup ResidualVM -> ScummVM
https://github.com/scummvm/scummvm/commit/20cb0ae22857f5c9cd61255d19ada89dacfb9ae1

"I18N: Update translations templates"
https://github.com/scummvm/scummvm/commit/0d0951e47a8cf7274ea8ed1df033fe30a8603383

The commit right after gives us the file read problems!

"PSP: Fix build failures with latest PSP toolchain"
https://github.com/scummvm/scummvm/commit/675695c7088b5d6b2b3c03a779134fab6f7f52e8

@Devildwarf this could very well be some kind of C++11 problem as you have mentioned before.
We're getting closer!

EDIT:
NOTE: the latter commit can be built with both CC+-4.6 and 11, which is interesting.
4.6 build works on PS4!
11 build even gives the "could not find" error on PPSSPP

 

[jacobsson]

Now that I seem to have found the commit it's time to do some digging.
To be continued tomorrow!

The latter mentioned commit above can be built with both gcc-4.6 and gcc-11.
gcc-4.6: works and boots games
gcc-11 (or 13+ in my case): gives me the "couldn't find any engine", even in PPSSPP

NOTE: Both are built as plugin-releases and only sci enabled for testing.

I'm now looking into rebuilding the gcc-11 version with all plugins enabled to see if that changes anything.

 

[Devildwarf]

Now that I seem to have found the commit it's time to do some digging.
To be continued tomorrow!

The latter mentioned commit above can be built with both gcc-4.6 and gcc-11.
gcc-4.6: works and boots games
gcc-11 (or 13+ in my case): gives me the "couldn't find any engine", even in PPSSPP

NOTE: Both are built as plugin-releases and only sci enabled for testing.

I'm now looking into rebuilding the gcc-11 version with all plugins enabled to see if that changes anything.

That does sound more like it could be a toolchain problem then instead of a source code problem (or the exceptions made for different GCC ?), that is interesting, and C++11 support was added to GCC in 4.8.1 so any version over that should be able to compile the C11++ version (although for some reason scummvm doesn't detect it in our GCC8 build ?) which makes me wonder if its somehow excluded or just the check fails in scummvm configuration.

 

[jacobsson]

I think you're right!
This is kind of alarming when using the new toolchain:

Warning: could not fixup imports, stubs out of order.
Ensure the SDK libraries are linked in last to correct this.
Continuing, your binare may or may not work.

@Devildwarf Any ideas on this?

 

[Devildwarf]

strange error, could be related to the issue though, have you tried messing with the library order ? and this only happens on the latest toolchain ? it could be a problem with that particular version, it might be worth trying some differnet versions of the toolchain to see if that error happens on each.
EDIT:- I will have a go at building it on the latest toolchain then go back a few versions and see what happens.

I think you're right!
This is kind of alarming when using the new toolchain:



View attachment 42549
@Devildwarf Any ideas on this?

OK found the reason, its the psp.spec file in the backends/platform/psp folder, the order of the libraries is bad and a few redudant ones, try changing it to

%rename lib    old_lib
*lib:
-lstdc++ -lpspprof -lpspvfpu -lpspdebug -lpspgu -lpspge -lpspdisplay -lpspctrl -lpsppower -lpsphprm -lpspaudio -lpspaudiocodec %(old_lib)

and it should fix the error

EDIT:- Also just noticed i double posted instead of editing my old post, sorry Admins/Moderators

 

[Charles_n_town]

OK found the reason, its the psp.spec file in the backends/platform/psp folder, the order of the libraries is bad and a few redudant ones, try changing it to

%rename lib    old_lib
*lib:
-lstdc++ -lpspprof -lpspvfpu -lpspdebug -lpspgu -lpspge -lpspdisplay -lpspctrl -lpsppower -lpsphprm -lpspaudio -lpspaudiocodec %(old_lib)

and it should fix the error

EDIT:- Also just noticed i double posted instead of editing my old post, sorry Admins/Moderators

Glad you figured it out. Same error when building infinity iirc with the wrong toolchain. I knew it was fixable in the Makefile, but I didn't bother trying to. That's with infinity though, not scummvm.

 

[jacobsson]

OK found the reason, its the psp.spec file in the backends/platform/psp folder, the order of the libraries is bad and a few redudant ones, try changing it to

%rename lib    old_lib
*lib:
-lstdc++ -lpspprof -lpspvfpu -lpspdebug -lpspgu -lpspge -lpspdisplay -lpspctrl -lpsppower -lpsphprm -lpspaudio -lpspaudiocodec %(old_lib)

and it should fix the error

EDIT:- Also just noticed i double posted instead of editing my old post, sorry Admins/Moderators

I was also thinking it's that change that broke something, do you have any idea why they did this?
I'm building a g++-11 build now, l'll report back as soon as I tested it

EDIT: sorry @Charles_n_town just realized you kinda pointed this out too, creds to you as well. I just didn't understand what was going on there.

 

[Devildwarf]

I was also thinking it's that change that broke something, do you have any idea why they did this?
I'm building a g++-11 build now, l'll report back as soon as I tested it

EDIT: sorry @Charles_n_town just realized you kinda pointed this out too, creds to you as well. I just didn't understand what was going on there.

I would guess it was a change in the toolchain that made it more important to stubs to be in the correct order maybe ? or maybe someone was messing with libraries and just screwed it up, its hard to say for sure, so many things are a factor when it comes to this, it does make me wonder how the unoffical SDK compares to the real SDK in this department too, i wonder if that would compile better or worse (or even just fail totally since scummvm was designed with the open source unofficial sdk in mind),
But i guess we can either try older open source SDK's, the earliest build of the current SDK appears to be 2020, it might be worth trying that one (if you don't want to install a VM just to test it you can always send me the source edited and i will try and compile on it and send you the result)

 

[jacobsson]

I would guess it was a change in the toolchain that made it more important to stubs to be in the correct order maybe ? or maybe someone was messing with libraries and just screwed it up, its hard to say for sure, so many things are a factor when it comes to this, it does make me wonder how the unoffical SDK compares to the real SDK in this department too, i wonder if that would compile better or worse (or even just fail totally since scummvm was designed with the open source unofficial sdk in mind),
But i guess we can either try older open source SDK's, the earliest build of the current SDK appears to be 2020, it might be worth trying that one (if you don't want to install a VM just to test it you can always send me the source edited and i will try and compile on it and send you the result)

Damn, right now I feel like I'm out of ideas. I don't why the breaking commit that allowed for both toolchains (8.6 and 13+) would build but 13+ won't link stuff like plugins correctly, if the unofficial sdk works that would kind of make sense. I have this patch attached that you could use for ease, I really appreciate it my friend!

 

[Devildwarf]

Damn, right now I feel like I'm out of ideas. I don't why the breaking commit that allowed for both toolchains (8.6 and 13+) would build but 13+ won't link stuff like plugins correctly, if the unofficial sdk works that would kind of make sense. I have this patch attached that you could use for ease, I really appreciate it my friend!

Thanks, im starting tests with the oldest build on the psp sdk github from 2020, here is a couple of scummvm 2.6.0 builds compiled with and without C++11 and with your patch above applied and a couple of tweaks to the configure file (to bypass the endianess check and C++ check), it appears to use gcc 9.3.0, hopefully 1 of them will work, if not then i will try one of the slightly later builds
https://drive.google.com/file/d/103G85X_wW5imCx9jj0DRumLPNdCNdaMW/view?usp=sharing
EDIT:- i should have probably started with the commit you mentioned above actually, only just now realised i was working on a later scummvm version, im too tired, its late here (and thats my excuse and im sticking to it)

 

[jacobsson]

Thanks, im starting tests with the oldest build on the psp sdk github from 2020, here is a couple of scummvm 2.6.0 builds compiled with and without C++11 and with your patch above applied and a couple of tweaks to the configure file (to bypass the endianess check and C++ check), it appears to use gcc 9.3.0, hopefully 1 of them will work, if not then i will try one of the slightly later builds
https://drive.google.com/file/d/103G85X_wW5imCx9jj0DRumLPNdCNdaMW/view?usp=sharing
EDIT:- i should have probably started with the commit you mentioned above actually, only just now realised i was working on a later scummvm version, im too tired, its late here (and thats my excuse and im sticking to it)

Thank you for the builds!
Unfortunately both prompts with "could not find any engine" message even in PPSSPP, did you build with all plugins included in the EBOOT (didn't see any plugin folders) or is there a specific engine I should test? (I tested with a SCI game)

 

[Devildwarf]

Thank you for the builds!
Unfortunately both prompts with "could not find any engine" message even in PPSSPP, did you build with all plugins included in the EBOOT (didn't see any plugin folders) or is there a specific engine I should test? (I tested with a SCI game)

opps sorry i didnt specify last night, i compiled with only scumm games in to speed up compile time, sorry i forgot to mention that, i run
./configure --host=psp --disable-all-engines --enable-engine=scumm,scumm-7-8,he
so any scumm games should work (like monkey island)

 

[jacobsson]

opps sorry i didnt specify last night, i compiled with only scumm games in to speed up compile time, sorry i forgot to mention that, i run
./configure --host=psp --disable-all-engines --enable-engine=scumm,scumm-7-8,he
so any scumm games should work (like monkey island)

That's good news, and no problem! I'll make sure to test during the day Thank you man

EDIT:
I managed to boot monkey island on the C+11 one in PPSSPP, but I don't think the patch is applied since it chooses create a new scummvm.ini to UMD instead of reading from disc0:/PSP_GAME/SYSDIR/scummvm.ini. So naturally it gives black screen on the PS4 since it needs the video fix as well.

The non C++11 did not boot in PPSSPP or PS4 (blackscreen).

 

[Devildwarf]

That's good news, and no problem! I'll make sure to test during the day Thank you man

EDIT:
I managed to boot monkey island on the C+11 one in PPSSPP, but I don't think the patch is applied since it chooses create a new scummvm.ini to UMD instead of reading from disc0:/PSP_GAME/SYSDIR/scummvm.ini. So naturally it gives black screen on the PS4 since it needs the video fix as well.

The non C++11 did not boot in PPSSPP or PS4 (blackscreen).

Thats strange, i just looked my the source and it looks like the patch is applied fine

so not sure why it would do that, maybe some other change in scummvm 2.6.0 breaks that change ? or another toolchain issue ? this might need to be looked into (and could possibly be part of the problem)

EDIT:- looking at the generated ELF it does seem to still be looking at ms0:/scummvm.ini for some strange reason, going to do a clean build and make sure nothing got messed up
EDIT:- yeah Clean build fixed the problem but its also identical to the none C++11 build now, not sure what is going on.....

 

[jacobsson]

Thats strange, i just looked my the source and it looks like the patch is applied fine
View attachment 42555
so not sure why it would do that, maybe some other change in scummvm 2.6.0 breaks that change ? or another toolchain issue ? this might need to be looked into (and could possibly be part of the problem)

EDIT:- looking at the generated ELF it does seem to still be looking at ms0:/scummvm.ini for some strange reason, going to do a clean build and make sure nothing got messed up

Yeah I've been fooling myself with some builds too, now I always do a make distclean and a configure to avoid any old data being used