PS3 Help compiling movian from source

[AlexRhine]

After finally being able to successfully setting up the latest ps3toolchain on my machine, I thought I would try my hand at compiling some projects to test it out. I picked Movian, which in hindsight was terrible decision. The default build script downloads and uses a separate toolchain with what appears to be PSL1GHT v1. It uses gcc 4 instead of 7, and the files are set up quite differently than current versions of the toolchain.

The build scripts themselves required a few patches. I had to fix some links that no longer worked, delete the fdiagnostic-color flag one from one of the submodules (libav) since the option isn't available in gcc4, and prevent the build script from auto-updating the submodules (which would cause the fdiagnostic flag to pop up again).

I also needed to replace some binaries from the downloaded toolchain with binaries built by the current toolchain ($(PS3DEV)/bin/) because they expected my system to have libcrypto.so.1.0.0, whereas the new binaries used libcrypto.so.1.1. Without this change, the self file was being generated incorrectly.

Now I am able to compile a pkg for movian (from the release/5.0 branch), but when I install and boot it up it just throws me back to XMB. And I have no idea how to debug this. Is it a problem with the downloaded toolchain running in my system, or is it a problem with using different binaries to make self files? Or maybe even something else in the source code?

I guess what would be appropriate in this case would be to port the ps3 build of movian to the newer toolchain/psl1ght; but alas, how to do so is currently beyond me.

Any ideas?

 

[pinky]

I hear ya, man. I'm having issues compiling wfs-extract for the wii u to decrypt my mlc partition or the internal storage basically. there's a problem with the original where it fails if it comes into contact with a folder with upper case letters. I get a little further each time. right now, I have a problem with boost. I'm trying to compile the project with visual studio 2019, and it's giving me issues. I don't think I need an sdk in this case, just certain libraries like windows sdk 10, because math.h doesn't seem to be part of 8.1, which this was written for. it's a damn headache.

 

[AlexRhine]

If anyone's interested, this is the compiled SDK (PSL1GHT + PS3Toolchain + A few libraries) that movian uses. It's compiled for Linux, and it seems to run fine on Debian 11 except for the binaries that require libcrypto.so.1.0.0. It might be useful to add it to the list of pre-compiled PSL1GHT SDKs on the PS3 dev wiki.

The biggest difference I've noticed compared to newer PSL1GHT is the presence and use of lv2-psl1ght.o in /host/ppu/ppu/lib. Other differences are file/folder structure and "missing" libraries. I'm guessing this is v1 of PSL1ght. I wonder, why was movian never ported to v2? Perhaps not enough interest?

Anyway, due to using an older toolchain/sdk, movian build scripts also compile their own version of zlib and bzip2. I would guess that my build problem is directly related to these libraries (since, on boot, movian is supposed to unzip an archive that's included in the elf).

 

[Checkity_CheckYoSelf]

If anyone's interested, this is the compiled SDK (PSL1GHT + PS3Toolchain + A few libraries) that movian uses. It's compiled for Linux, and it seems to run fine on Debian 11 except for the binaries that require libcrypto.so.1.0.0. It might be useful to add it to the list of pre-compiled PSL1GHT SDKs on the PS3 dev wiki.

The biggest difference I've noticed compared to newer PSL1GHT is the presence and use of lv2-psl1ght.o in /host/ppu/ppu/lib. Other differences are file/folder structure and "missing" libraries. I'm guessing this is v1 of PSL1ght. I wonder, why was movian never ported to v2? Perhaps not enough interest?

Anyway, due to using an older toolchain/sdk, movian build scripts also compile their own version of zlib and bzip2. I would guess that my build problem is directly related to these libraries (since, on boot, movian is supposed to unzip an archive that's included in the elf).

It would be great if it would offer better compatibility with more codecs. Any possibility of this?

 

[bucanero]

The latest PSL1GHT toolchain is probably different in too many ways with the custom/old toolchain used with Movian. I'd suggest to use their toolchain, and if needed install it on a Linux virtual machine.

replacing and fixing everything could be a huge effort... also in the last year or so, I've seen many changes to psl1ght related to compiler sizes, flags, pointers, and other low level stuff, that I'm pretty sure that will generate completely different binaries that can't be swapped correctly with a previous toolchain.
Maybe you can try using the old gcc 4 toolchain from Estwald/Hermes, the "PSDKv2"... that's might be closer than the current one.

Also on a side note, recently I've been rebuilding the toolchain myself, and at least in my setup, even simple samples were not booting correctly, and my own homebrews would crash and go back to XMB.

I ended up back-tracking to my own older builds of the ps3toolchain, the 2020-07-17 build works correctly:
https://github.com/bucanero/ps3toolchain/releases

(you can also try the autobuild from 2020-08-31, but it was giving me some issues too)

 

[AlexRhine]

Also on a side note, recently I've been rebuilding the toolchain myself, and at least in my setup, even simple samples were not booting correctly, and my own homebrews would crash and go back to XMB.

I ended up back-tracking to my own older builds of the ps3toolchain, the 2020-07-17 build works correctly:
https://github.com/bucanero/ps3toolchain/releases

(you can also try the autobuild from 2020-08-31, but it was giving me some issues too)

Hmm, I compiled my toolchain from the latest source from ps3dev, using a fork of your version of ps3libraries. I'll have to test it to check if it actually works correctly. If not, I'll give the pre-compiled ubuntu build a try. Thanks for the heads up!

As for compiling movian, I have no idea why it fails to boot. I'll try to set up a VM with an older version of ubuntu and check if it compiles correctly. Or maybe I'm using source code that contains a commit which breaks the PS3 build (which would be odd, since movian's checks indicate the PS3 builds for that version were successful). I'll try using older versions of the submodules, and if it doesn't work, I'll try compiling with source from the initial 5.0 release (from 2016). I'll try to avoid the last option.

 

[bucanero]

Hmm, I compiled my toolchain from the latest source from ps3dev, using a fork of your version of ps3libraries. I'll have to test it to check if it actually works correctly. If not, I'll give the pre-compiled ubuntu build a try.

I tried that exact same way ~2 weeks ago: using the latest ps3dev/ps3toolchain repo, and my own fork of ps3libraries. Most compiled binaries failed to run at all, and some had weird glitches. I was using the latest ps3dev/PSL1GHT too.
As I was building the toolchain on a different arch (arm64) I wasn't completely sure if it was the toolchain or my host... I didn't had a chance to test the same procedure in a standard intel x86 arch.

A quick check that I did was testing to compile and run PSL1GHT/samples/graphics/blitting . If the blitting.self failed, then the toolchain was broken. You can use my build of PS3LoadX and send the self over the network. If it doesn't work, then try another version.

 

[AlexRhine]

I've finally been able to compile a working pkg of movian. I set up a VM using Ubuntu Xenial and used a slightly-patched version of movian's master branch in github. I also tested building a pkg with the movian6 branch, but it wouldn't boot once I installed it on my PS3. I also wanted to set up Github actions to build the pkg automatically, but Xenial environments were removed a couple of days ago.

About my patches, I simply updated some non-working tarball links, removed the weird sfo/icon overwrite feature from the ps3 build, and changed the CATEGORY, TITLE_ID, and ICON0 to something different (both for aesthetic reasons and to differentiate my build from the official one; really this is the reason why I wanted to compile movian in the first place).

I also set up a mirror of Movian's toolchain on github. I'll add a note sometime later today to specify that it needs to run on Ubuntu Xenial to work correctly.

Note to self: Always try to set up a VM/container when using pre-compiled toolchains.

Edit: Forgot to mention, my compiling problems seemed to arise from running the build script on a recent version of Debian. The fdiagnostic warning I was getting wasn't supposed to occur in the normal build, so somewhere along the way a check was passing which shouldn't have passed. Well, that's what I get for not setting up a VM in the first place.

A quick check that I did was testing to compile and run PSL1GHT/samples/graphics/blitting . If the blitting.self failed, then the toolchain was broken. You can use my build of PS3LoadX and send the self over the network. If it doesn't work, then try another version.

I'll try to test out my compiled toolchain (x86 arch) today, and report back. It might be beneficial to check whether the issues arise from the different architecture or not.

 

[AlexRhine]

I tried that exact same way ~2 weeks ago: using the latest ps3dev/ps3toolchain repo, and my own fork of ps3libraries. Most compiled binaries failed to run at all, and some had weird glitches. I was using the latest ps3dev/PSL1GHT too.

Alright, I just tested compiling Apollo with the toolchain I have set up, and after installation it won't run at all. I'll try using with the pre-compiled toolchain you suggested in a VM and see how it goes. It was compiled on Ubuntu 20.04, right?

 

[bucanero]

Alright, I just tested compiling Apollo with the toolchain I have set up, and after installation it won't run at all. I'll try using with the pre-compiled toolchain you suggested in a VM and see how it goes. It was compiled on Ubuntu 20.04, right?

The pre-built toolchains from my repo (https://github.com/bucanero/ps3toolchain/releases) were built with the Ubuntu image available on GitHub , I think it's either ubuntu 18 or 20. Not really sure which one as it was done over a year ago.

Check the 07-17 build first, it doesn't include newer libs from ps3libraries but the compiler works properly (blitting.self runs correctly)

 

[AlexRhine]

The pre-built toolchains from my repo (https://github.com/bucanero/ps3toolchain/releases) were built with the Ubuntu image available on GitHub , I think it's either ubuntu 18 or 20. Not really sure which one as it was done over a year ago.

Check the 07-17 build first, it doesn't include newer libs from ps3libraries but the compiler works properly (blitting.self runs correctly)

Apollo now compiles properly with the 2020-07-17 toolchain (after running the latest ps3libraries scripts) in the VM I set up (Ubuntu 20.04). Thanks for the help.

By the way, I now realize that the sfo/icon overwrite feature in movian was probably include for a similar reason that you include the modification protection of the sfo, icon, and eboot in PKGi.

 

[bucanero]

Apollo now compiles properly with the 2020-07-17 toolchain (after running the latest ps3libraries scripts) in the VM I set up (Ubuntu 20.04). Thanks for the help.

By the way, I now realize that the sfo/icon overwrite feature in movian was probably include for a similar reason that you include the modification protection of the sfo, icon, and eboot in PKGi.

thanks for the confirmation about the 07-17 toolchain. I went ahead, rebuilt everything under my arm64 macOS using the scripts from the 07-17 branch and everything worked as expected. For ps3libraries I used my own (current) fork. I was able to build my pkgi, apollo, the blitting examples, and all the binaries worked as expected.
I can confirm that the issue wasn't related to my new host arch (arm64), but to the latest toolchain/psl1ght being broken.

I think that in the last year the PS3DEV developers added/changed some stuff that maybe wasn't properly tested... the big pain here is to find which change broke psl1ght... so for now I'll stick with my toolchain based on 2020-07-17.

Yes, Movian probably was doing it for a similar reason, but it looks like their solution was different: the app restores the original files when it starts. (I just notify the user and quit the app)