PS3 [RESEARCH] Playstation Home Developer Build - Information for fixing and modding stuff

[DeViL303]

I have some ideas to address that... What XML did you change, the xmb one? I was able to add the folding@home icon back (thanks to you and @LuanTeles ), so I'm guessing it's the same principle here?

Yes. The net_package_install method. Or net_package_install_with_eula in this case.

 

[marciolsf]

Yes. The net_package_install method. Or net_package_install_with_eula in this case.

Ah, perfect, that's how I had my xml file written. Do you know of a guide somewhere on how to convert the contents of the zip into a pkg file? From the looks of it, the folder structure is correct, I just need to convert it into .pkg

edit: found AldosTools, I should've known that's how we do it…

 

[RoboKing's Cosmos]

I've tried PlayStation Home years ago before it got shut down by $ony glad you guys are reviving it after all these years.

 

[DeViL303]

Ah, perfect, that's how I had my xml file written. Do you know of a guide somewhere on how to convert the contents of the zip into a pkg file? From the looks of it, the folder structure is correct, I just need to convert it into .pkg

edit: found AldosTools, I should've known that's how we do it…

Which zip is that? The 20GB one? or 30GB one?

If its those they need more work than that. I can go through it though if its those you are talking about.

 

[marciolsf]

Which zip is that? The 20GB one? or 30GB one?

If its those they need more work than that. I can go through it though if its those you are talking about.

It's the 20GB "1.86 offline client 2.0"

 

[DeViL303]

It's the 20GB "1.89 offline client 2.0"

OK. So to go from that zip to pkg you need to do a few things.

1. So you need to run a tool on it to make all file, file extension and folder names UPPERCASE. The ps3 will not find any file with a lower case letter in the path. I use Bulk rename tool, I also have a command line tool that will do everything quickly.

2. Delete all the BAR and SELF files, basically delete everything that is not in a folder. Then the easiest thing to do is take the BARs and EBOOT.BIN from the HEN version DH released. Copy those into the USRDIR folder.

3. Make sure the PARAM.SFO category is set to HG (or HM if you want it to show up in the PSN category), make sure it is set to bootable.

4. Use this attached tool to pack the pkg. (custom pkg tool by lmn and aldostools)

 

[DeViL303]

BAR files and SDAT files:

Something interesting about these archives, they have no TOC.. each file is named by a 8 Character hash like CRC-32. The hash used for file names inside the BARs is a hash of the real original filename and path before it was packed, and that we can not reverse. So basically the only way to figure out what the name of a file was originally, is to find it's path and filename somewhere else, and then try that filename and see if it matches the hash.

So it's this strange chicken/egg situation, where you need to already know the filename, to try it against the file and see if its the right one.

So when the eboot looks for file, it just seems to us like it is looking for a normal path, but behind the scenes it is actually hashing the filename, then checking for files that match that hash.. seems like a really crap way to do it, but maybe it was meant to obfuscate things, or there was some other reason.

Thanks a million to Bucanero for help with the BAR format.

 

[marciolsf]

OK. So to go from that zip to pkg you need to do a few things.

1. So you need to run a tool on it to make all file, file extension and folder names UPPERCASE. The ps3 will not find any file with a lower case letter in the path. I use Bulk rename tool, I also have a command line tool that will do everything quickly.

2. Delete all the BAR and SELF files, basically delete everything that is not in a folder. Then the easiest thing to do is take the BARs and EBOOT.BIN from the HEN version DH released. Copy those into the USRDIR folder.

3. Make sure the PARAM.SFO category is set to HG (or HM if you want it to show up in the PSN category), make sure it is set to bootable.

4. Use this attached tool to pack the pkg. (custom pkg tool by lmn and aldostools)

Thanks! I was having difficulties running the other pack tool (yay python), so this will definitely help. I know I saw the HEN version somewhere, I'm looking for that now. Edit: Found it!

 

[DeViL303]

Dumped and mapped some interesting system files today. Well, the names seem interesting, no idea what they do yet lol.

 

[aldostools]

BAR files and SDAT files:

Something interesting about these archives, they have no TOC.. each file is named by a 8 Character hash like CRC-32. The hash used for file names inside the BARs is a hash of the real original filename and path before it was packed, and that we can not reverse. So basically the only way to figure out what the name of a file was originally, is to find it's path and filename somewhere else, and then try that filename and see if it matches the hash.

So it's this strange chicken/egg situation, where you need to already know the filename, to try it against the file and see if its the right one.

So when the eboot looks for file, it just seems to us like it is looking for a normal path, but behind the scenes it is actually hashing the filename, then checking for files that match that hash.. seems like a really crap way to do it, but maybe it was meant to obfuscate things, or there was some other reason.

Thanks a million to Bucanero for help with the BAR format.

Interesting work of reversing... from your description what I understand is that the hashes in the BARs are used to authenticate the files, not to seek them. The hash probably is used as some kind of "signature" to ensure the integrity of the content against illegal modifications. It could be a CRC64, or a truncated hash like SHA1 or MD5, or a custom hash (which is harder to figure out).

 

[DeViL303]

Interesting work of reversing... from your description what I understand is that the hashes in the BARs are used to authenticate the files, not to seek them. The hash probably is used as some kind of "signature" to ensure the integrity of the content against illegal modifications. It could be a CRC64, or a truncated hash like SHA1 or MD5, or a custom hash (which is harder to figure out).

No it's not really like that, There are no proper filenames in the BAR. The files are named with a hash of the original file name and path. The hash is done on the whole path in lowercase, even though the PS3 accesses all files in uppercase. It's a strange set up.

The only way to find the name is to already know it and see if it matches.

It's not some protection either, cos I can swap any file I want.. as long as it has the same path and filename before packing the BAR, then it will have the same hash and filename inside the BAR and the system does not care.

 

[DeViL303]

It's kind of hard to explain and I am explaining it badly.

So lets say I extract a BAR file.. I end up with this kind of thing.

Now lets say I have a fair idea that there is a \hubosd\Purchases\ControllerPluggedInCheck.xml file in that folder somewhere but I do not know which file it is. What I can do is hash that string "\hubosd\Purchases\ControllerPluggedInCheck.xml" , see what the 8 character result is.. If the resulting 8 chars matches any file name in the folder, then that is the REAL name and path of that file.

Not sure if a system like this is used anywhere else but its weird to me.

 

[marciolsf]

No it's not really like that, There are no proper filenames in the BAR. The files are named with a hash of the original file name and path. The hash is done on the whole path in lowercase, even though the PS3 accesses all files in uppercase. It's a strange set up.

The only way to find the name is to already know it and see if it matches.

It's not some protection either, cos I can swap any file I want.. as long as it has the same path and filename before packing the BAR, then it will have the same hash and filename inside the BAR and the system does not care.

is the hashed path based on the ps3 filesystem? /dev_hdd0/ and all that

 

[DeViL303]

is the hashed path based on the ps3 filesystem? /dev_hdd0/ and all that

It's based on everything after "dev_hdd0/game/xxxxxxxxx/USRDIR/", So the hashes are just based on the end of the paths from the root game folder down, on PC when serving it via TM it would be the build folder. So basically it means you can run it from any folder as it ignores the first part of the path.

 

[DeViL303]

So it's a bit like there is a "path translator" built into the EBOOT. So let's say a user clicks on a scene. Lets say the first file called by that click is:

dev_hdd0/game/NPIA00010/USRDIR/ENVIRONMENTS/MY_SCENE/AMAZING.SCENE

So before accessing the file, the "path translator" trims that down to

ENVIRONMENTS/MY_SCENE/AMAZING.SCENE

then makes it lower case.

environments/my_scene/amazing.scene

Then hashes it to

4968415C

Then it will look for file called 4968415C in the BAR.. this is all totally invisible to the end user. Now I might have something wrong but this is the way it seems to work so far.

For example I could replace a 10KB AMAZING.SCENE file with a 100MB mp4, and when I create a bar with that file named to AMAZING.SCENE, it will have the exact same hash for the filename when I extract it again.

 

[marciolsf]

So it's a bit like there is a "path translator" built into the EBOOT. So let's say a user clicks on a scene. Lets say the first file called by that click is:

dev_hdd0/game/NPIA00010/USRDIR/ENVIRONMENTS/MY_SCENE/AMAZING.SCENE

So before accessing the file, the "path translator" trims that down to

ENVIRONMENTS/MY_SCENE/AMAZING.SCENE

then makes it lower case.

environments/my_scene/amazing.scene

Then hashes it to

4968415C

Then it will look for file called 4968415C in the BAR.. this is all totally invisible to the end user. Now I might have something wrong but this is the way it seems to work so far.

For example I could replace a 10KB AMAZING.SCENE file with a 100MB mp4, and when I create a bar with that file named to AMAZING.SCENE, it will have the exact same hash for the filename when I extract it again.

ah, thats a really good way to explain it! I think it's starting to click. I just re-read this post

When PS Home normally runs scenes, it mounts the contents of the SDAT to the file system. Then when you load another scene, it unmounts the first one, and mounts the next SDAT (SDAT is basically an encrypted BAR file, BAR files are mounted in the same way)

So the bar/dat/sdat files are mounted on the fly, but probably to make the files smaller and save space, the files within the archive are renamed based on the results of the hash. When the files are decompressed during the mount, instead of renaming them back (which might require some kind of look up), its faster to calculate the file name hash and then look up the resulting path/file name.

It all sounds a bit janky, but it's probably really fast to calculate, plus if it makes the files smaller it's better all around.

 

[DeViL303]

It wont really make any difference to the size, a few bytes I guess but that is not why they do it. Not sure why they do it really, obfuscation maybe, slow the modders down a bit maybe.

 

[marciolsf]

Well, after a few days of IRL stuff getting on the way, I finally completed all the steps you suggested (renames, clean files, etc) and now I have home booting up on RPCS3. I'm having some weird graphical glitches in the home apartment, but that's probably my video card. It failed to boot a few times, but updating to the latest version of RPCS3 took care of that.

Progress, finally! This is exciting.

 

[DeViL303]

Playstation Home Tip of the day..

In your HUBSTARTUP.TXT you can use a ContentServerOverride like:

ContentServerOverride file://CONTENTSERVER/

Then you do not need a CDN or a web server for anything, everything can be read locally from that folder with max performance.

I don't think many people know you can use "file://" in this way to replace URLs (in Playstation Home).

 

[aldostools]

Playstation Home Tip of the day..

In your HUBSTARTUP.TXT you can use a ContentServerOverride like:

ContentServerOverride file://CONTENTSERVER/

Then you do not need a CDN or a web server for anything, everything can be read locally from that folder with max performance.

I don't think many people know you can use "file://" in this way to replace URLs (in Playstation Home).

Hmmm... now I wonder if the webkit or some other sprx is blacklisting "file://" to prevent direct access to local resources from XML or web browser.

If we could whitelist it, some web files & javascripts could be hosted locally without a web server like webMAN or an online connection. Also the size limitation in some XML could be a thing of the past.