PS3 PS3 HDD Decryption Helper 2023-12-07

[bleepbleep]

sudo rm -rv ${HOME}/ps3/storage/hdd/dev_hdd1 *

the remove command syntax for removing multiple files::

rm file1.txt file2.txt

will remove file1.txt and file2.txt in the current directory

therefore:

rm -rv bleepbleep/ps3/storage/hdd/dev_hdd1 *

will probably remove "bleepbleep/ps3/storage/hdd/dev_hdd1" and "*"

I suggest not use space in path names for remove command.

I apologize for multiple edits of this post due to not being good at composing clear and concise explanations.

 

[Berion]

@Iridule In current form, for sure not because mount points are not the same.

Use PS4 HDD Decryption Helper and mount all partitions, go tu /update (partition 25) and remove all data from there (but also make copy, not contents inside but sector by sector this whole partition via dd - just in case). It is also FAT32 like in PS3 case so probably You can use the same script part but with correction on different partition and mount point than in PS3 case. Take in mind that Unmounter script needs updating like in PS3 case week ago (changed umount not from mount point but from device).

That's theory because I never have hacked PS4 (I have two: one Slim with broken ODD on fw 8.03 and Fat on newest fw so there is no way for me to hack them). Also no one was interested in giving me full disk to check if everything is ok. I based only on small dumps from each partition.

@bleepbleep Yes, but I don't want remove some specific files/dirs but everything. That's why asterisk I used. If in preview version, rm treat it what I wrote as two paths, it should remove "dev_hdd1" only and trying remove "*". But instead, he wipe out whole "~/ps3" contents, not to mention, it works fine before (or maybe it never was but it was my mistake during writing script ;]). So I'm still confused by that.

- - -
BTW: I wanted update mounting to covering up OtherOS but I'm stuck. Any help appreciated. ^^
https://www.psx-place.com/threads/bootloaders-doesnt-see-usb-and-dvd.39650/

From what I've seen, OOS is bootloader written to vflash5 and one partition sacrificed for HDD (from Linux level on PS3 point of view, from our it will be partition with sub-partitions, so the idea with dev_hdd2 turned to be wrong).

 

[Berion]

Berion updated PS3 HDD Decryption Helper with a new update entry:

PS3 HDD Decryption Helper (2023-03-29)

• Cosmetic changes in some scripts.
• Added "Activator Maker" script and icons for making activators leading to Mounter, Umounter and Backuper scripts.
• Added "PS3 HDD Backuper" script for copy (synchronizing) important user data.
• Added "PS3 HDD Dumper" script for make disk, partitions or sub-partitions images.
• Added "PS3 HDD Mounter (Decrypted)" and "PS3 HDD Umounter (Decrypted)" for mounting and unmounting decrypted storage.
• Added all scripts versions listing in "PS3 HDD Reporter" script.
• Added predefined LBA values for HPA setting up in "PS3 HDD Tasker" script for minimal and maximum HDD size supported by OFW.

Read the rest of this update entry...

 

[Berion]

Warning no.1: "PS3 HDD Backuper" script based on rsync. It is not simple copier, it copying data but also copying all file system attributes and time stamps. So access to some data needs root privileges like on mounted HDD. And speaking about time stamps: users with dead CMOS in PS3 shouldn't use the same backup path because rsync based on date & time written in source and target filesystem, and syncing changes in files based on that information. So with dead CMOS, on each boot, those settings back to default state and eg new saves or trophies will get such date. This will fool rsync, leading to data corruption in backup or not backup some data! If Your battery died, always use different backup path. Later I will make alternate version based on "cp" (simple copy) instead "rsync" for those users.

Warning no.2: In HPA setting up, take in mind that USB controllers often not support all ata commands/atapi standard. It is possible that it will not be possible to change HPA using them. Direct SATA or eSATA/poeSATA connection (but without USB/FW in the middle) to motherboard (eventually through PCIe controllers) is recommended. Some BIOSes and UEFIs lifting HPA in next boot after detected without ask or notify user. It will not be an issue if You already set it up and formatted HDD on PS3 because her partition table covered sectors set by HPA, but can be fooled user if he reboot computer with left HDD connected or when user decided for some reason to format HDD again on PS3 but is unaware that HPA was silently lifted.

- - -

I have also updated screenshots in resource section to cover up scripts etc. in current toolkit version.

- - -

BTW: Cool activators preview (aka shortcuts in Windows):

 

[60fpshacksrock]

@Berion
Is there a tool that can create a raw PS3 HDD backup?

 

[Berion]

@60fpshacksrock On Linux you can use PS3 HDD Decryption Helper, the Dumper script, option no.1:

It using dd, so if you are familiar with this app, of course you can do it on your own.

On Windows use: IsoBuster (right mouse click on device in left panel), DMDE (copy sectors option, from 0 to end) or HDD Raw Copy Tool (but do not compress image to *.imgc, you will gain nothing).

 

[60fpshacksrock]

@Berion

What I meant was: Is there a PS3 homebrew app that can do this? Basically: dump to USB, maybe an NTFS drive.

I know the PS3 has an option to backup the files, but I have never used it.

 

[Berion]

@60fpshacksrock If You asking about SBS copy on PS3 side which doing image on USB, then there is not. Because there is no sense in that. Raw disk image could be useful only on this specific PS3, not another and on this specific HDD or any equal or larger in size, not smaller. Not to mention USB 2.0 is damn slow and i.e 1TiB HDD would be backuping whole day! On PC with SATA-III or USB 3.0 this is max ~3 hours (depend of HDD read speed mostly, and target write speed of course).

If You are interesting in user data copy, then read this:
https://www.psx-place.com/threads/ultimate-userdata-backup-guide.29037/

Or ask Bucanero to make such app or enlarge Apollo functionality.

 

[60fpshacksrock]

@60fpshacksrock If You asking about SBS copy on PS3 side which doing image on USB, then there is not. Because there is no sense in that. Raw disk image could be useful only on this specific PS3, not another and on this specific HDD or any equal or larger in size, not smaller. Not to mention USB 2.0 is damn slow and i.e 1TiB HDD would be backuping whole day! On PC with SATA-III or USB 3.0 this is max ~3 hours (depend of HDD read speed mostly, and target write speed of course).

If You are interesting in user data copy, then read this:
https://www.psx-place.com/threads/ultimate-userdata-backup-guide.29037/

Or ask Bucanero to make such app or enlarge Apollo functionality.

My HDD on my fat model is only 80GB. Though I have a semi-dead slim with a 160GB HDD.
I just need to find the right torx screwdriver to fit them both and add the 160 to the fat model.

I have seen conflicting comments on different websites about which torx screwdriver is the right one.

And speaking of USB, my PC is from 2007-2008 and I am sure it's USB drivers are (were?) 2.0.
That said, my 2TB SSD is USB 3.0 or 3.1 with blue connectors, my PC writes to it at about 30MB's per second. Did MS somehow manage to add 3.0 drivers to 2.0 with updates?

Is that even possible?

 

[Berion]

It still be terribly slow. Real max speed for USB 2.0 which You can achieve is ~40MiB/s at best.

In CECHL04 is standard "cross" screw. I don't remember that CECH-2504 to have something different.

USB is backward compatible. If You connected 3.0 to 2.0 it will work but under 2.0 speed. It is physically impossible to pass that barrier. You need newer USB controller (so new motherboard or i.e USB 3.1 on PCIe card).

 

[Neal Cullum]

@60fpshacksrock On Linux you can use PS3 HDD Decryption Helper, the Dumper script, option no.1:

It using dd, so if you are familiar with this app, of course you can do it on your own.

On Windows use: IsoBuster (right mouse click on device in left panel), DMDE (copy sectors option, from 0 to end) or HDD Raw Copy Tool (but do not compress image to *.imgc, you will gain nothing).

Could i use option 1 or option 2 if i want to copy contents of HDD to restore it to a bigger HDD?

 

[gmipf]

@Berion The KO Manager isn't working on stock Ubuntu LTS 22.04 installation. (Option 6)

Right after unpacking the source it fails.

linux-hwe-5.19-5.19.0/ubuntu/include/README
 linux-hwe-5.19-5.19.0/ubuntu/ubuntu-host/Kconfig
 linux-hwe-5.19-5.19.0/ubuntu/ubuntu-host/Makefile
 linux-hwe-5.19-5.19.0/ubuntu/ubuntu-host/ubuntu-host.c
 linux-hwe-5.19-5.19.0/update-dkms-versions
 linux-hwe-5.19-5.19.0/update-version-dkms
 linux-hwe-5.19-5.19.0/virt/kvm/kvm_main.c
 linux-hwe-5.19-5.19.0/virt/kvm/pfncache.c
W: Download is performed unsandboxed as root as file 'linux-hwe-5.19_5.19.0-38.39~22.04.1.dsc' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
cp: cannot stat '/home/gmipf/ps3/apps/source/linux-5.19.0/fs/ufs/*': No such file or directory
warning: the compiler differs from the one used to build the kernel
  The kernel was built by: x86_64-linux-gnu-gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
  You are using:           gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
make[1]: *** No rule to make target 'fs/ufs/balloc.o', needed by 'fs/ufs/ufs.o'.  Stop.
make: *** [Makefile:1850: fs/ufs] Error 2


ERROR. Compilation failed!

In the script it also excepts a directory named

linux-$(uname -r | cut -d '-' -f1)

but I would replace that with

linux-*$(uname -r | cut -d '-' -f1)

since there are different flavors of kernels, if you don't add the star it only will support standard kernels. Ubuntu LTS 22.04 seems to default to the hwe kernel as an example. (At least after upgrading the kernel)

Other suggestions would be to not let the script work with files and directories as root. I don't want to use sudo to just remove the script directory afterwards. I also would suggest to use a more Linux standard archive like tar.gz instead of tar.7z

 

[gmipf]

Could i use option 1 or option 2 if i want to copy contents of HDD to restore it to a bigger HDD?

If you dump the hdd encrypted 1:1 (option 1) you still need to decrypt the dump afterwards. If you want to use the hdd contents e.g. copy to another hdd, you need a decrypted copy (option 2). In theory you could also just mount both ps3 hdds and copy the contents directly. But never tried any full transfer of user data though.

 

[Berion]

Could i use option 1 or option 2 if i want to copy contents of HDD to restore it to a bigger HDD?

You mixing two things in one sentence which a little confused me. Contents of HDD is not the same as HDD image. Contents are files and folders from filesystem(s), while SBS (sector by sector) image is whole structure, dumped as is (instead to sparse one, which is not possible to make in this case).

If You will create HDD image (must be encrypted for this task, so option no. 1, everything else results in recognizing HDD as empty) and restore it on larger HDD, You will not get extra space because custom partition table covering up old size and we don't have tools to edit both: partition table and UFS2 filesystem table to fit into new size. So it is kind a crippled cloning type and main purpose of "PS3 HDD Dumper" is not cloning but sharing image or just having full copy (i.e when user don't have/cannot dump EID Root Key, needed for decryption). Another issue is HDD assigning, so if You want restore disk image on different HDD, You must first making format of this HDD on target PS3 because doing that, she also assigning this HDD (noting model number, fw version and serial number of this HDD). Sony did everything they could to not allow user changing disks just like that without cleaning them first.

For You is "PS3 HDD Backuper" script which will copy important data from PS3 HDD and which You need to restore it by hand at this moment (from USB using file manager on PS3 side or FTP client - because I didn't write yet restoring script).

Take in mind that this script wasn't very well tested and just in case, user should check after copying if everything has been copied. Also take in mind that if You have dead CMOS battery in PS3, You shouldn't backup in the same dir after making changes because files aren't copy just like that, but syncing using rsync (it checks time stamps in files and copy only differences between target and source dir).

If You want do all of this manually, here is guide describing where is what exactly:
https://www.psx-place.com/threads/ultimate-userdata-backup-guide.29037/

 

[Berion]

@gmipf Oh, I fitting this for Mint kernel source. Thank You for suggestion, I will test it then and change that.

Looks like UFS source is incomplete. I will check that later (I'm Mint user so everything is fitting for Mint distribution, even live environment). Thanks for report.

By using my toolkit is not possible to mount two PS3 HDD at once because scripts searching for one pair of ata and vflash keys. It is easy change for someone who is familiar with Linux but I do not have in plans for that because introducing a lot of problems with file naming, "GUI" and tests.

I will look into that, don't remember specific case but I believe I've made it for a purpose but maybe it is just a mistake. ^^

psx-place not supporting *.gz extension, and gz is kind an ancient. Maybe *.xz would be better choice (it also using LZMA like *.7z)?

 

[Neal Cullum]

You mixing two things in one sentence which a little confused me. Contents of HDD is not the same as HDD image. Contents are files and folders from filesystem(s), while SBS (sector by sector) image is whole structure, dumped as is (instead to sparse one, which is not possible to make in this case).

If You will create HDD image (must be encrypted for this task, so option no. 1, everything else results in recognizing HDD as empty) and restore it on larger HDD, You will not get extra space because custom partition table covering up old size and we don't have tools to edit both: partition table and UFS2 filesystem table to fit into new size. So it is kind a crippled cloning type and main purpose of "PS3 HDD Dumper" is not cloning but sharing image or just having full copy (i.e when user don't have/cannot dump EID Root Key, needed for decryption). Another issue is HDD assigning, so if You want restore disk image on different HDD, You must first making format of this HDD on target PS3 because doing that, she also assigning this HDD (noting model number, fw version and serial number of this HDD). Sony did everything they could to not allow user changing disks just like that without cleaning them first.

For You is "PS3 HDD Backuper" script which will copy important data from PS3 HDD and which You need to restore it by hand at this moment (from USB using file manager on PS3 side or FTP client - because I didn't write yet restoring script).

Take in mind that this script wasn't very well tested and just in case, user should check after copying if everything has been copied. Also take in mind that if You have dead CMOS battery in PS3, You shouldn't backup in the same dir after making changes because files aren't copy just like that, but syncing using rsync (it checks time stamps in files and copy only differences between target and source dir).

If You want do all of this manually, here is guide describing where is what exactly:
https://www.psx-place.com/threads/ultimate-userdata-backup-guide.29037/

Thanks, yes got a bit confused with the 2 options. I will give that Backupper a try and see if it works o.k for me.

 

[gmipf]

@Berion I wanted to add one more thing to the suggestions. While I was working on another script, where I needed a .config file, I noticed that the KO manager also modify files directly at:

/usr/src/linux-headers-$(uname -r)

I had to reinstall "linux-headers-$(uname -r)". I would therefore suggest that you only work on a local working directory.

 

[Berion]

@gmipf Yes, but only changing UFS line. Everything else stays intact.

But TBH, while ufs.ko module compiling fine (at least on Mints). It is still in read only. For some reason it ignores config during compilation I think. I have tried several locations and results are the same. I have no idea why, and how to fix that, so in reality, KO Manager can producing only bswap16-ecb for now. Also in WSL2 environment in current kernels, none of modules compiling. Again, I don't know why and this frustrated me so much that I left all of that as is. Messing with kernel is new to me and I don't understand plenty of things. So in summary: reading PS3 HDD is totally fine, but writing to it is still not possible.

 

[gmipf]

@Berion

I think the failure lies here:

zcat /proc/config.gz >.config
sudo sed -i 's/# CONFIG_UFS_FS_WRITE is not set/CONFIG_UFS_FS_WRITE=y/' .config
yes "" | make oldconfig

Instead you should do:

make olddefconfig
sed -i 's/# CONFIG_UFS_FS_WRITE is not set/CONFIG_UFS_FS_WRITE=y/' .config

The first copy line is unnecessary since oldconfig or olddefconfig already generates the correct .config. But oldconfig is interactive, I think with yes "" you actually disable CONFIG_UFS_FS_WRITE again, since that is the default answer. olddefconfig generates the default .config and doesn't ask. After that you need to modifiy with sed, not before.

 

[gmipf]

After more than 1 week of writing and testing I want to present my work. I was working on another driver module auto compiling/installation script but since the process is nearly the same, I ported it for ufs and bswap16. I have done a version for Fedora and Ubuntu.

This set of scripts will help you to compile, load and permanently install the ufs and bswap16 kernel modules. It also provides support for Secure Boot enabled systems. Here are quick explanations what the files do:


compile.sh
Will automatically download the kernel source tree (with all build tool dependencies) and compile the modules. Re-running this will clean-up the old source tree and recompile again. This also needs to be run every time after updating/changing the running kernel.


enroll.sh
Custom built kernel modules need to be signed otherwise loading those will fail on Secure Boot enabled systems. This script automatically generates private/public keys and imports the public key to the system firmwares MOK list using the shim boot-loader. Run compile.sh after this script to also digitally sign the modules while compiling. This script only needs to be run once, as long as you don't delete the 'key' directory.


load.sh
Loads locally built custom kernel modules without permanently installing them. Is needed every time you reboot the system and/or mount PS3 HDDs.


unload.sh
Unloads locally built custom kernel modules from memory. Works with installed modules too but usually unnecessary in that case.


install.sh
Permanently installs the built kernel modules on the running kernel. Manually loading modules isn't necessary any more after installation. The 'cryptsetup' and 'mount' utility will automatically load the necessary modules. This also needs to be run every time after updating/changing the running kernel. (After recompiling of course.)


remove.sh
Removes the installed custom modules and restores the stock ufs module again.



CHANGELOG

2024-10-05

- Fixed 'install.sh' for newer Fedora versions.


2023-11-06 (ps3ufs-ubuntu.tar.gz & ps3ufs-fedora.tar.gz)

- Added support for kernel 6.4 and later. (Updated bswap16.c)
- Cosmetic improvements.