PS3 [Tutorial] HDD mounting and decryption on Linux

[Berion]

@ichikameha If You cloned only first sectors from the same HDD on the same PS3, then You did something wrong because exactly this is the repair procedure.
https://www.psx-place.com/threads/tutorial-fixing-windows-disk-initalization.27599/

You cannot decrypt PS3 HDD with broken/non existent partitions table because none of the tools allow that. First You must fix it.

@Eruil EOL Still You mixed mappers names.

And You can clone any HDD from any platform until is not ATA Password locked (like i.e on Xbox). So there is no any problems cloning PS3 HDD. Besides that if this have any sense to do (because have not). I believe You mislead something here. Cloning doesn't means any interference in partitions and fs tables - if You expecting that cloned HDD will use new HDD size, then You have right that specia ltool for that is mandatory and not exist yet.

@pinky Why not? There is no such thing as "emmc hdd" because HDD is Hard Disk Drive and eMMC is a flash memory. Once user put HDD, eMMC is no longer in use and HDD take the role acting the same like on the rest of PS3s.

 

[sandungas]

The problem when you initialize the hdd in windows depends of the windows version... i dont really know what does each windows version, but just to get an idea... windows XP is going to overwrite an MBR (only 1 sector), and windows 10 is going to overwrite it with a GPT (64 sectors as far i remember)

The hdd can be restored by recovering the data of that sectors... but the only way to generate the data in that sectors is by formatting a hdd of the same exact number of total sectors in the same PS3

The easyest way to achieve that is... yeah... by formatting the same hdd
1) Make a complete backup "byte by byte" of the damaged hdd (you can use clonezilla, is a noob friendly tool for hdd backup/restore)
2) Format the hdd in the PS3
3) Extract the first sectors (either only 1... or 64)
4) Restore the backup from step 1 (includes the damaged sectors)
5) Fix the damaged sectors (either only 1... or 64)

There is another alternative though (not tested as far i know, but i guess it should work)
Instead of using the same hdd you can use a different hdd of same or bigger size and reduce his capacity using the HPA feature
https://en.wikipedia.org/wiki/Host_protected_area

The point is... if the original hdd (the one that was damaged by the initialization in windows) have 356.834.734 sectors... then the other hdd needs to be reduced to the same number of sectors exactly

 

[Berion]

@sandungas Wrong. Depend of HDD size, all Windows choosing MBR or GPT. For GPT are 34 sectors (which is exact size of full first GPT, some units like i.e Nvidia Shield using second one at the end). More than that is probably overkill for PS3 partition table. Anyway, above there is my tutorial which covering up both cases.

And nice idea of HPA "bypass".

 

[Berion]

@Eruil EOL Untested but should works. After creating mappers following by my tutorial but without mounting any partition fs, do:

dd if=/dev/mapper/ps3hdd2 of=/home/mint/ps3/superblock.img bs=512 count=256
`/home/mint/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65599 count=1 conv=notrunc
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65667 count=1 conv=notrunc

After that, remove mappers etc.

Remember to change filename from "find_ps3_ufs2_byte_locations" to "find_ps3_ufs2_byte_locations.sh" and to add executable attribute to that file.

 

[Eruil EOL]

@Eruil EOL Untested but should works. After creating mappers following by my tutorial but without mounting any partition fs, do:

dd if=/dev/mapper/ps3hdd2 of=/home/mint/ps3/superblock.img bs=512 count=256
`/home/mint/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65599 count=1 conv=notrunc
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65667 count=1 conv=notrunc

After that, remove mappers etc.

Remember to change filename from "find_ps3_ufs2_byte_locations" to "find_ps3_ufs2_byte_locations.sh" and to add executable attribute to that file.

so.. as i understood using Live mint cinnamon 19.2 (i hope this is the correct way to do it ) :

sudo su
insmod '/home/mint/ps3/bswap16.ko'
cryptsetup create -c bswap16-ecb -d /dev/zero ps3hdd-bs /dev/sdX
cryptsetup create -c aes-xts-plain64 -d /home/mint/ps3/ata_key.bin -s 256 ps3hdd /dev/mapper/ps3hdd-bs
kpartx -a /dev/mapper/ps3hdd
cryptsetup create -c aes-xts-plain64 -d
dd if=/dev/mapper/ps3hdd2 of=/home/mint/ps3/superblock.img bs=512 count=256
`/home/mint/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65599 count=1 conv=notrunc
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65667 count=1 conv=notrunc
kpartx -d /dev/mapper/ps3hdd && cryptsetup remove ps3hdd
cryptsetup remove ps3hdd-bs
rmmod bswap16
exit

i dont know if its better to bs=1024 or to bs=512 in 1tb hdd (you said in the other post that its better to use bs=1024

 

[Berion]

You have no idea on what You looking at, didn't You?

First, not "sdX" but Your PS3 HDD (sda, sdb, sdc, sdd etc. it must be true of course). Second, this line is not needed: "cryptsetup create -c aes-xts-plain64 -d" because what kind of mapper do You want to make and from what? It doesn't have sense. The rest is ok.

And You still going back to this 512 vs 1024... You are no longer using bswap16-nbd, which demand from You such changes in source code before compiling. You are now using bswap16-ecb. Why are You still asking about it?

In other thread You said that You want get back allocation from "space" to "time", and from 1% to 8%. This will not do that. Repeating the precess will not reverse the changes. You should kept Your old superblock backup (before "unlocking" 7% of reserved space) and write it back.

 

[Eruil EOL]

sdX (X its a variable ... so it can be anything ) . in my case sda, sdb, sdc, etc... its assigned depending in which sata port i connect ps3 hdd. its obvious that i dont have enough sata ports to watch a real dev sdX

i know exactly what im looking at. im going to back up all seagate 1tb data and im going to reformat it with ps3 console ( to remove the partition table im going to use hdd low level format tool) . then i ll do the unlock 8% , but not with bswap16 made by einys. ill made it with your bswap16.ko . i tried to fix 1tb hdd problem freezing at 68 % but if i canot i will do all again .

" cryptsetup create -c aes-xts-plain64 -d /home/mint/ps3/ata_key.bin -s 256 ps3hdd /dev/mapper/ps3hdd-bs" ... its not suppossed to use this ata key.bin to decrypt gameos partition ? (dev hdd 0 ) . you re saying its pointless.. then ok.

and i dont want anymore to undo what ive done.

im just trying to do the same procedure described in the other post but with your new tools . was just asking about commands (like your quick guide) to do it. i said it before ... im below the basic knowledge about linux systems

thats it . i dont want you to get a diabetic coma. just relax dude

 

[Berion]

sudo su
insmod '/home/mint/ps3/bswap16.ko'
cryptsetup create -c bswap16-ecb -d /dev/zero ps3hdd-bs /dev/sdX
cryptsetup create -c aes-xts-plain64 -d /home/mint/ps3/ata_key.bin -s 256 ps3hdd /dev/mapper/ps3hdd-bs
kpartx -a /dev/mapper/ps3hdd
cryptsetup create -c aes-xts-plain64 -d
dd if=/dev/mapper/ps3hdd2 of=/home/mint/ps3/superblock.img bs=512 count=256
`/home/mint/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65599 count=1 conv=notrunc
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65667 count=1 conv=notrunc
kpartx -d /dev/mapper/ps3hdd && cryptsetup remove ps3hdd
cryptsetup remove ps3hdd-bs
rmmod bswap16
exit

Second, this line is not needed: "cryptsetup create -c aes-xts-plain64 -d" because what kind of mapper do You want to make and from what? It doesn't have sense.

"cryptsetup create -c aes-xts-plain64 -d /home/mint/ps3/ata_key.bin -s 256 ps3hdd /dev/mapper/ps3hdd-bs" ... its not suppossed to use this ata key.bin to decrypt gameos partition ? (dev hdd 0 ) . you re saying its pointless.. then ok.

Try focus and analyse this. :P

 

[Eruil EOL]

im analyzing, in the way that i understand it ...

sudo su = its used to get elevated privileges

insmod '/home/mint/ps3/bswap16.ko' = it integrates bswap.ko kernel module to kernel

cryptsetup create -c bswap16-ecb -d /dev/zero ps3hdd-bs /dev/sda (or sdb, etc...)
cryptsetup create -c aes-xts-plain64 -d /home/mint/ps3/ata_key.bin -s 256 ps3hdd /dev/mapper/ps3hdd-bs = it asigns and creates a ps3 hdd decrypted partition (byteswaped? )

kpartx -a /dev/mapper/ps3hdd = map that decrypted partition

cryptsetup create -c aes-xts-plain64 -d = this its the not necesary command that you mentioned before , lets say it s incomplete, repeated and i need to delete it

dd if=/dev/mapper/ps3hdd2 of=/home/mint/ps3/superblock.img bs=512 count=256
`/home/mint/ps3/find_ps3_ufs2_byte_locations.sh superblock.img` = it dumps gameos table using find_ps3_ufs2_byte_locations (renamed as find_ps3_ufs2_byte_locations.sh + chmod +x ) located in einys repo. (i guess)

printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65599 count=1 conv=notrunc
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65667 count=1 conv=notrunc = it set optimization type

kpartx -d /dev/mapper/ps3hdd && cryptsetup remove ps3hdd
cryptsetup remove ps3hdd-bs = this is for disable and unmount

rmmod bswap16 = i dont know the purpose of that command.
exit = closes terminal running tasks

 

[ichikameha]

@ichikameha If You cloned only first sectors from the same HDD on the same PS3, then You did something wrong because exactly this is the repair procedure.
https://www.psx-place.com/threads/tutorial-fixing-windows-disk-initalization.27599/

You cannot decrypt PS3 HDD with broken/non existent partitions table because none of the tools allow that. First You must fix it.

Thank @Berion for your reply and your help. I do understand that the partition table needs to be corrected.
After reformatting, it is only the first sector that has any difference. I am formatting the original hdd with 4.86 OFW on a CECH2501B system.
the other fifteen sectors are identical in HxD. the tutorial mentions 34 sectors, but on my drive, after the first 16 sectors the disk is blank for thousands of sectors ahead.
These are the offsets of the first sector that have differences.
000000c0-000000e0
00000150-00000170

The two variables I could consider here are:
1. The new firmware creates a partition table with different data that the stock hdd had
2. I put CFW on my system to obtain the EID key, but that has not appeared to affect the first 16 sectors after formatting.

Please tell me what I could do different. I am grateful, Thanks again.

 

[Berion]

this its the not necesary command that you mentioned before , lets say it s incomplete, repeated and i need to delete it

Exactly.

it asigns and creates a ps3 hdd decrypted partition (byteswaped? )

cryptestup i.e creating or removing mappers. "ps3hdd-bs" (as I name it; the name can by any i.e "my_girlfriend"), as I said before, creates mapper which represent PS3 HDD but with converted byte order from Big Endian to Little Endian. That's is the purpose of every bswap16 module/app. "ps3hdd" (as I name it; the name can by any i.e "Kasia") is mapper which represent PS3 HDD not only with reversed byte order but also decrypted by specified key using specified algorithms etc.

this is for disable and unmount

No. You never mount any filesystem in above commands listing. "kpartx -d" removing partitions mapping. "cryptsetup remove" removing mappers.

i dont know the purpose of that command.

It removes loaded earlier bswap16 module from memory. Not necessary in live distribution. Just like the "exit" command. If You using live i.e Mint only for PS3 HDD messing.

 

[Eruil EOL]

cryptestup i.e creating or removing mappers. "ps3hdd-bs" (as I name it; the name can by any i.e "my_girlfriend"), as I said before, creates mapper which represent PS3 HDD but with converted byte order from Big Endian to Little Endian. That's is the purpose of every bswap16 module/app. "ps3hdd" (as I name it; the name can by any i.e "Kasia") is mapper which represent PS3 HDD not only with reversed byte order but also decrypted by specified key using specified algorithms etc.

basically its like a "partition name " and a "drive letter" (comparing with windows ) . just to make the recognition of the decrypted drive.

so i guess the commands quick guide in LIVE mint its reduced to this (sdc its an example) :

sudo su
insmod '/home/mint/ps3/bswap16.ko'
cryptsetup create -c bswap16-ecb -d /dev/zero ps3hdd-bs /dev/sdc
cryptsetup create -c aes-xts-plain64 -d /home/mint/ps3/ata_key.bin -s 256 ps3hdd /dev/mapper/ps3hdd-bs
kpartx -a /dev/mapper/ps3hdd
dd if=/dev/mapper/ps3hdd2 of=/home/mint/ps3/superblock.img bs=512 count=256
`/home/mint/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65599 count=1 conv=notrunc
printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65667 count=1 conv=notrunc
kpartx -d /dev/mapper/ps3hdd && cryptsetup remove ps3hdd
cryptsetup remove ps3hdd-bs

if i use my ubuntu 19.10 (5.3.0.46 generic kernel) ext 4 partition (i use a ryzentosh with win 10 / Ubuntu 19.10 and catalina 15.2) i just only run extra commands and i change to "/home/my user name" in all previous steps (running my new compiled bswap16.ko to run with latest ubuntu kernel). i suppose

rmmod bswap16
exit

 

[Berion]

basically its like a "partition name " and a "drive letter" (comparing with windows ) . just to make the recognition of the decrypted drive.

It is for recognition of the decrypted on the fly drive but hard to compare with Windows. Windows using letters for automatically mounted fs on found partitions, while Linux mount fs on demand and where ever You want where ever on fs tree. I.e in my tutorial, I mounting UFS2 from mapper (which represent i.e partition after some data operations), in "$home/ps3/dev_hdd0/" but this can be any, i.e somewhere in "/mnt" or /"media". Windows doesn't have such capability and even don't have any consistent devices or partition naming convention (depend on application he using different nomenclature like GPT GUID, internal GUID from registry, disk n partition n etc.). It is f*g chaos.

For the rest of question, answer is yes.

BTW: In theory this should works with WSL2, not only on real hardware with launched i.e Ubuntu. And on current testing branch, Linux fs appearing in Explorer, just like i.e "library".

 

[Berion]

If You cloned only first sectors from the same HDD on the same PS3, then You did something wrong because exactly this is the repair procedure.
https://www.psx-place.com/threads/tutorial-fixing-windows-disk-initalization.27599/

You cannot decrypt PS3 HDD with broken/non existent partitions table because none of the tools allow that. First You must fix it.

Thank @Berion for your reply and your help. I do understand that the partition table needs to be corrected.
After reformatting, it is only the first sector that has any difference. I am formatting the original hdd with 4.86 OFW on a CECH2501B system.
the other fifteen sectors are identical in HxD. the tutorial mentions 34 sectors, but on my drive, after the first 16 sectors the disk is blank for thousands of sectors ahead.
These are the offsets of the first sector that have differences.
000000c0-000000e0
00000150-00000170

The two variables I could consider here are:
1. The new firmware creates a partition table with different data that the stock hdd had
2. I put CFW on my system to obtain the EID key, but that has not appeared to affect the first 16 sectors after formatting.

Please tell me what I could do different. I am grateful, Thanks again.

34 sectors for GPT, and 1 sector for MBR. If Your drive have less then TB in capacity, then Windows write MBR to it (which he naming: disk initialization). Replace only this one as You can find in my tutorial (You don't even need understand it, "just follow the pictures CJ" ;p).

BTW: Not exactly whole HDD from 0 to last sector is encrypted. It is normal that some areas on start and some at the end have zeroes.

 

[Eruil EOL]

eruil@Ryzentosh:~$ sudo su
root@Ryzentosh:/home/eruil# insmod '/home/eruil/ps3/bswap16.ko'
root@Ryzentosh:/home/eruil# cryptsetup create -c bswap16-ecb -d /dev/zero ps3hdd-bs /dev/sdc
root@Ryzentosh:/home/eruil# cryptsetup create -c aes-xts-plain64 -d /home/eruil/ps3/ata_key.bin -s 256 ps3hdd /dev/mapper/ps3hdd-bs
root@Ryzentosh:/home/eruil# kpartx -a /dev/mapper/ps3hdd
root@Ryzentosh:/home/eruil# dd if=/dev/mapper/ps3hdd2 of=/home/eruil/ps3/superblock.img bs=512 count=256
256+0 registros leídos
256+0 registros escritos
131072 bytes (131 kB, 128 KiB) copied, 0.0447525 s, 2.9 MB/s
root@Ryzentosh:/home/eruil# `/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`
stat: no se puede efectuar `stat' sobre 'superblock.img': No existe el archivo o el directorio
/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh: línea 65: [: 65536: se esperaba un operador unario
Minimum: orden no encontrada
root@Ryzentosh:/home/eruil# printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65599 count=1 conv=notrunc
1+0 registros leídos
1+0 registros escritos
1 byte copied, 0.000640321 s, 1.6 kB/s
root@Ryzentosh:/home/eruil# printf '\x01' | dd of=/dev/mapper/ps3hdd2 bs=1 seek=65667 count=1 conv=notrunc
1+0 registros leídos
1+0 registros escritos
1 byte copied, 0.000616455 s, 1.6 kB/s
root@Ryzentosh:/home/eruil# kpartx -d /dev/mapper/ps3hdd && cryptsetup remove ps3hdd
root@Ryzentosh:/home/eruil# cryptsetup remove ps3hdd-bs
root@Ryzentosh:/home/eruil# rmmod bswap16
root@Ryzentosh:/home/eruil# exit
exit
eruil@Ryzentosh:~$

its says something like :
stat: cannot perform 'stat' on 'superblock.img': File or directory does not exist
/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh: line 65: [: 65536: unary operator expected
Minimum: order not found
whatever that means lol

i used a 320 gb test hdd and it worked.
before: 263 / 298 free space
now: 283 / 298 free space

 

[Berion]

Sounds very bad, but if You saying it works... However, could You type in terminal below command? Do You get the same results (>> stat: cannot perform 'stat' on 'superblock.img': File or directory does not exist)? If yes, then it looks like syntax of stat has changed since author of this script created it and we need figure this out (well, at least this is my assumption).

stat --printf="%s" /home/eruil/ps3/superblock.img

Line 65 is loop while, but I don't fully understand this script.

 

[Eruil EOL]

Sounds very bad, but if You saying it works... However, could You type in terminal below command? Do You get the same results (>> stat: cannot perform 'stat' on 'superblock.img': File or directory does not exist)? If yes, then it looks like syntax of stat has changed since author of this script created it and we need figure this out (well, at least this is my assumption).

stat --printf="%s" /home/eruil/ps3/superblock.img

Line 65 is loop while, but I don't fully understand this script.

i suppose that is a different syntax. i remember that einy used ubuntu 16.04.1 and some commands changed over ubuntu 18. SIZE=$(stat --printf="%s" $FILE inside find_ps3_ufs2_byte_locations ) maybe refers to watch allocation space in gameos superblock before and after 8% optimization


when i paste as 1 single command :

dd if=/dev/mapper/ps3hdd2 of=/home/eruil/ps3/superblock.img bs=512 count=256
`/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`

the very same terminal splits it in 2 commands :

1. dd if=/dev/mapper/ps3hdd2 of=/home/eruil/ps3/superblock.img bs=512 count=256 ... it creates a superblock.img file in /home/eruil/ps3 folder

2. `/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`. it shows like find ps3 ufs2 . sh doesnt exist : stat: cannot perform 'stat' on 'superblock.img': File or directory does not exist
/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh: line 65: [: 65536: unary operator expected
Minimum: order not found

 

[Eruil EOL]

i tested stat --printf="%s" /home/eruil/ps3/superblock.img but nothing happened , it never shown any error

root@Ryzentosh:/home/eruil# stat --printf="%s" /home/eruil/ps3/superblock.img
root@Ryzentosh:/home/eruil#

but if you run again dd if=/dev/mapper/ps3hdd2 of=/home/eruil/ps3/superblock.img bs=512 count=256
`/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh superblock.img` its splitted again

root@Ryzentosh:/home/eruil# dd if=/dev/mapper/ps3hdd2 of=/home/eruil/ps3/superblock.img bs=512 count=256
256+0 registros leídos
256+0 registros escritos
131072 bytes (131 kB, 128 KiB) copied, 0.00336279 s, 39.0 MB/s
root@Ryzentosh:/home/eruil# `/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`

when you press Enter:

root@Ryzentosh:/home/eruil# dd if=/dev/mapper/ps3hdd2 of=/home/eruil/ps3/superblock.img bs=512 count=256
256+0 registros leídos
256+0 registros escritos
131072 bytes (131 kB, 128 KiB) copied, 0.00336279 s, 39.0 MB/s
root@Ryzentosh:/home/eruil# `/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh superblock.img`
stat: no se puede efectuar `stat' sobre 'superblock.img': No existe el archivo o el directorio
/home/eruil/ps3/find_ps3_ufs2_byte_locations.sh: línea 65: [: 65536: se esperaba un operador unario
Minimum: orden no encontrada
root@Ryzentosh:/home/eruil#

if you run :

dd if=/dev/mapper/ps3hdd2 of=/home/eruil/ps3/superblock.img bs=512 count=256

then :

cd /home/eruil/ps3

and finally

./ps3/find_ps3_ufs2_byte_locations.sh superblock.img

heres the result =

root@Ryzentosh:/home/eruil# dd if=/dev/mapper/ps3hdd2 of=/home/eruil/ps3/superblock.img bs=512 count=256
256+0 registros leídos
256+0 registros escritos
131072 bytes (131 kB, 128 KiB) copied, 0.0122897 s, 10.7 MB/s
root@Ryzentosh:/home/eruil# cd /home/eruil/ps3
root@Ryzentosh:/home/eruil/ps3# ./find_ps3_ufs2_byte_locations.sh superblock.img
Minimum free space already configured to 1%
Optimization type already set to SPACE
root@Ryzentosh:/home/eruil/ps3#

 

[Berion]

I have updated tutorial for bswap16-ecb.ko. Changes:

• fixed one important error when user must choose proper algo
• coloured some important differences and info on red
  
• changed links colour to green to not be mislead with paths on blue (before, links were also blue)
  
• added few more questions and answers
• some tiny Polish syntax errors
  
• updated 'thanks section' (@mlody95pl)

I have removed tutorial from 2019-01-22-rev3 which based on bswap16-nbd.elf as it is totally obsolete now and could be misleading for new comers. It is still available in quick guide, also I will not remove the apps and sources for it, so users can still use them for some reason. Just in case.

 

[Berion]

@Eruil EOL You don't need to dump every time superblock. It is only used to determine if changes are needed or not on real device (based on this dump). Now You have overwrite clean superblock by modified one from mapper. :P

Could You then type and check what is written in this text file after type it (but probably it will be empty); of course on "clean superblock", not the one dumped from "unlocked free space HDD":

stat --printf="%s" /home/eruil/ps3/superblock.img >> /home/eruil/ps3/test_output.txt