PS3 [TUTORIAL] Unlock up to 8% extra total space on the PS3 internal hard drive

Update: picard (a.k.a. 3141card) has released a homebrew app (available here) that changes the reserved space from 8% to 1% automatically. Running the app again will revert the changes.

https://mega.nz/file/zdk1lKqZ#aj4WFdOeosH_ncIB5flNtUxPs341PSYaXdLF2DdA45A

PS3 Unlock HDD Space by picard (aka 3141card)
All credits to einsteinx2 on psx-place.com for his great tutorial:
https://www.psx-place.com/threads/t...l-space-on-the-ps3-internal-hard-drive.20773/

HOWTO:
  • Install the package on your custom firmware PS3 and run it. It finds the active superblock of the gameOS partition and do the necessary patches. Usually 8 percent of the memory is reserved, the app sets this value to 1 percent.
  • The optimization value is set to 1, for minimize disk fragmentation.
  • The changes are permanent. The changed active superblock is written to the HDD with the system fs umount of the gameOS partition.
  • Running the app again restores the original values, if desired.

You use this app at your own risk! I am not responsible for any damage caused to your PS3 system!​

If my PS3 app was useful to you, I would be very happy about a donation :)
https://www.paypal.com/paypalme/3141card

picard (aka 3141card)

-- (Original Post Continued Below) --

Need some extra space on your internal PS3 HDD, if so then perhaps @einsteinx2 may have a genius method for unlocking upto 8% of space on your Internal HDD, while the space is more then likely allocated for performance reasons by Sony (use this guide at your own risk), some of this saved space could be considered overkill, as the creator of this method makes a valid point that some of this saved space on larger HDDs is more then some of the early model PS3 complete HDD size, so its likely some of that space allocation is a bit overkill on the part of Sony. einsteinx21 has provided a great tutorial found below explaining in detail about this recent discovery

-STLcardsWS


depositphotos_10075144-stock-photo-protected-hdd-chain-and-lock.jpg

Unlock up to 8% extra total space on the PS3 internal hard drive


  • The What

    By default, even on custom firmware, the PS3 reserves 8% of your total internal hard drive space. From my searching online, I don't believe anyone has ever successfully unlocked that wasted space, so I decided to give it a shot. Turns out not only is it possible, but it's relatively easy thanks to some existing tools created by the community. This guide will explain how to reclaim that wasted space by manually modifying the metadata of the UFS2 formatted GameOS partition using Linux, as well as the potential cons (primarily performance, though I haven't experienced any performance issues yet myself).

    I believe the pros outweigh the cons though, and have been using this now without issue on my personal PS3 with a 1.5TB drive for a few days now, installing tons of games without a hitch.

    The Why


    While external drives used with the PS3 are all FAT32 formatted (or NTFS if you have CFW), the GameOS partition on the internal hard drive is formatted using the UFS2 filesystem with a layer of encryption on top.

    Like other *nix file systems such as Ext2/3/4, UFS2 can reserve part of the drive's space to only be used by the system or the root user. This is to reduce fragmentation and also prevent the drive from completely filling up, possibly freezing the computer.

    By default, UFS2 reserves 8% of any drive's free space, in this case meaning it can't be used by the PS3 for installing games. This is why when you first install that shiny new 1.5TB hard drive you will see that not only is it only actually 1.36 TB because hard drive manufacturers really love counting in base 10, but you also lose another 111.4 GB.

    Now generally having this reserved space is not a bad thing, however the problem is that as drives grow larger, the amount of wasted space becomes larger than some of the smaller PS3's entire hard drives! Clearly that much isn't needed to prevent fragmentation.

    The tunefs documentation mentions that "the file system's ability to avoid fragmentation will be reduced when the total free space, including the reserve, drops below 15%. As free space approaches zero, throughput can degrade by up to a factor of three over the performance obtained at a 10% threshold." Note that these numbers are already higher than the 8% default that all UFS tools as well as Sony use. Also note that it says as free space drops to 0%, performance may be up to 3 times slower than normal. However, it's unclear whether that only affects newly written--presumably more fragmented files--or all files. It seems like this is a worthy tradeoff, especially since if you do notice any performance issues you can simply delete some games or other data to free up the space. By changing the minimum free space, performance is not changed at all unless you choose to fill the drive completely...but at least you can now make that choice.

    UFS2 supports two write optimization modes: time and space. Time optimization is the default--and is used by the PS3--and allows for faster writes at the expense of potentially higher fragmentation (though generally only as the drive reaches capacity). Since we're allowing ourselves to fill almost the entire drive, these instructions also change the optimization mode to space. That means that the filesystem will spend more time during writes to ensure the files are less fragmented, insinuating that it will shuffle blocks around to make contiguous space.

    The tunefs documentation says that "the file system can either try to minimize the time spent allocating blocks, or it can attempt to minimize the space fragmentation on the disk. Optimization for space has much higher overhead for file writes. The kernel normally changes the preference automatically as the percent fragmentation changes on the filesystem." However, since the PS3 uses a custom operating system and this documentation is from FreeBSD, I think it's best to manually set the space option anyway. I have not noticed any performance degradation so far since making this change, though I still have plenty of free space on my drive at the moment.

    Photos
    Here are the before and after photos. They were taken immediately before removing the drives and immediately after reinserting them.
    BEFORE
    160GB_drive_before.jpg
    AFTER
    160GB_drive_after.jpg
    BEFORE
    1.5TB_drive_before.jpg
    AFTER
    1.5TB_drive_after.jpg


  • The How

    While it's possible to mount a PS3 hard drive in Linux and view its decrypted partitions, unfortunately the tunefs.ufs tool doesn't appear to work. It always complains that it can't find the superblock. However, the file command does work fine to show the UFS2 filesystem info when tested on a dump of the start of the partition.

    So instead of using the tunefs utility to change the minimum free space and optimization type, I wrote a script to manually scan and test single byte changes to a dump of the partition's superblock using the file command until I found the correct locations to change.

    I tested this on 2 hard drives of vastly different sizes: 160 GB and 1.5 TB. On both drives, I found that the superblock was located in the same location--the standard 128 block aka 65,536 byte offset. I also found that the locations to set the minimum free space percentage and optimization type were in the same place on both drives--byte 65,599 and 65,667 respectively. However, I highly recommend running the find_ps3_ufs2_byte_locations script anyway just to confirm before making any changes to your drive.

    Once you know the correct offsets, changing the values is simple. To adjust the minimum free space, simply write the percent in hex to byte 65,599--e.g. for 1% free space write 0x01 or for the default 8% write 0x08. To change the optimization type, write a hex 0x01 to byte 65,667--the default is 0x00 for time optimization.

    I think this should be possible to do as a PS3 homebrew app so that it can just be done directly on the device without even removing the drive. Unfortunately I have zero experience with writing PS3 homebrew, but maybe someone else with more experience can use this information to do it. I'd love to see this as an option in the REBUG custom firmware settings.

    Instructions

    1. Dump your eid root key using IrisMan/MultiMan/etc
    2. Setup a computer or virtual machine with Ubuntu 16.04. The rest of these steps are done on that machine. I tested using Parallels on a MacBook Pro, but it should work on just about anything as well as other distros.
    3. Clone my repository: git clone https://github.com/einsteinx2/PS3-Reclaim-HDD-Space.git
    4. Change to the new directory as we'll do all of the work there: cd PS3-Reclaim-HDD-Space
    5. Rename your eid root key file to eid_root_key.bin and place it in the PS3-Reclaim-HDD-Space directory
    6. Generate your hdd keys: ./ps3hdd_keygen.sh
    7. Become root since most of this requires it: sudo -s
    8. Find the device name: fdisk -l (In my case, using an external USB enclosure, it was /dev/sdb)
    9. Make virtual byte swapped encrypted device
      1. If you have a drive 1TB or less (not confirmed the exact limit): ./makedev bswap16.512 /dev/sdb
      2. If you have a drive larger than 1TB (or maybe it's 1TB and larger, I don't have a 1TB drive to test): ./makedev bswap16.1024 /dev/sdb
    10. Create decrypted device: cryptsetup create -c aes-xts-plain64 -d ./hdd_key.bin -s 256 ps3hdd_crypt /dev/nbd0
    11. Map decrypted partitions: ./kpartx -a /dev/mapper/ps3hdd_crypt
    12. View decrypted partitions (ps3hdd_crypt2 is the UFS2 GameOS partition): ls -la /dev/mapper/
    13. View current free space: [ -d /mnt/PS3GameOS ] || mkdir /mnt/PS3GameOS && mount -t ufs -o ufstype=ufs2,ro /dev/mapper/ps3hdd_crypt2 /mnt/PS3GameOS && df -h | grep "Avail\|ps3hdd_crypt2" && umount /mnt/PS3GameOS
    14. Dump the superblock of the GameOS partition: dd if=/dev/mapper/ps3hdd_crypt2 bs=512 count=256 of=GameOS_superblock.img
    15. Confirm the seek values for the next 2 commands: ./find_ps3_ufs2_byte_locations GameOS_superblock.img
    16. Set minimum free space to 1%: printf '\x01' | dd of=/dev/mapper/ps3hdd_crypt2 bs=1 seek=65599 count=1 conv=notrunc
    17. Set optimization type to "space": printf '\x01' | dd of=/dev/mapper/ps3hdd_crypt2 bs=1 seek=65667 count=1 conv=notrunc
    18. View the now larger free space: mount -t ufs -o ufstype=ufs2,ro /dev/mapper/ps3hdd_crypt2 /mnt/PS3GameOS && df -h | grep "Avail\|ps3hdd_crypt2 && umount /mnt/PS3GameOS
    19. Disconnect device: kpartx -d /dev/mapper/ps3hdd_crypt && cryptsetup remove ps3hdd_crypt && ./stop-nbd0
    20. Pop the drive back in your PS3 and enjoy the extra space! Note that I left 1% reserved space rather than going all the way to 0% to ensure that the drive never completely fills up, as I'm unsure what problems that would cause for the PS3's operating system.

  • Source code

    For ease of use, this repo contains precompiled binaries for Ubuntu 16.04 64bit. If you need or prefer to compile yourself, here are the tools used:

    bswap16: https://github.com/sguerrini97/nbdcpp (note that for >1TB drives you must change <unsigned BS=512> to <unsigned BS=1024>)

    kpartx: https://git.opensvc.com/multipath-tools/.git/

    ps3hdd_keygen.sh: http://www.psx-place.com/threads/hdd-keys-generating-scripts.10610/page-2#post-125197


  • Credits

    I would never have figured this out if it weren't for others' hard work.

    Huge thanks to Berion at PSX-Place for the hdd key generation script as well as pointing me to the information on mounting a PS3 HDD in Linux. His post here contains the script and the link: http://www.psx-place.com/threads/hdd-keys-generating-scripts.10610/page-2#post-125197

    Huge thanks to sguerrini97 at Playstation Hax for implemnenting PS3 hard drive mounting support for modern Linux kernels. Here's the post about it: https://playstationhax.xyz/forums/topic/4671-mounting-ps3-hdd-on-newer-linux-kernels and the GitHub repo: https://github.com/sguerrini97/nbdcpp.

    Thanks to dsroche for writing the original nbdcpp implementation that sguerrini97 forked, and thanks to Glevand for the original work on mounting the PS3 hard drive in OtherOS and for the great information here on the PS3 dev wiki: http://www.psdevwiki.com/ps3/Mounting_HDD_on_PC. Also thanks to anyone else that worked on PS3 hard drive mounting or anything else I'm not aware of.

 
Last edited by a moderator:
@Agoni212 It is NOT dangerous to the console itself and it CANNOT brick it. In worst scenario You'll lost Your user data...

@x23are If You need step by step guide with many things explained in details, You need to wait a little longer (one or two weeks). My tutorial is not for fs table manipulation but for write access to the partitions on Linux, outside PS3 environment, but it will definitely cover up the step where You stuck. I have almost completed, ten pages. :)

View attachment 13499

cool
when this tutorial is ready?
 
@fresh You will get this error in cases of:
  • when bswap somehow is incompatible with nbd-server/linux distro
  • when You have occupied loop0 slot (it needs to be empty for nbd0 mapping... somehow ;))
Solution:
  • compile bswap by Yourself from source (You need as I remember g++ to be installed)
  • use Linux Mint 18.1 .2 or .3 and do not update it (just install newest available nbd-server)
  • install it where ever You want, even on deadly slow pendrive, just don't use Live session as this one using loop0 slot for SquashFS mount
Author said it works on Ubuntu 16.04 with precompiled binaries. I didn't test it so I mentioned only about Mint which is enough stupid-proof for me and have normal GUI (I hate Unity and Gnome3). :D

Which exactly version aru u using? Linux Mint 18.1 "Serena" Cinnamon (64-bit) will be good?
On Ubuntu 16.4 and some Mint ive got:

Code:
" Requested offset is beyond real size of device /dev/nbd0 "[code]
 
@armorsnake I made an tutorial:
https://forum.cdrinfo.pl/f106/poradnik-montowanie-pamieci-wewnetrznej-ps3-na-linuksie-97569/
You can also download precompiled binaries and scripts from there.

Linux Mint 18.3 with nbd-server v3.13-262-g69a15c0-dirty (on i.e v3.16.2 doesn't work, also doesn't work on 19.0 v2). I don't know why. Tried 18.1 and 18.2 and still good to go.

PS: I didn't translate it yet to English because I'm still trying solve the FAT mysteries and looking a better way to UFS write support than ufs.ko.
 
very interesting and usefull, change from time to space only affects writes correct, your mod could fix the issues with running a true SSD harddrive if set to space and allocated more space for frag. ive had issues running SSDs using every brand under the sun, most SSDs trim/frag themself but the ps3 doesnt no its a SSDs and tries to frag it resulting in file system corrupt every couple wks

ill run som tests if i evr get time most of my SSDs r 3/4s full or r in my main devices
 
Last edited:
Finally anyone have a video tutorial?? I'm interesting in unlock 8% but I'm not sure how can do this (easy method? =))
No need for a video tutorial, I'm currently working on a homebrew app that will run on the PS3 itself to do this. This method was really just a proof of concept to see if could even be done at all.

I was delayed a while due to problems getting PS3 homebrew code compiling (and work getting in the way), but I've resolved the issues I had and am working on a proof of concept app now. No ETA, but I'll post here when I have it finished.
 
change from time to space only affects writes correct
Yep exactly. Reads work the same either way.

And that change can be made independently of the reserved space change. I'll probably include a toggle for that separately in the homebrew app I'm working on.
 
THANKS einsteinx2!!! for your work!! I wait your app!! Thank you show much!!
Thanks! However, I do need to temper expectations a bit before anyone gets too excited...

I did a lot of investigative work last night trying to get the proof of concept working that would just show the current reserved space setting (assuming that once I can get that working, writing to it should be relatively simple).

However, I immediately ran into some serious roadblocks. To get a bit technical, it appears I'm not able to open the /dev_hdd0 mount point using the standard read() syscall or fread() c function (though I can open other files, so my code is written correctly), and I've tried various ways to access the underlying drive path with no success so far.

I'm working a bit blind as I have limited experience with the PS3 operating system and homebrew development, but there is a possibility that this either may not be possible to accomplish (due to restrictions on homebrew app permissions or some other reason) or that it may be possible but just much more complicated than I thought and outside of my ability to write.

In any case, I'm still early in the investigative stage, so I definitely haven't tried everything yet and I still have some hope it'll be possible. If I end up getting completely stuck, I'll upload my work in progress code and try and solicit help from developers more experienced with the PS3's operating system.

I'll hold off reporting back here again until I either make some progress, or reach an impassible road block.
 
However, I immediately ran into some serious roadblocks. To get a bit technical, it appears I'm not able to open the /dev_hdd0 mount point using the standard read() syscall or fread() c function (though I can open other files, so my code is written correctly), and I've tried various ways to access the underlying drive path with no success so far.

<trying summon PS3 elder spirits> ^^"
@aldostools @Zar @zecoxao @deank @bguerville @esc0rtd3w @sandungas
 
Last edited:
Specifically if anyone knows the hard drive device paths, that would potentially be very helpful (for example I tried the standard FreeBSD format of /dev/adas1a and other variations, as well as /dev/hdd0 without success, though I need to look into how to list the directories on the base filesystem if it's possible so I'm not poking around in the dark).

Also if anyone knows whether it's possible to directly access the actual partitions at all without manually decrypting them, that would be very helpful as well. /dev_hdd0 seems to be just the mount point for the decrypted partition, so I'm not sure if it can be used to manipulate raw bytes on the partition or if it only allows for accessing files.

I need to modify the partition header, which is simple enough to do on Linux once the drive is mounted, but it's done using the hard drive device paths directly not from the mount points, so I think it's unlikely /dev_hdd0 will give me the access I need. However, if I do find the device paths but still need to manually decrypt the partition again before accessing the partition table, that would probably be outside of my ability to code.

I'm also looking into the PS3 specific lvl2fs filesystem syscalls, to see if that's what I need to use, as I'm very unfamiliar with all this, so I'm learning as a I go.

I'm going to keep tinkering with this in my free time, and will get my work in progress up on Github soon so others can take a look.
 
I'm ont an elder spirit but it feels good you think I am. lol.

I'm not sure...
to access to /dev_hdd0 you must set the correct capability and control flags, for example : https://github.com/Zarh/ManaGunZ/blob/master/Makefile#L53

I'm not sure too, i didn't test it.
To access to raw data.. You can use sys_storage syscalls.

Code:
#define FLASH_NAND            0x100000000000001ULL
#define FLAS_NAND_EXT        0x100000500000001ULL
#define FLASH_NOR            0x100000000000004ULL
#define FLASH_NOR_EXT        0x100000200000004ULL

#define ATA_HDD                    0x101000000000007ULL
#define BDVD_DRIVE                0x101000000000006ULL
#define PATA0_HDD_DRIVE            0x101000000000008ULL
#define PATA0_BDVD_DRIVE        BDVD_DRIVE
#define PATA1_HDD_DRIVE            ATA_HDD
#define PATA1_BDVD_DRIVE        0x101000000000009ULL
#define CARDCTRL                0x101000000000010ULL  // ?
#define BUILTIN_FLASH            0x100000000000001ULL
#define MEMORY_STICK            0x103000000000010ULL
#define SD_CARD                    0x103000100000010ULL
#define COMPACT_FLASH            0x103000200000010ULL

#define USB_MASS_STORAGE_1(n)    (0x10300000000000AULL+(n)) // For 0-5
#define USB_MASS_STORAGE_2(n)    (0x10300000000001FULL+((n)-6)) // For 6-127
#define USB_MASS_STORAGE(n)        (((n) < 6) ? USB_MASS_STORAGE_1(n) : USB_MASS_STORAGE_2(n))

#define    HDD_PARTITION(n)    (ATA_HDD | ((uint64_t)n<<32))
#define FLASH_PARTITION(n)    (BUILTIN_FLASH | ((uint64_t)n<<32))

typedef struct
{
    char   Label[8];
    u32    unknown02;
    u32    zero01;
    u32    unknown03;
    u32    unknown04;
    u8     zero02[16];
    u64    sector_count;
    u32    sector_size;
    u32    unknown05;
    u8     writable;
    u8     unknown06[3];
    u32    unknown07;
} __attribute__((packed)) device_info_t;


s32 sys_storage_open(u64 device_ID, u32* fd)
{
    lv2syscall4( 600, device_ID, 0, (u64)fd, 0 );
    return_to_user_prog(s32);
}

s32 sys_storage_close( u32 fd)
{
    lv2syscall1( 601, fd );
    return_to_user_prog(s32);
}

s32 sys_storage_read( u32 fd, u64 start_sector, u64 nb_sector, const void* buffer, u32 *number_byte_read, u64 flags )
{
    lv2syscall7( 602, fd, 0, start_sector, nb_sector, (u64)buffer, (u64)number_byte_read, flags );
    return_to_user_prog(s32);
}

s32 sys_storage_write( u32 fd, u64 start_sector, u64 nb_sector, const void* buffer, u32 *number_byte_wrote, u64 flags )
{
    lv2syscall7( 603, fd, 0, start_sector, nb_sector, (u64)buffer, (u64)number_byte_wrote, flags );
    return_to_user_prog(s32);
}

s32 sys_storage_get_device_info(u64 device_id, device_info_t *device_info)
{
    lv2syscall2( 609, device_id, (u64)device_info );
    return_to_user_prog(s32);
}

void DumpRawData()
{
    u32 source;
    u32 read;
    int ret=-1;
    u64 device_id = HDD_PARTITION(1);
 
     
    device_info_t device_info;
    ret = sys_storage_get_device_info(device_id, &device_info);
    if( ret != 0 ) {
        free(info);
        return;
    }
 
    ret = sys_storage_open(device_id, &source );
    if( ret != 0 ) {
        free(info);
        return;
    }
 
    u8 *sector = (u8 *) malloc(device_info.sector_size);
    if(sector==NULL) {
        free(info);
        return;
    }
 
    char temp[64];
    if(strcmp(info[i].mount_point, "/")==0) strcpy(temp, "/dev_hdd0/tmp/rawdata_root.bin");
    else sprintf(temp, "/dev_hdd0/tmp/rawdata_%s.bin", &info[i].mount_point[1]);
 
    FILE *f;
    f = fopen("/dev_usb000/raw.bin, "wb");
    if(f==NULL) {
        free(info);
        free(sector);
        return;
    }
 
    u64 count = device_info.sector_count;
 
    for(i=0x0; i < count; i+=0x1) {
        sys_storage_read( source, i, 1, sector, &read, 0);
        fwrite(sector, sizeof(sector), 1, f);
    }
 
    fclose(f);
    sys_storage_close(source);    
    free(sector);
}
 
@Zar Thanks, that helps a ton! I had a feeling I may have been using the wrong syscall. I'll try this out and see how it goes. I'm only trying to read/write single bytes to 2 locations in the partition inside the superblock, and I can confirm from the read whether I'm getting the correct location or not.
 
Hmmm, there was a tool/app made by gitbrew team named "dump ptab pkg" (a bit confusing name, because is not explicit enought)
That tool was making a copy of the first 2 sectors of the hdd :)
In it you could see the partition table, with the offsets and the ACL's (access control lists) for every partition/region, etc...

I dont know how it was made, but the tool was running in gameOS, and was made for 3.40 custom firmware (way before the custom firmwares implemented other complex patches)
So... i guess it should work to read any sector of the hdd
For writing... dunno, but hopefully is the same... most probably is going to be needed to write a full sector, or a page/block (after backuping it so most of the data writen is the same before writing, you just change the 2 bytes)
 
Thanks @sandungas that's great to hear and really gives me confidence that this can work! And as you said, reading/writing sectors/blocks at a time instead of bytes shouldn't be an issue.

I'm going to try and squeeze in a few hours
this week to play with the proof of concept. I have some friends that just came into town, so it may be next week, but I'll report back once I either have a prototype or hit a wall again.

Thanks everyone for all the help so far, I couldn't do this without the community!
 

Featured content

Trending content

Back
Top