PS3 Jonnysp's ird library is down.

[AlexRhine]

Please note that since I can only use HEN on my PS3 console I cannot create IRDs on there, nor can I dump encrypted ISO images. However, my decrypted dump that I made using my PS3 was a 100% match for the decrypted dump that I created using PS3Dec from my original PC-dumped ISO. The reason why I mention this is to confirm that my dumps and the Redump one *are* fine and the Blu-ray video content works perfectly as well.

Wait, why aren't you able to dump encrypted discs in HEN? If you were able to build a "proper" decrypted ISO with MGZ, then you should be able to dump encrypted discs, given that dumping decrypted ISOs relies on being able to read the disc as encrypted (FYI, MGZ basically builds a decrypted ISO by getting the header from the original blu-ray and then building the ISO like you would build it with an ird file, at least in HEN w/ test version 1.41). Or do you mean that you dumped the disc as decrypted with MM/Irisman and then compared the md5sums of all files in this iso to the ones in the iso you dumped in your PC?

TL;DR Hybrid discs are broken in current IRD files

It isn't necessarily hybrid discs, but rather any disc which places multi-extent files in non-contiguous offsets. However, the problem lies more on programs not taking this into account rather than the ird format itself. The offset/md5sum table can be interpreted as using the offset of the first extent of the file, without necessarily guaranteeing that the entire file is written sequentially. AFAIK, the only program available to build ISOs from ird files is PS3-ISO-Rebuilder, so that's probably where the bug lies.

Edit: Though, there is probably also a bug with whatever program was used to build the ird file you're analyzing. It probably used a library (such as libcdio) which assume all multi-extent split files are sequentially written, and then calculated the md5sum of a region on the disc as if the entire file was sequential (starting at the offset of the first extent).

 

[sandungas]

Since I was having trouble getting hybrid disc titles to rebuild I got hold of and dumped an actual disc myself using my PC: Phineas and Ferb Across the 2nd Dimension [BLES01376].

Just to be sure we are using the same slang, the hybrid discs are the ones that contains 1 folder named PS3_VPRM, and 1 or more folders named PS3_GAME (and incase of the disc containing multiple bootable games the folders would be named PS3_GM01, PS3_GM02, PS3_GM02, etc...), right ?
https://www.psdevwiki.com/ps3/PARAM.SFO#Blu-Ray_disc_game_structure_details

You was having the same problem in several hybrid discs or is just in phineas and ferb ?
Im asking becaue i dont see any direct relationship in the fact that is an hybrid disc with a theoretical requirement to split any file
I mean... if the sony tools was able to create an hybrid disc containing 100% contiguos files then the fact that phineas and ferb have some splitted files is just a coincidence

Btw, that "flaw" of the IRD format is indirectly related with makeps3iso, the short story is... since some versions ago of cobra/mamba (more than 1 year ago) it was implemented a feature that allows to run the games "burned" in optical media (in its decrypted format)... lets say... you can burn a PS3 ISO in a writable bluray disc and play with it by inserting it in the drive like if it was original
Not much people uses it but it works, to achieve the highest compatibility with the PS3 game library is needed to create the ISO with makeps3iso, but...
It doesnt works if the ISO is bigger than 25GB (dual layer discs, the games that are this big can be counted with the fingers of a hand, but are a few)
The reason is because there is always a file "splitted" in between the 2 layers, and the isos generated by makeps3iso doesnt stores any info about that splitting
And the IRD of this kind of "BD dual layer" games is not able to record the info about that splitting either

 

[AlexRhine]

The reason is because there is always a file "splitted" in between the 2 layers, and the isos generated by makeps3iso doesnt stores any info about that splitting
And the IRD of this kind of "BD dual layer" games is not able to record the info about that splitting either

Well, as it is right now, makeps3iso already "splits" files by default. Any file of size bigger than 4 GiB needs to be written as multi-extent. In the case of this game, however, the "extensions" of these files are interlaced with others, which is odd. I assume that they are recorded this way to allow the different videos on disc to be read from a similar offsets. However, I always assumed that for all PS3 discs the splits of multi-extent files would be written sequentially on the disc, in order to allow for UDF to just use the entire size like it normally would. But, given these offsets, I assume UDF also allows split files for some reason (probably for read-speed purposes).

As for burned ISOs of dual-layer games not working... I don't think it's a problem with makeps3iso (or split files, for that matter). It sounds more like a problem with how cobra/mamba is handling reading the mounted disc (which it wasn't really designed to use, I suppose). Or maybe it's a problem with disc-burning software, where Sony expects some sort of string in the OSA/ISA of dual-layer discs containing games.

As for the info about dual-layer discs not being stored in irds... I'm not sure. A disc being dual-layer is technically abstracted when reading the disc, and the second layer becomes an extension of the first, continuing where the previous LBN/LBA left off. So I would assume all file information would still be stored in the original header of the disc, and not in a separate filesystem of sorts in the second layer. So, the filesystem that is read from the disc isn't really "aware" of whether files are stored in a second layer or not.

But, the real problem seems to arise from triple-layer discs. Yeah, those do exist in the form of hybrid-discs for PS3 (given how much size the movies included end up taking up). The odd thing is that I've never seen an ird of a triple-layer disc. Maybe the 3K3Y team never thought to support such discs with the release of their ODE/IRD-Program. I've been curious as to whether MGZ is capable of creating an ird for such a disc. Though, the only "challenge" involved in this procedure I think would be properly handling LBN and calculating the md5s of gigantic movie files. The only American hybrid disc of this size that I know of is Tekken-Hybrid, though.

 

[datman]

Hiya! Since there were a few questions asked I thought I'd just reply in general times in a bullet point style as it'd be easier... probably! Just trying to keep this tidy...ish!

• I can't install a full CFW on my PS3 so use HEN. I haven't got ManaGunZ installed, I use multiMAN to dump an ISO and I was fairly certain that because I am on HEN I cannot dump anything other than a decrypted ISO. I've tried various things to dump an encrypted image but HEN doesn't give me enough privelege rights or whatever the correct terminology is. If it is possible then I stand corrected but I did look into this a fair bit when I first started dumping stuff on the console.

• I have been dumping PS3 discs using DICUI/MPF for quite a while on my PC and haven't come across anything that I've dumped that doesn't match with what's already in Redump.

• There have been a few discussions on places like Vimm's Lair and "other sites" where people have been having issues rebuilding hybrid discs (i.e. ones which contain a game + regular Blu-ray movie content). That's what I was referring to when I say hybrid discs - I believe that's their proper name as well(?). It's possible that other discs in the IRD library also use non-contiguous split files but I've not come across any so far.

• Any IRDs that I built myself have been via Redump2IRD. I am pretty confident that I am using it correctly. I have come across one or two IRDs which were incorrect and didn't rebuild discs correctly but they are fairly uncommon (please see my Last of Us IRD I posted on the previous page, a BD50, by the way, so any flaws in the IRD format regarding layers doesn't affect rebuilding). However, all the hybrid discs that I have gotten "official" IRDs for from existing archives/libraries online fail to rebuild discs properly using either PS3-ISO-Rebuilder or 3k3y's IsoTools (and 3k3y is who came up with the IRD format in the first place... right??). Unpacking and analysing IRDs (as I have done) shows that IRDs don't store anything other than the first sector of a file so in the event that a file isn't written continguously to a disc then the IRD information is incomplete and kinda pointless.

TL;DR I have done quite a lot of testing with hybrid discs, IRD creation and rebuilding and I'm just posting my findings regarding a slight flaw in the format that will affect any discs which have files that are not written contiguously. In my personal experience this appears to be hybrid discs only but there may be others as well.

 

[AlexRhine]

Any IRDs that I built myself have been via Redump2IRD. I am pretty confident that I am using it correctly. I have come across one or two IRDs which were incorrect and didn't rebuild discs correctly but they are fairly uncommon (please see my Last of Us IRD I posted on the previous page, a BD50, by the way, so any flaws in the IRD format regarding layers doesn't affect rebuilding). However, all the hybrid discs that I have gotten "official" IRDs for from existing archives/libraries online fail to rebuild discs properly using either PS3-ISO-Rebuilder or 3k3y's IsoTools (and 3k3y is who came up with the IRD format in the first place... right??). Unpacking and analysing IRDs (as I have done) shows that IRDs don't store anything other than the first sector of a file so in the event that a file isn't written continguously to a disc then the IRD information is incomplete and kinda pointless.

You're confusing the output of ird_tools with the information that is stored inside an ird file. This is what an ird really stores. So some basic disc information, sections of the disc (header, footer, pic, etc), and region/file hashes. The "file 1 key" is the offset/sector of the file's whose hash is shown. Notice that a file's path/name and encryption status aren't stored explicitly in the file. So where does ird_tools and PS3-ISO-Rebuilder get that information? It's all in the header. These programs typically read the ISO-9660 filesystem to get the name of the files, and the encryption table to get a file's encryption status in the original disc. In the header you will also find, of course, how multi-extent files are stored on the disc. So the information about their offsets is all there. It's just that Zar's ird_tools doesn't print it because it assumes that all "split" files (at least in the ISO-9660 fs) are sequential. I guess that 3K3Y team also assumed all files would be stored sequentially on disc, do they didn't take that into account when trying to rebuild from an ird file (and maybe even when creating them). A common iso handling library like libcdio also doesn't take into account files split non-sequentially, and throws a warning when it detects one of such manner.

Again, the problem is that ird_tools, 3K3Y team's tool, and PS3-ISO-Rebuilder don't take this problem into account. Now, it also looks like Redump2IRD doesn't take this into account as well, and calculates the hashes of these non-sequential files as if they were sequential. Of course, it is possible to calculate the hashes correctly, but it needs to be coded differently. The ird format isn't necessarily the problem, because the offsets/sectors used as keys can be re-interpreted as "The offset/sector of the first extent of this file."

Now, another thing you MUST take into account is that decrypted ISOs generated by MultiMan and Irisman aren't actually using the original header of the Blu-Ray disc. They re-create an ISO using the mounted files from the disc. The only reason I mention this is because I know that some people don't know the difference, so they use some tutorials to insert an encryption key into the ISO, re-encrypt the disc, and then generate an invalid IRD file (with a header that isn't the original).

 

[sandungas]

As for burned ISOs of dual-layer games not working... I don't think it's a problem with makeps3iso (or split files, for that matter). It sounds more like a problem with how cobra/mamba is handling reading the mounted disc (which it wasn't really designed to use, I suppose). Or maybe it's a problem with disc-burning software, where Sony expects some sort of string in the OSA/ISA of dual-layer discs containing games.

I dont want to insist much in this because i dont know where was the problem but i still think it could be indirectly related
Cobra (software) can handle this dual layer decrypted ISO's without problems, the problem is somewhere else inside the ISO

Let me resume the story chronologically, this was a feature found by deank and it seems is the kind of thing he found while he was looking at the code (probably in IDA or real time debugging) and he realized there was some lines of code that allowed to do "the magic trick"
So... he was confident it "should" work, applyed the patch, and started burning discs... but was not working because the ISO was generated with the latest versions of the "cobra ISO tools" (the software mantained by the cobra ODE manufacturers)
At some point deank realized that it was working by generating the ISO with an older version of the cobra ISO tools... in other words... it seems the people of cobra introduced some problem/incompatibility in the latest versions of his tools that was breaking this feature
So... the feature was published with a recomendation to use an old version of the cobra ISO tools, the compatibility with the PS3 game library was fine, etc...

But eventually someone found a few PS3 games that was not working when burned in a writable bluray, and some days later they realized this same games was working fine if the ISOs was generated with makeps3iso... so the compatiblity increased a bit more... and the suggested tool to use to generate the ISOs become makeps3iso, at that time most people (me included) thought the compatibility with the PS3 game library was 100%
But some days later someone realized the games bigger than 25GB was not working :/
And some days later some other people found the solution, what they was doing was to use the official generator tools from the SDK
They never explained exactly what they did but we can deduce they was "cloning" the way how the files was "splitted" in the original dual layer disc






*There are also the other "multiboot disc" that are not fully supported by backup managers (with several EBOOT.BIN files located in folders named PS3_GM01, PS3_GM02, etc...)
https://www.psdevwiki.com/ps3/LIC.DAT#Multiboot_PS3_Game_Discs

 

[datman]

You're confusing the output of ird_tools with the information that is stored inside an ird file. This is what an ird really stores. So some basic disc information, sections of the disc (header, footer, pic, etc), and region/file hashes. The "file 1 key" is the offset/sector of the file's whose hash is shown. Notice that a file's path/name and encryption status aren't stored explicitly in the file. So where does ird_tools and PS3-ISO-Rebuilder get that information? It's all in the header. These programs typically read the ISO-9660 filesystem to get the name of the files, and the encryption table to get a file's encryption status in the original disc. In the header you will also find, of course, how multi-extent files are stored on the disc. So the information about their offsets is all there. It's just that Zar's ird_tools doesn't print it because it assumes that all "split" files (at least in the ISO-9660 fs) are sequential. I guess that 3K3Y team also assumed all files would be stored sequentially on disc, do they didn't take that into account when trying to rebuild from an ird file (and maybe even when creating them). A common iso handling library like libcdio also doesn't take into account files split non-sequentially, and throws a warning when it detects one of such manner.

Again, the problem is that ird_tools, 3K3Y team's tool, and PS3-ISO-Rebuilder don't take this problem into account. Now, it also looks like Redump2IRD doesn't take this into account as well, and calculates the hashes of these non-sequential files as if they were sequential. Of course, it is possible to calculate the hashes correctly, but it needs to be coded differently. The ird format isn't necessarily the problem, because the offsets/sectors used as keys can be re-interpreted as "The offset/sector of the first extent of this file."

OK, fair enough. That makes sense. I've looked at the problem the wrong way round.

So, as you say, the problem really lies only with the tools, not the IRD files themselves. If the tools interpreted them 100% correctly then it should be possible to rebuild *any* PS3 disc. I don't know if anyone would be able to patch/update IsoTools but I think the source code for PS3-ISO-Rebuilder is on GitHub so it's potentially possible that someone could update it but it's going to be down to whether anyone who knows how to can be bothered for what are a few fairly niche cases.

 

[Deterous]

Hi, just wanted to add onto this thread that current IRD files *do* contain incorrect file hashes for non-contiguous files. You can check that 2 files have incorrect hashes for the game BLES-01376.
That includes IRDs made from 3k3y ISOTools, Redump2IRD, and ManaGunZ.
I have created a library to generate IRD files that fixes this issue, as well as make reproducible IRDs (1:1 correspondence with redump ISOs). The library is on my GitHub here:
github.com/Deterous/LibIRD

Also, I made a command-line tool to create IRD files from just the ISO: github.com/Deterous/LibIRD/releases/
It is much faster than Redump2IRD and does not require the disc key/disc ID/PIC. If the hashes and key are in redump, just run `irdkit.exe create game.iso` and it will create an IRD for you. It can also do bulk IRD creation for a whole folder, or even recursively search within a folder. More advanced options are available.
Additionally, it can print information about ISO and IRD files with `irdkit.exe info game.iso` and even compare two IRDs to see what differs with `irdkit.exe diff game1.ird game2.ird`

However, the IRD file with correct hashes (BLES-01376) still does not work with PS3-ISO-Rebuilder. A new PS3 ISO rebuilding software will need to be used for these problematic discs.