PS3 USB Attached SCSI ( UAS ) problem with external NTFS hard drive

[TheArcuzHunter]

I have 2 PS3s, one with HFW 4.88.1 HEN and the other with CFW evilnat 4.88.2.
HFW PS3 isn't recognizing Both of my USB external hard drives in "USB Attached SCSI ( UAS )" storage controllers, but it does read 1 of them (red case external hard drive) in irisman 4.88 on CFW 4.88.2.
I'm still using the same irisman, But for some reason the blue case external hard drive doesn't show up no matter how long I wait in the file manager, and these are both the same external hard drive enclosure with a Normal USB A to Micro B USB cable, with USB 3.0 interface.
The red casing one has my physical copies of my disc games on it in PS3ISO in ISO format and the CFW PS3 reads it just fine on the first try every time.
The blue casing one has 2 folders, one with thousands of family archives and documents and one with a few ps3 tools inside of them.
What can I do to make it detect both of them on the HFW PS3 and the blue one on the CFW PS3? Any help is appreciated!!

 

[TheArcuzHunter]

Side note, the blue casing hard drive doesn't have any transported isos on it! Only tools, but I wanted to digitalize a few more disc games I have into the blue casing one, but irisman won't read the external hard drive. Again, any help is appreciated and thank you for reading this!

 

[TheArcuzHunter]

If anyone can give out any information I highly appreciate it!

 

[bombo_tondo]

Yep, some USB Attached SCSI chips are weird...
Unfortunately I think there's nothing you can do with that.
Have you tried other File Managers?
For example I found a way to recognize NTFS into multiMAN (I'm on Evilnat 4.88.2 [Cobra 8.3]):

• Soft-Disable COBRA
• Shutdown PS3
• Remove USB
• Start PS3
• Start standard multiMAN
• Connect the NTFS USB drive
• Click PFS Driver and wait for it to scan attached drive
  
• Look for pvd_usb000 inside File Manager
  

 

[bguerville]

Yep, some USB Attached SCSI chips are weird...
Unfortunately I think there's nothing you can do with that.
Have you tried other File Managers?
For example I found a way to recognize NTFS into multiMAN (I'm on Evilnat 4.88.2 [Cobra 8.3]):

• Soft-Disable COBRA
• Shutdown PS3
• Remove USB
• Start PS3
• Start standard multiMAN
• Connect the NTFS USB drive
• Click PFS Driver and wait for it to scan attached drive
  
• Look for pvd_usb000 inside File Manager
  

The PFS driver is readonly, it doesn't support all external disks, some cannot be recognised, even when you provide multiman with a USB.cfg file to force the device ID detection.

Other file managers, like the ones included in Irisman or Managunz, use a ntfs library (originally written for Android iirc) ported to ps3. The level of support is much better & that library can read & write.

In webman/sman & other sprx plugins, due to memory limitations, the ntfs library was modified to support a lower threshold of ntfs clusters, depending on the ntfs partition size, one needs to use a partitioning tool to increase the cluster size & reduce the total number of ntfs clusters for the partition to be usable in the plugin file manager or FTP.

 

[TheArcuzHunter]

The PFS driver is readonly, it doesn't support all external disks, some cannot be recognised, even when you provide multiman with a USB.cfg file to force the device ID detection.

Other file managers, like the ones included in Irisman or Managunz, use a ntfs library (originally written for Android iirc) ported to ps3. The level of support is much better & that library can read & write.

In webman/sman & other sprx plugins, due to memory limitations, the ntfs library was modified to support a lower threshold of ntfs clusters, depending on the ntfs partition size, one needs to use a partitioning tool to increase the cluster size & reduce the total number of ntfs clusters for the partition to be usable in the plugin file manager or FTP.

Thank you both four answers! @bombo_tondo and @bguerville !

Also, what exactly do I need to look up? Just "how to decrease the ntfs clusters in my external hard drive"?

Both of mine have 320 GB total storage, and the blue casing one I spoke of in the OP has an old internal ps3 hard drive into the casing of a Hard drive adapter to USB, I formatted it and everything a while back and my PC recognizes and uses it like normal, so even then, just decreasing the ntfs clusters will do the trick (maybe)?

 

[bguerville]

Also, what exactly do I need to look up? Just "how to decrease the ntfs clusters in my external hard drive"?

Both of mine have 320 GB total storage, and the blue casing one I spoke of in the OP has an old internal ps3 hard drive into the casing of a Hard drive adapter to USB, I formatted it and everything a while back and my PC recognizes and uses it like normal, so even then, just decreasing the ntfs clusters will do the trick (maybe)?

The ntfs cluster size issue can only impact the sprx plugins using the libntfs_ext library in other words, projects like sMan/wMM & other sprx plugins, all other backup managers are not impacted.

This issue should mostly come up with big partitions using the default ntfs cluster size (4kb).
I am not sure about the maximum size of ntfs partition formatted with 4kb clusters before it can be subject to these problems, it shouldn't be difficult to calculate though.

When he added libntfs support to sMan, deank realised that in his sprx project, the CACHE_DEFAULT_PAGE_COUNT (in ntfs.h) variable usually set to 128 for every other self project, could only be set to 16 maximum otherwise some memory related issues began to appear, this default cache page count reduction led de facto to compatibility issues with big partitions using the 4kb cluster size that most formatting tool use by default. Unfortunately nobody looked any further, this change was adopted for all other sprx projects & we are left with this ntfs cluster issue. It's not a problem when you know about it but still, let's hope that one day, someone will spend time to find a better solution than having to modify partition properties, there has to be a way to support all ntfs cluster sizes within specs.

In truth, the best way would be to add ntfs support at kernel level directly so that ntfs would be natively supported by GameOS, accessible from XMB, apps & homebrews alike. Similarly, we would need to add exFAT, GPT & TRIM support too, maybe even SAMBA, that would be nice.
Also I see no reason why we shouldn't be able to port the freebsd 4.4 source code to support all these things to the kernel as kernel plugins rather than use third party libraries.

Any decent partition manager can be used to modify the ntfs cluster size without having to reformat the partition or lose data.
For instance, on Windows MiniTool, Partition Magic, etc. and on Linux Gparted, etc..

On external storage, ntfs is most relevant for iso file storage as those files are always very big, using a higher cluster size makes sense anyway.
When you use a ntfs partition only for iso storage, you could in theory use much bigger clusters than the default 4kb, like 64kb or even the maximum 128kb I suppose.

If you use the ntfs partition to store many small files however it's better to keep the cluster size fairly low so as to avoid lost disk usage space.
If you are having problems with sMan or wMM & the partition's cluster size is currently set to default 4kb, you can try setting it to 8kb, the total number of clusters will be halved & the problem should be solved.

 

[bombo_tondo]

In truth, the best way would be to add ntfs support at kernel level directly so that ntfs would be natively supported by GameOS, accessible from XMB, apps & homebrews alike. Similarly, we would need to add GPT & TRIM support too, maybe even SAMBA, that would be nice.
Also I see no reason why we shouldn't be able to port the freebsd 4.4 source code to support all these things to the kernel as kernel plugins rather than use third party libraries.

<Off-Topic>
@bguerville

I always wondered about filesystems support on homebrews scenes (I am speaking of all systems, not only Sony's consoles)... ...
Generally speaking, why devs (thanks to all of devs by the way! Respect for you all! ) prefer adding custom support for closed (and feature-poor) filesystems, rather than using free/libre opensource filesystems (feature-rich)? It for the sake of friendly usage / adoption for n00b end users like us?
</Off-Topic>

 

[bguerville]

<Off-Topic>
@bguerville

I always wondered about filesystems support on homebrews scenes (I am speaking of all systems, not only Sony's consoles)... ...
Generally speaking, why devs (thanks to all of devs by the way! Respect for you all! ) prefer adding custom support for closed (and feature-poor) filesystems, rather than using free/libre opensource filesystems (feature-rich)? It for the sake of friendly usage / adoption for n00b end users like us?
</Off-Topic>

In 2006, S#ny devs chose encrypted UFS2 with special hardware taking care of the encryption for the ps3 internal hdd. It was a fair choice to make given requirements, constraints & objectives, and it was open source & feature rich.
For external storage, they provided fat32 read/write support, also a fair choice at the time.

As to homebrew devs, well they first used natively supported FAT32 then later a read/write ntfs library for Android (ntfs-3g) was ported to GameOS by Estwald & ntfs support was gradually added to all backup managers & their respective file managers.
Deank never bothered to add ntfs support to multiman's file manager, only to the game backup functionalities. Multiman's file manager uses a PFS driver with limited readonly ntfs support which was added before the libntfs port iirc.
More recently an exFAT library (as well as SAMBA & NFS libraries) was also ported to GameOS & some backup managers like Managunz or wMM were updated to use it.

All the fs libraries used by ps3 homebrew devs are open source & the great majority of homebrews is open source, there are very few exceptions, the source of projects like multiman 4.xx, psnpatch, sen enabler is closed.

NTFS, exFAT & FAT32 are the most commonly used partition types out there, I am not sure to what extent support for other file systems would help the scene.

On other consoles, the problematic is exactly the same, devs use whatever is available natively & with time, they try to add fs support if/when relevant, within specs & hardware based limitations, by leveraging open source libraries they usually need to port first.

Developing for a console like the ps3 is not easy, the OS (GameOS) is proprietary (source closed), a mixture of open source freebsd 4.4/netbsd 2.0(?) with custom parts, running on a unique hardware setup & a custom PowerPC architecture (ppc was used mostly in Mac computers at the time) .
Technically speaking, freebsd 4.4 source code for fs support should be reusable however it is never a done deal, the code still needs to be ported for GameOS & its specificities, ultimately not everything can easily be ported.

Look at it this way, imagine if you had a PC running a closed source Operating System with only MBR/FAT32 file system support, no way to modify the kernel without using hacks & no readily available development kit to create low level drivers & libraries.
How would you go about adding ntfs or exfat support to your system?
The best a dev could do is port an existing ntfs or exfat library for a similar architecture (if there is one) then use that library to add ntfs/exfat read/write support to a file manager.
However to add ntfs support system wide, hacking would be needed as well as advanced C/C++ compilation skills, the kinda work that is often beyond the skills of a developer.
This description sums up the ps3 situation keeping in mind that the ps3 also uses various layers of security complicating matters further, especially on metldr.2 models.