There are none proprietary file systems on PS3 HDD. Partitions (because there are many) using FAT16, FAT32 and UFS2. Some doesn't have any filesystem.
The only proprietary fs which Sony ever used was APA+PFS in PS2 HDD (if we can call APA a file system, it is some kind of mandatory header glued to every partition, yet can contains data like executable, icon etc.), MCFS in PS2 MC (both by Sony) and TexFAT (by Microsoft) on PSV/PSTV internal flash partitions and memory cards.
Unlocked reserves stick to UFS2 because it is one the features of this fs. The same can be done on PS4 because it also using UFS2 (at least on partition no.27).