Not in that order... one of the tricks involved is to use a feature named
HPA supported by many/all modern HDD's that stores the HDD capacity (in other words, the number of sectors) inside the HDD BIOS
Lets say... a hdd of 2TB have from factory 24.654.345 of sectors... and you configure it to "crop" the capacity to only 24.000.000 sectors
By doing that you are "hiding" a lot of sectors that are completly invisible to the BIOS of most/all PC's (and to the PS3)
Technically... the PS3 doesnt really knows that the HDD has been "cropped"... is just taken as an HDD with a smaller capacity
-----------
But there are 2 different ways to take advantage of this trick...
If you take an HDD of 2TB you can reduce it to something around what
@atreyu187 said... and use an old PS3 firmware to perform the HDD partition creation, filesystems creation, file installation, etc... because the old PS3 firmwares was not so restrictive with the HDD capacity
The payback is that you are not going to be able to run any of the filesystem manteinance functions... some happens periodically at the background when you are in XMB, and the most important is the option that appears in recovery menu named "restore filesystem" (the same process triggered automatically after some PS3 freezes/crashes). This process is the responsible of cleaning-up the small errors that uses to cummulate in the filesystems, if you "break" it your PS3 is not going to be able to clean-up and repair filesystems ever and eventually the HDD filesystems will collapse and you will lose all the HDD contents
But... everything depends of which kind of use you do of your PS3... if you keep this in mind and you do a periodical backup of worthy files (saves, whatever) you can use this trick to maximize HDD capacity and forget a bit about it, the day it collapses you format it, and start again
The other way is to try to reduce the capacity to the point where the HDD manteinance functions keeps working normally, i dont know the size but it should be something intermediate in between 1TB and 1.5TB... maybe 1.2TB ? dunno
I never did read anyone trying this... most people was interested in maximizing the capacity (in other words... not worrying about the broken filesystem manteinance functions)