libntfs_ext library compilation with cellsdk

Zar said:
I think we should do light version of the lib without I/O and without EXT support. with a "#define _CELLSDK_LIGHT" tag ?
No one is using ext hdd and the use of stdio function slow the execution, I noticied it on ManaGunZ.
Moreover, the size is too big too include it in sprx like webMAN MOD, it's better to reduce the lightest possible size of the sprx.
Click to expand...
Interesting. None of stdio functions could be useful for webMAN MOD?

When you tell too big you mean too big for the RAM ?

The better would be to totally porting the library to cellsdk, and let the dev compile it like they want it. For now, if we have a working base, then we could add ext support and I/O later. But we need to test deeper all the main functions of the library first.

#define _CELLSDK_LIGHT is like without -DPS3_STDIO and -DWITH_EXT_SUPPORT, it seems not revelant according to me. I mean PS3_STDIO and ext support are additionals features, not the purpose of the lib itself.
 
I tried to compile webMAN with the library. I included ntfs.h in ftp.h, i added the library in the makefile. I also just used this function "ntfsMountAll" to simplify things because i hadn't test "ps3ntfs_diropen" and "ps3ntfs_dirnext" yet.

While compiling when "ppu-lv2-prx-fixup" is called i've got a lot of undefined reference.

Code: 
undefined reference to `malloc'
undefined reference to `memalign'
undefined reference to `realloc'
undefined reference to `memmove'
undefined reference to `free'
undefined reference to `mbsrtowcs'
undefined reference to `mbsinit'
undefined reference to `mbrtowc'
undefined reference to `mbsinit'
undefined reference to `wcrtomb'
undefined reference to `ceil'
undefined reference to `floor'
undefined reference to `_Errno'
undefined reference to `_Geterrno'
undefined reference to `time'
undefined reference to `strdup'

So i called the function from main.c and included ntfs.h in main.c instead of ftp.h and the sprx built fine without errors.
 
Last edited:
freddy38510 said:
I tried to compile webMAN with the library. I included ntfs.h in ftp.h, i added the library in the makefile. I also just used this function "ntfsMountAll" to simplify things because i hadn't test "ps3ntfs_diropen" and "ps3ntfs_dirnext" yet.

While compiling when "ppu-lv2-prx-fixup" is called i've got a lot of undefined reference.

Code: 
undefined reference to `malloc'
undefined reference to `memalign'
undefined reference to `realloc'
undefined reference to `memmove'
undefined reference to `free'
undefined reference to `mbsrtowcs'
undefined reference to `mbsinit'
undefined reference to `mbrtowc'
undefined reference to `mbsinit'
undefined reference to `wcrtomb'
undefined reference to `ceil'
undefined reference to `floor'
undefined reference to `_Errno'
undefined reference to `_Geterrno'
undefined reference to `time'
undefined reference to `strdup'

So i called the function from main.c and included ntfs.h in main.c instead of ftp.h and the sprx built fine without errors.
Click to expand...
Have you remembered to use
#define NTFS_EXT 1 in the flags.h or something similar wherever otherwise the ntfs part in ftp.h won't be used afaik...
Maybe you didn't use that part of the code if you tested ntfsMountAll...

Just tried myself to run it as aldostools requested but am getting errors from compiling ftp.h from line 562 onwards due to variables declared in the #else part of the code like slen, is_root, d_path_len, mode & dirtype...

Edit:
After modifying the algorithm in ftp.h, it compiles fine until like freddy38510 posted previously it hits a long list of undefined references to malloc, memalign, free etc....

Reverted all changes with ftp.h & used main.c directly with a new web command. Obviously ntfs.h is now included in main.c, the undefined reference errors keep piling up all the same...

Not sure yet why the errors stopped for freddy38510 but not for me..

Btw all this testing is done on Windows & Cygwin...
 
Last edited:
bguerville said:
Have you remembered to use
#define NTFS_EXT 1 in the flags.h or something similar wherever otherwise the ntfs part in ftp.h won't be used afaik...
Maybe you didn't use that part of the code if you tested ntfsMountAll...

Just tried myself to run it as aldostools requested but am getting errors from compiling ftp.h from line 562 onwards due to variables declared in the #else part of the code like slen, is_root, d_path_len, mode & dirtype...

Edit:
After modifying the algorithm in ftp.h, it compiles fine until like freddy38510 posted previously it hits a long list of undefined references to malloc, memalign, free etc....
Click to expand...
Same as you, but when i called the function from main.c i've got no undefined references. Is this related to ftp.h because it's a header file ?

malloc, memalign, etc are functions from stdlib.h and it's already included in the webman project. So we missing something...
 
freddy38510 said:
Same as you, but when i called the function from main.c i've got no undefined references. Is this related to ftp.h because it's a header file ?

malloc, memalign, etc are functions from stdlib.h and it's already included in the webman project. So we missing something...
Click to expand...
Just look at my edits in my previous post.
Even when I try using the lib in main.c I still get the errors....
Gotta take a break for a couple of hours but I will get back at it soon..
 
bguerville said:
Just look at my edits in my previous post.
Even when I try using the lib in main.c I still get the errors....
Gotta take a break for a couple of hours but I will get back at it soon..
Click to expand...
I did a "git reset --hard" so i can't retrieve what i did when it worked. That was probably a false positive (called the function somewhere in the code where it's not compiled related to flags file). I tried in a new separate c file, and got the undefined references too.

I think, before i look in the library code, i will try to compile with a simple sprx project.
 
It's a good idea as It could be sprx related...
We know that self files can be compiled but libntfs has never been used in a sprx before as psl1ght does not support prx compilation..
 
bguerville said:
It's a good idea as It could be sprx related...
We know that self files can be compiled but libntfs has never been used in a sprx before as psl1ght does not support prx compilation..
Click to expand...
If we can compile elf without those errors, why have we got them while we compiling prx ? Maybe we are not linking the library in the good way. What ppu-lv2-prx-fixup fixes actually ?
 
I think i found how to solve that. Just add "lc" to PPU_PRX_LDLIBS. This will solve some undefined references.
We have to find the others libraries that are missing.

Edit: Add "lm" too.

Edit2: "lc_stub" instead of "lc" and "lm_stub" instead of "lm"

Edit3: The only undefined reference is now
Code: 
`_Errno'
I take a break.

Edit4:
Code: 
[freddy:~/Desktop/PS3/webMAN-MOD/lib] $ nm -A *.a | grep _Errno
libntfs_ext.a:ntfs.ppu.o:                 U _Errno
libntfs_ext.a:ntfsinternal.ppu.o:                 U _Errno
libntfs_ext.a:ext2_internal.ppu.o:                 U _Errno

U = The symbol is undefined
Click to expand...

Code: 
[freddy:/opt/cell/target/ppu/lib] $ nm -A *.a | grep _Errno
libc.a:errno.o:0000000000000000 B _Errno
nm: xldnorm.o: aucun symbole
nm: xmtx.o: aucun symbole
nm: xtls.o: aucun symbole
libcs.a:errno.o:0000000000000000 B _Errno
nm: gcctinfo.o: aucun symbole
nm: xfpostox.o: aucun symbole

B = The symbol is in the uninitialized data section (known as BSS).
Click to expand...

Edit5: I fixed it.
In ntfs.c, ntfsinternal.c and ext2_internal.c when i porting the library i defined "__PME__" before included errno.h to solved reference to "EADDRNOTAVAIL" and "EADDRINUSE".

So i removed #define __PME__ and define EADDRNOTAVAIL and EADDRINUSE in cellos_lv2.h like that
Code: 
#define EADDRNOTAVAIL EAGAIN
#define EADDRINUSE EBUSY
I'm not sure if i defined them with the good equivalent. Tell me what you think about that @bguerville , @Zar .
For information in errno.h if "__PME__" is defined :
Code: 
#define EADDRNOTAVAIL (-14)
#define EADDRINUSE    (-15)
Otherwise it's not defined.

EADDRNOTAVAIL is used in the library when the maximum number (10) of mounts available at one time is exceeded.

EADDRINUSE is used in this context:
Code: 
    // Check that the requested mount name is free
    if (ntfsGetDevice(name, false)) {
        errno = EADDRINUSE;
        return false;
    }

What i found somehow equivalent :
Code: 
** The resource is temporarily unavailable.
*
* e.g. The number of threads in the system is exceeding the limit.
*/
#define EAGAIN        -2147418111    /* 0x80010001 */

Code: 
/** The device or resource is busy */
#define EBUSY        -2147418102    /* 0x8001000A */

Anyway, the prx can now be compiled fine with the library.
 
Last edited:
Sorry for the previous long post.

I tried to make the library lighter (see last commit on the repo). Maybe if we don't link libc_stub and includes the missing functions in libc.c we can minimize the size of the plugin. What do you think about that guys ?
 
Success indeed as far as compilation is concerned.. Lol..
Nicely done Fred...

Yes libc_stub functions can be added to libc.c, that's exactly what has been done previously...

The defines are good but not sure about the values.

The original ones were
#define EADDRINUSE 112
#define EADDRNOTAVAIL 125
Not that it matters here...

But why did you have to use EBUSY & EAGAIN instead of defining them with the values used in ppu/include/errno.h ?

A lighter version might actually be appropriate & we all wish for webMAN-MOD to remain as small a plugin as possible however we can't keep it under the allocation threshold that Zar mentioned forever.
As wMM gets new features we got really close to that boundary. Aldo has already reduced the plugin size 2 or 3 times at least recently. There is probably more optimisation that can be done but no matter what, we are so close already to that limit it will be difficult not to reach it with ntfs support added...

FYI webMAN-MOD Full Edition is currently at 287kb with an added web command mounting ntfs drives & outputting the 1st mounted drive root contents...
That's over 100kb added to the plugin....
The libntfs_ext.a library is smaller than it used to be at 660kb, it was previously 1220kb...(psl1ght libntfs_ext usually used is over 1450kb)
 
Last edited:
bguerville said:
The defines are good but not sure about the values.

The original ones were
#define EADDRINUSE 112
#define EADDRNOTAVAIL 125
Not that it matters here...

But why did you have to use EBUSY & EAGAIN instead of defining them with the values used in ppu/include/errno.h ?
Click to expand...
That's the values from psl1ght. In cellsdk i found negative values. Look closer to errno.h taking care of "#ifdef". Then look to sys/return_codes.h
I replaced EADDRINUSE and EADDRNOTAVAIL with some defines in return_codes.h

FYI webMAN-MOD Full Edition is currently at 287kb with an added web command mounting ntfs drives & outputting the 1st mounted drive root contents...
That's over 100kb added to the plugin....
The libntfs_ext.a library is smaller than it used to be at 660kb, it was previously 1220kb...(psl1ght libntfs_ext usually used is over 1450kb)
Click to expand...

That's because i removed c files from ext2fs folder from compilation because we don't want ext support. So there is left half the files we have at the beginning.

libc_stub.a is at 1264kb and libm_stub.a is at 1076kb. I don't know how many it adds to the project, but we can try to remove them and includes missing functions in c files.

What about memory profile usage (Min, standard, max, etc) with ntfs support ? Could it lighten scanning ?
 
freddy38510 said:
That's the values from psl1ght. In cellsdk i found negative values. Look closer to errno.h taking care of "#ifdef". Then look to sys/return_codes.h
I replaced EADDRINUSE and EADDRNOTAVAIL with some defines in return_codes.h



That's because i removed c files from ext2fs folder from compilation because we don't want ext support. So there is left half the files we have at the beginning.

libc_stub.a is at 1264kb and libm_stub.a is at 1076kb. I don't know how many it adds to the project, but we can try to remove them and includes missing functions in c files.

What about memory profile usage (Min, standard, max, etc) with ntfs support ? Could it lighten scanning ?
Click to expand...

I know about the error values, what I meant was that the values I gave were the originals (psl1ght) but didn't matter anymore. What I expected you to use were the ps3sdk values - 14 & - 15 so I was wondering why you didn't define those values directly....
What was the point in defining them as other errors instead? Sorry I could not look at sys/errno.h as I am not at home tonight...
To me, #define EOPNOTSUP ENOTSUP sounds good because it is exactly the same error but #define EADDRNOTAVAIL EAGAIN when cellsdk actually uses a EADDRNOTAVAIL in errno.h makes me pause... ?
Or maybe I missed something yesterday when I looked and EAGAIN uses the same value as EADDRNOTAVAI, as I said I can't check right now because I don't have the sdk with me.. If I have missed something then please forgive my babbling ok..? Lol

About the library file size being reduced, yes I saw the makefile commit eventually & realised yesterday that you had removed half the project files making the library smaller.

WMM memory profile usage is currently tweaked by the user according to the number of games (& game type) the user intends to list using "Refresh Xml". At the moment, without using ntfs, users can list up to about 800/1000 games, depending on which categories were chosen for scanning in setup.ps3. If you choose categories that are actually devoid of entries, you will still lose part of the preallocated memory. That reduces the maximum amount of entries...
There must be a little flaw in the wMM code actually because why allocate memory to an empty list...?

Another possibility to keep the plugin smaller might be to cheat by creating a separate dynamic sprx, capable to deliver the ntfs information wMM needs whenever called....
PrepNTFS could probably even be turned into such a sprx actually... Shame it was made with psl1ght though.. another port on top... lol...
I have already started something like that for md5 hashing a few weeks/months ago, I never finished the project but I got the concept to work iirc... I am pretty sure it can be done with libntfs...
Hell maybe we could turn it into a system sprx that could be used by other brews as well...

Before looking into integration strategies, the library functions need to be tested thoroughly though. I am hoping to do this tonight, I already have everything setup for it, I just need a few hours free time for debugging...
 
Last edited:
I had a deeper look into the example, the originals function was designed to be in loop and I removed the loop that's why it wasn't working. Estwald even wrote it in comment, I didn't notice.

So, I wrote one example from the ground. It almost test every functions and every one of them is working. Every function are written in a loop to 'force' it, It didn't work without this loop. Perhaps, it can be fixed with the use of errno...

I didn't take a look at sprx, yet.
 
Zar said:
I had a deeper look into the example, the originals function was designed to be in loop and I removed the loop that's why it wasn't working. Estwald even wrote it in comment, I didn't notice.

So, I wrote one example from the ground. It almost test every functions and every one of them is working. Every function are wrote in a loop to 'force' it, It didn't work without this loop. Perhaps, it can be fixed with the use of errno...

I didn't take a look at sprx, yet.
Click to expand...

Oh. That's good news...
So every function is tested & works well?
 
Zar said:
But this ones aren't rly important... I think 'opendir' / 'dirnext' / 'closedir' function are the only function used in prepntfs :p
Click to expand...
Sure but full ntfs integration in wMM may require the use of more functions..
How many do you use in Managunz so far?
 
Almost, all of them in the file manager. As you can see in the sample there is several way to mount NTFS, managunz is using ntfsmountAll because it's simpler. I'm using standard function to have only one uniform write for FAT32 and NTFS.
 
You must log in or register to reply here.

Similar threads

Back
Top