PS3 LV2Bridge - Payload and VSH Module that allows communication between VSH and LV2

[StarmanX32]

This is a project I'm working on with @TheRouLetteBoi for the PS3. LV2Bridge consists of two parts: A payload and a VSH Module (SPRX)

Both parts implement an internal server that communicate with each other. In the payload, we're gonna implement a custom syscall that allows a user to call VSH functions through a syscall, RouLette also had the idea to allow the SPRX to call LV2 functions without heavily relying on Syscalls.

How is this useful?
It could be useful for some people, this project allows you to make VSH and LV2 calls without the need to hook VSH functions in your project and load your project as a VSH module or use 1000 different syscalls and have a risk that not everything works properly on all OS environments. This is also personally useful for me, as with this project, I'm improving myself in programming and comprehension on what's happening inside the system when we execute the payload and vsh module code.

It's still WIP (I started setting up everything yesterday) and also wrote the first implementations yesterday, so it's far from perfect, all C code (as of 13.10.2021) is written by me. RouLette already sent some improved codes that I'm gonna use in the codebase later.

We'd like to hear from the other people and especially the developers here on what you think about the project.

I wasn't able to test the project yet and I need to recheck if I calculated the correct addresses for the networking functions in the LV2 kernel correctly. (Pretty sure I calculated it wrong )

Here's the link to the project: https://github.com/PHTNCx64/lv2bridge

 

[StarmanX32]

Btw. if you have any suggestions or improved code, feel free to make a request on the GitHub page!


Gesendet von iPhone mit Tapatalk

 

[bguerville]

Conceptually I am not sure I understand the benefits of going through lv2 to call vsh functions, the kernel is not any wiser as to what a user app should call in vsh as a user app itself might be.
To call vsh functions & subs, you are much better off remaining in userland..
What we might do at kernel level is hook the module loading function so that sprx plugins can be patched dynamically whenever they are loaded but beyond that, once loaded, access to the plugins functions is always better done in userland, same as vsh.

As to using a server/client strategy to communicate between userland/kernel, it adds much overhead & no particular benefit over a custom syscall using an opcode system.


In other words, if the aim of the game is to be able to call lv2 functions with arguments directly from userland & return the results to userland, there is already a syscall 15 implementation for that very purpose.
For everything else, you can extend the syscall 8 opcodes or the ps3mapi sub-opcodes.

 

[StarmanX32]

Conceptually I am not sure I understand the benefits of going through lv2 to call vsh functions, the kernel is not any wiser as to what a user app should call in vsh as a user app itself might be.
To call vsh functions & subs, you are much better off remaining in userland..
What we might do at kernel level is hook the module loading function so that sprx plugins can be patched dynamically whenever they are loaded but beyond that, once loaded, access to the plugins functions is always better done in userland, same as vsh.

As to using a server/client strategy to communicate between userland/kernel, it adds much overhead & no particular benefit over a custom syscall using an opcode system.


In other words, if the aim of the game is to be able to call lv2 functions with arguments directly from userland & return the results to userland, there is already a syscall 15 implementation for that very purpose.
For everything else, you can extend the syscall 8 opcodes or the ps3mapi sub-opcodes.

Thanks for your opinion.

The goal here is to easily access VSH functions from any type of project. From different SPRX Modules (Mod menus for example) to homebrew applications that do not hook any module to the VSH. The server/client strategy was suggested by me as I'm not really familiar with other IPC techniques like pipes for example.

You could imagine it like that, you don't need to create an extra VSH module to call VSH notify for example, you can just make a call to the custom syscall and it will pass the request to our VSH module and our VSH module handles the request and makes a call on VSH notify and returns it back to the syscall.


Gesendet von iPhone mit Tapatalk

 

[bucanero]

it would be nice if at some point you could share a sample tool or homebrew that uses your bridge, so others could understand better the possibilities behind it

just as an example: let's say I port a library to PSL1GHT, then I try to do a basic sample to see if the library works and also to show other devs how they could use it.

 

[bguerville]

The vsh notification sub you mentioned is a vsh plugin export. Exports are created to be reusable, it's the whole point behind the import/export system.
When s#ny compiles a vsh plugin sprx file, its exported functions are added to a stub library generated by the compiler & whenever another project requires those exports, that stub library is linked.
We don't have all those stub libraries to use ourselves as s#ny doesn't redistribute those files in the official sdk. They are often needed for prx plugin compilation due to regular libc functions not being linkable but devs were never meant to develop prx plugins for vsh!
Homebrew stubs libraries for the most common vsh exports were made a few years ago but the entire vsh plugin set is not covered afaik.
Ultimately the export NID & the name of the plugin in which the export is defined is all we really need to use an export.

In other words, when you need to use a vsh plugin export in a userland app, you have 2 options:
1. Make your own library in which you register the NIDs for the exports you need & add the library to your project's additional library list, the exports will be linked at compile time.
That's how many psl1ght sdk accompanying vsh plugin libraries are made iirc, just registering exports with NID, and that is how those vsh export libraries you reused in your project (like libstdc_export_stub.a etc..) were made too.
For instance libc functions cannot be linked in sprx projects for plugins so you must use vsh exports instead otherwise the project won't link.
Yet you didn't need a lv2 bridge or a custom syscall to use the vsh libc functions in your sprx, the stdc_export_stub.a just provided them for you as exports. The same thing can be done with any other project needing to use vsh or vsh plugin exports.
Another example is the notification you mentioned earlier. The export can be found in the libvshtask_export_stub.a library & the declaration is:

extern int32_t vshtask_A02D46E7(int32_t arg, const char *msg);

#define vshtask_notify(msg) vshtask_A02D46E7(0, msg)

In your app, all you need to do is add the library to the library list processed by the linker, add the declaration above in the code (just include the vsh task export header file ) then you can call:

vshtask_notify("Hello World");

If you wanna help devs using available vsh exports, the best is probably to complete the set of export libraries that devs can link into their projects along with corresponding include files to declare the exports.

2. Use the NIDs & the getNIDfunction to define the exports needed by the project at runtime.
I recommend this only with small projects though, bigger projects are easier to manage with export libraries.

How is it better to call vsh exports through a system of kernel payload + a userland sprx plugin communicating via network socket than through the app code directly with one of the 2 options above?

And even if you wanted to make a dedicated sprx plugin to resolve vsh exports at runtime for other apps, you still wouldn't need a lv2 bridge and/or a dedicated custom syscall to do it..

And just to be clear so there is no misunderstanding, this is no criticism, please don't take my comments the wrong way, all development projects are welcome & encouraged, I am only trying to understand your project as you did ask for our opinions, suggestions & feedback.
Like bucanero suggested maybe you should add a sample to illustrate what you have in mind because as is, I still don't see the practical rationale behind your bridge concept or its added value. But then again maybe I missed something, a practical example (sample) may clarify things.
.

 

[StarmanX32]

The vsh notification sub you mentioned is a vsh plugin export. Exports are created to be reusable, it's the whole point behind the import/export system.
When s#ny compiles a vsh plugin sprx file, its exported functions are added to a stub library generated by the compiler & whenever another project requires those exports, that stub library is linked.
We don't have all those stub libraries to use ourselves as s#ny doesn't redistribute those files in the official sdk. They are often needed for prx plugin compilation due to regular libc functions not being linkable but devs were never meant to develop prx plugins for vsh!
Homebrew stubs libraries for the most common vsh exports were made a few years ago but the entire vsh plugin set is not covered afaik.
Ultimately the export NID & the name of the plugin in which the export is defined is all we really need to use an export.

In other words, when you need to use a vsh plugin export in a userland app, you have 2 options:
1. Make your own library in which you register the NIDs for the exports you need & add the library to your project's additional library list, the exports will be linked at compile time.
That's how many psl1ght sdk accompanying vsh plugin libraries are made iirc, just registering exports with NID, and that is how those vsh export libraries you reused in your project (like libstdc_export_stub.a etc..) were made too.
For instance libc functions cannot be linked in sprx projects for plugins so you must use vsh exports instead otherwise the project won't link.
Yet you didn't need a lv2 bridge or a custom syscall to use the vsh libc functions in your sprx, the stdc_export_stub.a just provided them for you as exports. The same thing can be done with any other project needing to use vsh or vsh plugin exports.
Another example is the notification you mentioned earlier. The export can be found in the libvshtask_export_stub.a library & the declaration is:

extern int32_t vshtask_A02D46E7(int32_t arg, const char *msg);

#define vshtask_notify(msg) vshtask_A02D46E7(0, msg)

In your app, all you need to do is add the library to the library list processed by the linker, add the declaration above in the code (just include the vsh task export header file ) then you can call:

vshtask_notify("Hello World");

If you wanna help devs using available vsh exports, the best is probably to complete the set of export libraries that devs can link into their projects along with corresponding include files to declare the exports.

2. Use the NIDs & the getNIDfunction to define the exports needed by the project at runtime.
I recommend this only with small projects though, bigger projects are easier to manage with export libraries.

How is it better to call vsh exports through a system of kernel payload + a userland sprx plugin communicating via network socket than through the app code directly with one of the 2 options above?

And even if you wanted to make a dedicated sprx plugin to resolve vsh exports at runtime for other apps, you still wouldn't need a lv2 bridge and/or a dedicated custom syscall to do it..

And just to be clear so there is no misunderstanding, this is no criticism, please don't take my comments the wrong way, all development projects are welcome & encouraged, I am only trying to understand your project as you did ask for our opinions, suggestions & feedback.
Like bucanero suggested maybe you should add a sample to illustrate what you have in mind because as is, I still don't see the practical rationale behind your bridge concept or its added value. But then again maybe I missed something, a practical example (sample) may clarify things.
.

The vsh notification sub you mentioned is a vsh plugin export. Exports are created to be reusable, it's the whole point behind the import/export system.
When s#ny compiles a vsh plugin sprx file, its exported functions are added to a stub library generated by the compiler & whenever another project requires those exports, that stub library is linked.
We don't have all those stub libraries to use ourselves as s#ny doesn't redistribute those files in the official sdk. They are often needed for prx plugin compilation due to regular libc functions not being linkable but devs were never meant to develop prx plugins for vsh!
Homebrew stubs libraries for the most common vsh exports were made a few years ago but the entire vsh plugin set is not covered afaik.
Ultimately the export NID & the name of the plugin in which the export is defined is all we really need to use an export.

In other words, when you need to use a vsh plugin export in a userland app, you have 2 options:
1. Make your own library in which you register the NIDs for the exports you need & add the library to your project's additional library list, the exports will be linked at compile time.
That's how many psl1ght sdk accompanying vsh plugin libraries are made iirc, just registering exports with NID, and that is how those vsh export libraries you reused in your project (like libstdc_export_stub.a etc..) were made too.
For instance libc functions cannot be linked in sprx projects for plugins so you must use vsh exports instead otherwise the project won't link.
Yet you didn't need a lv2 bridge or a custom syscall to use the vsh libc functions in your sprx, the stdc_export_stub.a just provided them for you as exports. The same thing can be done with any other project needing to use vsh or vsh plugin exports.
Another example is the notification you mentioned earlier. The export can be found in the libvshtask_export_stub.a library & the declaration is:

extern int32_t vshtask_A02D46E7(int32_t arg, const char *msg);

#define vshtask_notify(msg) vshtask_A02D46E7(0, msg)

In your app, all you need to do is add the library to the library list processed by the linker, add the declaration above in the code (just include the vsh task export header file ) then you can call:

vshtask_notify("Hello World");

If you wanna help devs using available vsh exports, the best is probably to complete the set of export libraries that devs can link into their projects along with corresponding include files to declare the exports.

2. Use the NIDs & the getNIDfunction to define the exports needed by the project at runtime.
I recommend this only with small projects though, bigger projects are easier to manage with export libraries.

How is it better to call vsh exports through a system of kernel payload + a userland sprx plugin communicating via network socket than through the app code directly with one of the 2 options above?

And even if you wanted to make a dedicated sprx plugin to resolve vsh exports at runtime for other apps, you still wouldn't need a lv2 bridge and/or a dedicated custom syscall to do it..

And just to be clear so there is no misunderstanding, this is no criticism, please don't take my comments the wrong way, all development projects are welcome & encouraged, I am only trying to understand your project as you did ask for our opinions, suggestions & feedback.
Like bucanero suggested maybe you should add a sample to illustrate what you have in mind because as is, I still don't see the practical rationale behind your bridge concept or its added value. But then again maybe I missed something, a practical example (sample) may clarify things.
.

Thanks for your feedbacks @bucanero and @bguerville.

We're gonna develop both components and see how far we can get and provide some samples then.

@bguerville I initially had this idea about this bridge project. If I'm not mistaken, applications and their corresponding modules are isolated by the OS from each other and you can't just access functions from another application other than yours without some form of an IPC technique. So I had the idea to create a SPRX Module that gets loaded as a VSH module and create an internal server that exposes many VSH functions (not just libc, but also functions that are only available in the VSH). The reason why I came up with the custom syscall is that any application and module on the PS3 can easily call a syscall and request a service from the kernel. So I thought if I could create a 2nd server (well it's a client that makes requests to the sprx module) in the LV2 kernel, then developers that create applications and modules that are completely isolated from the VSH can still easily call VSH functions without creating VSH modules themselves.

So let's say you create an application (self) and run it on your PS3 and you want to display a VSH notification message but you don't have created a VSH module for this, with the bridge you can just do:

system_call_3(206, 0x100, 8, "Hello World");
Where 0x100 is a request to make a vsh notify call, 8 is the icon id and the string is the message you want to display.

That's what I meant, hopefully it makes more sense to you now


Gesendet von iPhone mit Tapatalk

 

[bguerville]

Thanks for your feedbacks @bucanero and @bguerville.

We're gonna develop both components and see how far we can get and provide some samples then.

@bguerville I initially had this idea about this bridge project. If I'm not mistaken, applications and their corresponding modules are isolated by the OS from each other and you can't just access functions from another application other than yours without some form of an IPC technique. So I had the idea to create a SPRX Module that gets loaded as a VSH module and create an internal server that exposes many VSH functions (not just libc, but also functions that are only available in the VSH). The reason why I came up with the custom syscall is that any application and module on the PS3 can easily call a syscall and request a service from the kernel. So I thought if I could create a 2nd server (well it's a client that makes requests to the sprx module) in the LV2 kernel, then developers that create applications and modules that are completely isolated from the VSH can still easily call VSH functions without creating VSH modules themselves.

So let's say you create an application (self) and run it on your PS3 and you want to display a VSH notification message but you don't have created a VSH module for this, with the bridge you can just do:

system_call_3(206, 0x100, 8, "Hello World");
Where 0x100 is a request to make a vsh notify call, 8 is the icon id and the string is the message you want to display.

That's what I meant, hopefully it makes more sense to you now


Gesendet von iPhone mit Tapatalk

OK I understand now.
Sorry for being a bit slow on the intake.. Lol

So basically you mean to make an alternate one way IPC system to communicate with the vsh.self process from a user app/game process. And in some way, you are extending the existing wMM concept with a dedicated kernel payload to do it.

Iirc IPCs are built around a message queue system & semaphores for synchronisation, is that what you aim to recreate?

As a proof of of concept sample, all you need is a small self showing a XMB notification while running, even if it's just a black screen.

 

[bucanero]

OK I understand now.
Sorry for being a bit slow on the intake.. Lol

As a proof of of concept sample, all you need is a small self showing a XMB notification while running, even if it's just a black screen.

for that quick example, @StarmanX32 you can take a sample from PSL1GHT, such as /samples/graphic/blitting

that sample has a basic "pslight" message floating on the screen, and while that app is running you could call the notify popup.

that sample has no dependencies, you only need the psl1ght sdk so every dev could easily check out and try your bridge

 

[StarmanX32]

@bguerville IPC also works through sockets, they're aren't just used for remote connections, I think Unix in general even has a socket type that's meant to be used between processes in one system. Implementing IPC through sockets is the easiest way, but idk honestly about performance when an application for example sends many requests to the bridge at once, so yea we could still try to implement it with named pipes or message queues..

@bucanero Thanks for the sample, I'll take a look at it.

If you guys wonder why we use a new syscall for that rather than extending Syscall 8, RouLette has no problem with it and IMO Syscall 8 is already a fairly big Syscall and keep extending it isn't really great for me, but if many developers wishes the bridge to be implemented in Syscall 8, we can do that!


Gesendet von iPhone mit Tapatalk

 

[bguerville]

@bguerville IPC also works through sockets, they're aren't just used for remote connections, I think Unix in general even has a socket type that's meant to be used between processes in one system. Implementing IPC through sockets is the easiest way, but idk honestly about performance when an application for example sends many requests to the bridge at once, so yea we could still try to implement it with named pipes or message queues..

@bucanero Thanks for the sample, I'll take a look at it.

If you guys wonder why we use a new syscall for that rather than extending Syscall 8, RouLette has no problem with it and IMO Syscall 8 is already a fairly big Syscall and keep extending it isn't really great for me, but if many developers wishes the bridge to be implemented in Syscall 8, we can do that!


Gesendet von iPhone mit Tapatalk

It could be a candidate feature for PS3MAPI (included in Cobra & Mamba), you would only need to add a new sub opcode.
I would recommend not to make yet another custom syscall, cfw & 3rd party tools would need to be updated because of it & it makes for more patching work when it's not absolutely necessary.

I was curious as to how best to go about implementing IPC so I checked the Linux PS3 IPC implementation.
It's as I thought it was, the freebsd implementation is likely to be similar.

Here is a ps3 Linux version if ever you care to look for reference.
https://kernel.googlesource.com/pub/scm/linux/kernel/git/geoff/ps3-linux/+/refs/tags/v4.1.6/ipc

 

[StarmanX32]

It could be a candidate feature for PS3MAPI (included in Cobra & Mamba), you would only need to add a new sub opcode.
I would recommend not to make yet another custom syscall, cfw & 3rd party tools would need to be updated because of it & it makes for more patching work when it's not absolutely necessary.

I was curious as to how best to go about implementing IPC so I checked the Linux PS3 IPC implementation.
It's as I thought it was, the freebsd implementation is likely to be similar.

Here is a ps3 Linux version if ever you care to look for reference.
https://kernel.googlesource.com/pub/scm/linux/kernel/git/geoff/ps3-linux/+/refs/tags/v4.1.6/ipc

Would it be also possible to make it its own API with its own opcode? I mean I don't have an issue with it being a part of PS3MAPI but PS3MAPI is like mostly focused on Process Memory functions, IDPS/PSID, peek poke and system info retrieving functions and lv2bridge is about making VSH function calling much easier. That's just my suggestion. But choose what's the best option for this new potential candidate


Gesendet von iPhone mit Tapatalk

 

[bguerville]

I only suggested PS3MAPI because I think it's the most logical place in the current framework as it offers some process related features but also because it implies minimal changes in Cobra/Mamba & no changes at all for existing projects.

But then again you can choose to use a separate payload (loadable at runtime using Cobra/Mamba) & a separate syscall altogether but keep in mind that projects like Cobra, Mamba, CFW Tools, wMM, PSNPatch etc.. may need to be updated if you do.

 

[StarmanX32]

I only suggested PS3MAPI because I think it's the most logical place in the current framework as it offers some process related features but also because it implies minimal changes in Cobra/Mamba & no changes at all for existing projects.

But then again you can choose to use a separate payload (loadable at runtime using Cobra/Mamba) & a separate syscall altogether but keep in mind that projects like Cobra, Mamba, CFW Tools, wMM, PSNPatch etc.. may need to be updated if you do.

No no, I meant making it a part of Cobra but with its own opcode rather than being a part of PS3MAPI, beacuse of the differences of the bridge and PS3MAPI. There are still minimal changes to Cobra and would probably not require any other project to be updated for this (well except they choose to use vsh functions then.) But making it a part of PS3MAPI isn't a problem for me either (probably not an issue for RouLette either, gonna ask him)


Gesendet von iPhone mit Tapatalk

 

[StarmanX32]

@bguerville We're gonna migrate the code to Cobra in the PS3MAPI section. I'll get back to you guys once we've migrated the code base to Cobra and tested the code.