PS3 [RESEARCH] Downloading all unknown file types to HDD and Flash

[DeViL303]

UT... the files end up in dev_hdd0/vsh/task/0000000x and it keep changing.

So, can i use your auto exec script function, to do this on boot up , for example:

scan dev_hdd0/vsh/task/ and ALL subfolders for *.pkg and move them all to dev_hdd0/packages
scan dev_hdd0/vsh/task/ and ALL subfolders for *.p3t and move them to dev_hdd0/theme
scan dev_hdd0/vsh/task/ and ALL subfolders for *.ISO and move them to dev_hdd0/PS3ISO
scan dev_hdd0/vsh/task/ and ALL subfolders for *.PUP and move them to dev_hdd0/updater/01
scan dev_hdd0/vsh/task/ and ALL subfolders for *.sprx and move them to dev_hdd0/plugin
scan dev_hdd0/vsh/task/ and ALL subfolders for *.zip and move them to dev_hdd0/tmp/downloader/

There is an issue with this method, if the file is only partially downloaded it will still get moved. So i think this needs to be a manual called function? only used when needed to fix file locations? What do you think @aldostools

 

[Lambada]

Hmm maybe if its not that complicated:
If the file isn't complete yet it could be saved in an incomplete folder than when it's complete it would move to it's correct path. Second alternative: the extension of the file could be for example nameOfFile.pkg.incomplete and when its complete renamed to nameOfFile.pkg, than moved to it's correct path.


Sent from my Redmi Note 5 using Tapatalk

 

[lmn7]

Well, I was able to find videodownloader_plugin.sprx from 4.55 PUP. I'm not sure what I was expecting, but it wasn't that.

I triggered it from Silk, it is a plugin for viewing video files I guess? When you visit an MP4 URL in the Silk browser, it buffers the video and opens it in a full screen where you have options to play, rewind, pause, etc. just like from the XMB.

I think this plugin is actually useful and I have no idea why they removed it. It works surprisingly well for a browser plugin.

 

[DeViL303]

Well, I was able to find videodownloader_plugin.sprx from 4.55 PUP. I'm not sure what I was expecting, but it wasn't that.

I triggered it from Silk, it is a plugin for viewing video files I guess? When you visit an MP4 URL in the Silk browser, it buffers the video and opens it in a full screen where you have options to play, rewind, pause, etc. just like from the XMB.

I think this plugin is actually useful and I have no idea why they removed it. It works surprisingly well for a browser plugin.

Cool, yeah, i think we can do that now, somehow, if mp4 is right format, it will play direct instead of downloading. they must have moved the feature into some other plugin or something, as i think this was like 4.78 that i tested that. I was thinking to have tutorial videos linked on the XMB, idea never got off the ground.

Did you use the 4.55 sprx on OFW without resigning? or was it CFW ? Just curious.

 

[lmn7]

Cool, yeah, i think we can do that now, somehow, if mp4 is right format, it will play direct instead of downloading. they must have moved the feature into some other plugin or something, as i think this was like 4.78 that i tested that. I was thinking to have tutorial videos linked on the XMB, idea never got off the ground.

Did you use the 4.55 sprx on OFW without resigning? or was it CFW ? Just curious.

I used the original 4.55 file without resigning on OFW. We can use certain files from previous firmware versions just fine, that's what we're doing with webkit on HFW, but if you go back too far it won't work. Some files are obviously off limits like vsh.self and such.

 

[DeViL303]

I think we can exploit this extra download plgin slot on CFW, we can have 2 download plugins, patched differently, one renamed to videodownloader_plugin, then we can call then differently based on the mimetype xml, i must look into that more. as it would double up our file types supported.

 

[lmn7]

I think we can exploit this extra download plgin slot on CFW, we can have 2 download plugins, patched differently, one renamed to videodownloader_plugin, then we can call then differently based on the mimetype xml, i must look into that more. as it would double up our file types supported.

Yeah, you can pretty much have unlimited plugins if you do it this way. The problem is that the mimetypes are the same for a lot of content, so how are you going to trigger different plugins?

 

[DeViL303]

Yeah, you can pretty much have unlimited plugins if you do it this way. The problem is that the mimetypes are the same for a lot of content, so how are you going to trigger different plugins?

well if i can trigger even 2 its a massive help, we dont need to add many file types really. I dont think we can add unlimited plugins, as it needs to be supported by the firmware. I suppose we could remove other plugins and use their slots, but just adding randomly named plugins will not work.

 

[lmn7]

well if i can trigger even 2 its a massive help, we dont need to add many file types really. I dont think we can add unlimited plugins, as it needs to be supported by the firmware. I suppose we could remove other plugins and use their slots, but just adding randomly named plugins will not work.

Yeah that's true, but what I mean is replacing dummy plugins or ones that aren't really used like strviewer for example. But yeah using that method should cover most important file types.

 

[aldostools]

There is an issue with this method, if the file is only partially downloaded it will still get moved. So i think this needs to be a manual called function? only used when needed to fix file locations? What do you think @aldostools

If the partial download can be deleted before reboot, I think it wouldn't be a big issue... unless the user don't pay attention to what he/she is doing.

Here is a test build of webMAN MOD full edition that implements the wildcards and recursive scan.
https://www.sendspace.com/file/p1drcm

The syntax of the command is like this example:
move /dev_hdd0/downloads/*.pkg,/dev_hdd0/packages
move /dev_hdd0/downloads/*.iso,/dev_hdd0/PS3ISO
del /dev_hdd0/downloads/*.tmp
copy /dev_hdd0/downloads/*.sprx,/dev_hdd0/plugins
list /dev_hdd0/tmp/wmtmp/*.ntfs[,/dev_hdd0/myisos.txt

The wildcards with recursive scan is also supported in /del.ps3 and /copy.ps3 web commands

And now you can find files using /dev_hdd/tmp/*.TXT (it will create the file: /dev_hdd0/tmp/wmtmp/filelist.txt)

NOTE: The wildcard does not accept prefixes like /LANG*.TXT or multiple *.
However you can scan like this: /dev_hdd/tmp/*LANG

Before consider your idea of an incremental variable, let's explore a script like this (it's just the idea, I haven't tested if it works)

Spoiler

if exist /dev_hdd0/tmp/coldboot01.now
ren /dev_hdd0/tmp/coldboot01.now=/dev_hdd0/tmp/coldboot01.raf
copy /dev_hdd0/tmp/coldboot02.raf=/dev_blind/vsh/resource/coldboot.raf
ren /dev_hdd0/tmp/coldboot02.raf=/dev_hdd0/tmp/coldboot02.now
elseif exist /dev_hdd0/tmp/coldboot02.now
ren /dev_hdd0/tmp/coldboot02.now=/dev_hdd0/tmp/coldboot02.raf
copy /dev_hdd0/tmp/coldboot03.raf=/dev_blind/vsh/resource/coldboot.raf
ren /dev_hdd0/tmp/coldboot03.raf=/dev_hdd0/tmp/coldboot03.now
else
ren /dev_hdd0/tmp/coldboot03.now=/dev_hdd0/tmp/coldboot03.raf
copy /dev_hdd0/tmp/coldboot01.raf=/dev_blind/vsh/resource/coldboot.raf
ren /dev_hdd0/tmp/coldboot01.raf=/dev_hdd0/tmp/coldboot01.now
end if

 

[DeViL303]

If the partial download can be deleted before reboot, I think it wouldn't be a big issue... unless the user don't pay attention to what he/she is doing.

Here is a test build of webMAN MOD full edition that implements the wildcards and recursive scan.
https://www.sendspace.com/file/p1drcm

The syntax of the command is like this example:
move /dev_hdd0/downloads/*.pkg,/dev_hdd0/packages
move /dev_hdd0/downloads/*.iso,/dev_hdd0/PS3ISO
del /dev_hdd0/downloads/*.tmp
copy /dev_hdd0/downloads/*.sprx,/dev_hdd0/plugins
list /dev_hdd0/tmp/wmtmp/*.ntfs[,/dev_hdd0/myisos.txt

Nice, im going to have a play with this now.

The problem is, the user will not be able to reboot when a file is downloading in the background, i suppose that is ok. But we could remove that limitation by not using a auto running script, and instead having a button for doing it manually, or we have a option toggle for the auto running script. Its just a shame to lose the ability to reboot due to adding this.

On other note, i have solved sbk, sb2, ptf and PBP downloading to other paths, partially. It was not downloading them at all, now:

So Im making progress all the time, Kaizen as the Japanese say.

 

[aldostools]

Nice, im going to have a play with this now.

The problem is, the user will not be able to reboot when a file is downloading in the background, i suppose that is ok. But we could remove that limitation by not using a auto running script, and instead having a button for doing it manually, or we have a option toggle for the auto running script. Its just a shame to lose the ability to reboot due to adding this.

Although the batch script can be played at startup using "autoexec.bat" (I hope MS don't demand me LOL), they can be played from an EBOOT.BIN using a custom wm_url_launcher or through a combo. Example: /play.ps3/dev_hdd0/tmp/wm_scripts/move_downloads.bat

 

[DeViL303]

Although the batch script can be played at startup using "autoexec.bat" (I hope MS don't demand me LOL), they can be played from an EBOOT.BIN using a custom wm_url_launcher or through a combo. Example: /play.ps3/dev_hdd0/tmp/wm_scripts/move_downloads.bat

Oh , brilliant. That solves that then. Is there already a way to have optional autorun scripts, chain them together, even just link to 1 level of others?

For example, i put path to clear_whats_new_cache.bat, in the autoexec.bat? And if that file exists then it get executed, if file does not exist then nothing happens, no error. this way i can toggle the name of clear_whats_new_cache.bat , to add auto running features that are optional.

 

[aldostools]

Oh , brilliant. That solves that then. Is there already a way to have optional autorun scripts, chain them together, even just link to 1 level of others?

For example, i put path to clear_whats_new_cache.bat, in the autoexec.bat? And if that file exists then it get executed, if file does not exist then nothing happens, no error. this way i can toggle the name of clear_whats_new_cache.bat , to add auto running features that are optional.

In theory you can call a script from another script. I haven't tested that feature though. Not sure if the memory stack currently assigned is enough to do that multiple levels.

EDIT:
Keep in mind that the "batch script" is a very recent feature, and it still lacks many important functions of a real script language.

If calling a script from another doesn't work, you can paste the code of the sub-scripts into the main script.
Only don't exceed the 999 lines or 64KB in size.

 

[DeViL303]

Making nice progress. More work to do.

 

[Kazama]

I have realized something, Haxxens orignal download list xml, was actually working all along.

@lmn7 try this on OFW. I think it will work fine. You have to background download, or it wont work

Put that in dev_flash.vsh/resource/explore/xmb/.

The downloads will get to 99%, but its really 100%.

I have tried this method placing xml file. When i download a .pkg file i'm unable to download. It says unknown error occurred (80010006) I'm on ps3 HFW 4.84

When I download the package file which is rename to .PNG , it downloads with background download. After download finished the file goes to pictures folder. It shows corrupted data and I cant able to install using package installer. Plz tell me.

 

[DeViL303]

OK, So I need help with this mod, its so close, that i think if we put out heads together we can solve it in a few hours most likely.

@aldostools @ lmn7 @kozarovv @Joonie @habib @anyone who might be able to help.

So I have about 3 different methods on the go now, they all have their plus points, I need help deciding which path is best to go down. Maybe we need to use multiple methods to achieve everything.

Its a little complicated, as the method all have limitations, so I am going to try explain issues.


Method 1: One file type at a time method

In this method we patch download_plugin.sprx, we just change "p3t" to our preferred file type here:

That is enough to download to the dev_hdd0/theme folder, but we can also change the path "theme" to our preferred path on dev_hdd0 here:

it is possible to go up a level with ../ in the sprx, and then access other partitions, but we are limited on space as its hex, but I think we can go longer and wipe out the next entry as its only for mnv movies which are not popular and could be patched out temporarily while downloading another filetype, and switched back after, or we can download pkg files to dev_hdd0/package (note the missing "s") , this will fit into the space we have available so is not a bad patch.


Method 1 Good points:

• Easy to do
• It allows any file type
• It allows lots of paths depending on depth.
• The download completes fully
• The file is available after download.

Method 1 Bad points:

• Only 1 file type allowed per patch
• Only 1 path allowed per patch
• Requires sprx patch so makes OFW patch difficult
• Long paths on other partitions might be an issue

Method 2: Adding paths individually method

I really like this mod, i think if we can get this to work 100% it is the way to go, In this mod, I add all my paths to the download_list.xml.

So it looks like this:

I have downloads fully working directly to dev_blind, so this method could be used to install the HAN support files directly to flash lol. Also @lmn7 @ShaolinAssassin This will work to simplify all the coldboot/wave/font installers, they can be downloaded directly to correct location, and it will work on OFW.

So this is the method I want to perfect. This is 100% xml mod only.

So i have attached a download_list.xml , please try it out. I have also attached a bookmarks xml, this bookmarks xml links to lots of sample file types and makes testing quicker:

Here is the error that is killing hdd downloads when they get to 100%

<Text name="msg_error_download_failure">An error occurred during the download operation.</Text>


Bonus interesting discovery, well maybe everyone knew, but i did not. We can pass a null value in xml, and it works.

So this patch, stops the sprx from adding "/tmp/downloader/" to this path, the nul kills the entry dead, and give me access to root of HDD.


I have another method too, which i will explain later.


I need to know more about remapping paths for that one, symlinks etc, but i need it explained in noob speak. Links with loads of C code is no good for me, but i will be able to get the concepts, just not the lingo.

 

[DeViL303]

With method 2, we have background downloading, but it has issues. Instead of file being deleted after install, it just never completes on XMB, says 99% but is really 100%. Also file then ends up in vsh/task/0000000x/ and stays there, So its better in one way, as file is complete and still on HDD, but its worse, as file is in random folder, and not where we want it to go

Method 2 good points:

• Works to any path
• any file type
• Will work on OFW as its only an xml mod
• can choose path individually
• works 100% to dev_blind and dev_flash2 already
• allows background downloading (issues)
• can add unlimited paths
• Can restrict other paths by not adding them
• Its 99.9% perfect.

Method 2 bad points:

• File is being deleted
• background download uses random folder
• must add hard coded paths

 

[ShaolinAssassin]

That might sounds a stupid question but well : are you sure that device_path means the complete location where the file will be downloaded (full path) ? Because if we look at the original file, music, photos and videos has all the same "device_path" (/dev_hdd0). Could it just mean the device on which the file will be downloaded ? (BTW, this could be related : what does "logical_sceme" means ?)

 

[lmn7]

That might sounds a stupid question but well : are you sure that device_path means the complete location where the file will be downloaded (full path) ? Because if we look at the original file, music, photos and videos has all the same "device_path" (/dev_hdd0). Could it just mean the device on which the file will be downloaded ? (BTW, this could be related : what does "logical_sceme" means ?)

Yeah, it's a weird thing. It does work for downloading files to dev_flash, and I assume it's working on hdd0 as well, but it errors out and deletes the file after the download has finished.

I tried many things to get around that, but I don't think it's possible without modding some SPRX file.