PS2 HDLGameInstaller

[jolek]

NNNNNNNNOOOOOOOOOOOOOOOO!!!

Today I've tried HDLGameInstaller v0.817 09/09/2018 on Windows XP.
It sometimes takes 3-4 attempts to get connected with PS2, also when I "hit" 99%:

I haven't got these problem on Win 10.

When I've more time I'll check the drivers versions.
Also I'll try 0817A.
Maybe those in Win XP are in some way outdated?

 

[sp193]

HDLGameInstaller v0.818 released!

Changelog:

• Fixed hang at boot and when settings are changed, if Ethernet link settings were set.
• Fixed saving of settings, when booting from the HDD unit.
• Reading & writing to the HDD will be grouped, to improve performance.
• Moved updating of game list to when the connection is closed/lost, to improve on waiting times between installations.
• Fixed crash when the game list is updated after the connection is closed, due to the menu trying to access it.
• Compiled with PS2SDK updates: new NETMAN module for better reliability. New USBHDFSD module for proper access of USB devices.
• Fixed incorrect logic for reading game list at boot, shortening bootup.

Changelog for the PC client

• Fixed order of controls for game update form.
• Success message will no longer be displayed if the user aborts a game installation.

Downloads/links:
HDLGameInstaller project page: - HDLGameInstaller support page -

Maybe those in Win XP are in some way outdated?

Maybe. But only Realtek knows.

 

[jolek]

Well, with Win XP I'm still having problems connecting with the console:

With Win 10...
Is there any way to test Windows Firewall?
I mean to add permission before the transfer begins, without having to do it manually in Windows Firewall setting?

Sometimes when I add permission, or forget to select private network, the transfer will not begin,
it'll stuck at 0% and after some time the error pops out "The connection to the console has been lost. Please reconnect".

 

[sp193]

Well, with Win XP I'm still having problems connecting with the console:


With Win 10...
Is there any way to test Windows Firewall?
I mean to add permission before the transfer begins, without having to do it manually in Windows Firewall setting?

Sometimes when I add permission, or forget to select private network, the transfer will not begin,
it'll stuck at 0% and after some time the error pops out "The connection to the console has been lost. Please reconnect".

You can try to temporarily disable Windows firewall, for the public and private zones. If it can connect, then it was a firewall problem.
While a lack of flow control can cause the connection to break down, it often still manages to get established and you will observe some progress. So a total failure to begin any transfers at all, is an indicator of a blocked connection.

 

[jolek]

I know that this is probably a Windows firewall "feature", but I wonder can you add some test button\option
to prepare firewall to add exception before transferring a game?
E.g while taking list of games from PS2?

 

[jolek]

Feature request for PC client:

• Can PC client check if a game is on HDD before installing (while adding it on the job list)?
  
  
  
  Currently after adding game to the list there is no warning.
  Overwrite process "pops" when the game will be installed:
  
  
  .
  It may help, if someone wants to install few games, but he doesn't remember if the game is on PS2.
  I mean, I added 10 games to the list, leave the PC for a while or even more,
  when I return (e.g. after 30 min) overwrite process "pops" and whole process will stop\pause.
  
  
• Can the PC client shows currently installed game, not only overall process?
  It can be helpful is someone is installing few games and something goes wrong (game will not be successful installed).
  User will know with what game he had problem.
  
  
• Alphabetically sort games titles?
  It may helps with finding game on PS2 HDD.
  Currently games are sorted by date of add.

Anyway, it's only a "simple" feature request. I'm not demanding anything.
If these features somehow can be added then... I or maybe someone else will be happy.
If not, well it's a great app even without these additional features.

 

[jolek]

@sp193, hare is my problem when I want to connect PC (Win XP) with PS2 (HDL 0.819):
http://www.mediafire.com/file/sj1ww7ffw3eik72/HDL_181118.7z.
Unfortunately quality is extremely low, but it has "happy end".

Spoiler

Tomorrow I'll try to upload Wireshark packet capture via PM.

 

[sp193]

HDLGameInstaller v0.820 released!

Changelog for v0.820 (as of 2018/11/28):

• PS2SDK update: fixed timeouts causing an overflow of the SetAlarm tick parameter.
• Reordered code for disconnecting clients, to prevent the possibility of disconnecting a new client that reuses the same client slot.
• (NEW 2018/11/25): Fixed the problem with semaphores not being freed when the client disconnects.
• (NEW 2018/11/25): Fixed the problem with PlayStation discs being accepted as valid discs.
• (NEW 2018/11/25): Fixed the problem with the error message for unsupported discs not getting displayed.
• (NEW 2018/11/28): Fixed incorrect logic for handling disc types.

Changelog for the PC client

• Attempted to fix the problem with compatibility settings not binding at installation time.
• Added timeout for the data connection, to prevent the software from getting stuck if the connection gets blocked.

Downloads/Links
HDLGameInstaller project page (downloads at bottom): https://sites.google.com/view/ysai187/home/projects/hdlgameinstaller

psx-place resource thread: http://www.psx-place.com/threads/hdlgameinstaller.20787

 

[jolek]

On my side currently everything is fine with Win XP.
I mean, I can install multiple games without an error,
even the transfer will resume after I "unblock" firewall when I run HDL for the first time.

With Win 10 I have only a problem with firewall, when i run HDL for the first time.
I mean, when the firewall will ask :
Allow HDL to communicate on these network:

• Private networks...
• Public networks...

When I waited few seconds to add permission:
"The connection to the console has been lost..."
Also my PS2 will freeze at:
"A computer has been connected...", so I'll need to reboot it.
This issue, is absent on Win XP.

Maybe adding an option to add exception or test firewall will be a good idea?

Spoiler: low irrelevant issues

When I insert CD-Audio or unsupported disk and I "hit" Start to Install game I'll get:

Please wait
Reading disk...

Nothing will be happening, however when I eject a disk

Error
Please insert a disk

Although, HDL will resume "its work".

With PS1 game I'll get:

Error
An error occurred while reading the disk.

After I press OK, HDL will resume "its work".

Maybe add warning that these media types are not supported?
Although, who will be willing to install CD-Audio?!

For me it's the best version so far, even the transfer is very decent.
Thanks.

 

[sp193]

Thanks for your help again.

Regarding the previous problems with the software seemingly not being so compatible with Windows XP, it was caused by your PC taking considerably long (1~2 seconds) to send the client version number. Although the timeout programmed into the PS2 server is 30s, 30s caused the tick counter to overflow. That in turn caused the timeout to only last a few seconds instead.

On my side currently everything is fine with Win XP.
I mean, I can install multiple games without an error,
even the transfer will resume after I "unblock" firewall when I run HDL for the first time.

With Win 10 I have only a problem with firewall, when i run HDL for the first time.
I mean, when the firewall will ask :
Allow HDL to communicate on these network:

• Private networks...
• Public networks...

When I waited few seconds to add permission:
"The connection to the console has been lost..."
Also my PS2 will freeze at:
"A computer has been connected...", so I'll need to reboot it.
This issue, is absent on Win XP.

There is no specialized code for this. Did you wait a while, like for 30s?
It may take some time for connect() to time out.

Maybe adding an option to add exception or test firewall will be a good idea?

AFAIK, there is no way to programmatically add an exception. There is no standard API for this, so it isn't possible to make it compatible with the products of the various vendors out there.

It's for reasons like this one that passive FTP exists, as that the client can initiate the data connection and firewalls have less reasons to block it. But I currently made HDLGameInstaller to work somewhat like active FTP, whereby the server initiates the data connection with the client. I didn't want to come up with more new mechanisms for this, due to a lack of skills to be sure that it'll work well and time.

When I insert CD-Audio or unsupported disk and I "hit" Start to Install game I'll get:

Nothing will be happening, however when I eject a disk

Although, HDL will resume "its work".

There is existing code for indicating the presence of unsupported discs. I cannot see why it doesn't work.
But Sony did document that the PS2 can hang if unsupported discs are inserted. They're likely referring to the SCPH-50000 and above.

With PS1 game I'll get:

After I press OK, HDL will resume "its work".

There was a typo, causing PS discs to be accepted. I have corrected this. Thank you!

For me it's the best version so far, even the transfer is very decent.
Thanks.

Thank you too!

 

[jolek]

There is no specialized code for this. Did you wait a while, like for 30s?
It may take some time for connect() to time out.

I've made an another test, this time with version from 25\11\2018.
When I'll wait longer than 30 sec in Win XP to add exception for firewall I'll get:

However after l losing connection I can reconnect without a problem and now the game can be installed.

When I'll wait longer that ~15\20 sec (I do not know exactly how much) transfer will not resume:

Although after waiting additional ~30 sec, I can re-install the game.

When I'll add exception pretty fast, there won't be any problems.

Hover I don't remember how long with previously version I waited to add exception (maybe 10 sec?!).

With Win 10 currently it's the same thing.
When I'll wait longer than 30 sec in Win 10 to add exception for firewall I'll get:

However after l losing connection I can reconnect without a problem and now the game can be installed.

When I'll wait longer that ~15\20 sec (I do not know exactly how much) transfer will not resume:

Although after waiting addidtional ~30 sec, I can re-install the game.

When I'll add exception pretty fast, there won't be any problems.

AFAIK, there is no way to programmatically add an exception. There is no standard API for this, so it isn't possible to make it compatible with the products of the various vendors out there.

It's for reasons like this one that passive FTP exists, as that the client can initiate the data connection and firewalls have less reasons to block it. But I currently made HDLGameInstaller to work somewhat like active FTP, whereby the server initiates the data connection with the client. I didn't want to come up with more new mechanisms for this, due to a lack of skills to be sure that it'll work well and time.

There might be 2 ways to bypass this problem:

1. Add warning and tutorial how to add firewall exception.
   Here is a tutorial for Win 7, 8 (with images):
   https://www.sevenforums.com/tutorials/542-windows-firewall-add-remove-exception.html.
   This method should also work with Win 10, but not with XP.
   
   Here is a tutorial for all versions (without images):
   http://lexisnexis.custhelp.com/app/...1/~/adding-exceptions-to-the-windows-firewall.
   
   
2. I remember that while installing new software, exception can be added at the end of an installer.
   But this method will required an installer and don't know if it's the right way.

There is existing code for indicating the presence of unsupported discs. I cannot see why it doesn't work.
But Sony did document that the PS2 can hang if unsupported discs are inserted. They're likely referring to the SCPH-50000 and above.

There was a typo, causing PS discs to be accepted. I have corrected this.

Now (with 25\11\2018) no matter what unsupported media (PC game, CD-Audio, PS1 game) I'll put, I'll get:

Please wait
Reading disk...

Why not:

Unsupported media
Please insert PS2 game.

At least for PS1 games.

Spoiler: Added warnings in the resource

Warning:

1. This app required Microsoft Visual C++ 2010 Redistributable Package (x86) to launch PC client.
   
   
2. Before you'll launch PC client for the fist time, it is highly advised to
   Add Program Exceptions in the Windows Firewall, because "unblocking" exception during transfer might cause connection lost.
   
   Here is a tutorial, how to do it (with images), for Win 7, 8, 10:
   https://www.sevenforums.com/tutorials/542-windows-firewall-add-remove-exception.html.
   For all Windows systems (without images):
   https://lexisnexis.custhelp.com/app...1/~/adding-exceptions-to-the-windows-firewall.
   When you'll not add Program Exceptions this error will be shown:
   
   
   

 

[sp193]

I've made an another test, this time with version from 25\11\2018.
When I'll wait longer than 30 sec in Win XP to add exception for firewall I'll get:

However after l losing connection I can reconnect without a problem and now the game can be installed.

When I'll wait longer that ~15\20 sec (I do not know exactly how much) transfer will not resume:

Although after waiting additional ~30 sec, I can re-install the game.

When I'll add exception pretty fast, there won't be any problems.

Hover I don't remember how long with previously version I waited to add exception (maybe 10 sec?!).

With Win 10 currently it's the same thing.
When I'll wait longer than 30 sec in Win 10 to add exception for firewall I'll get:

However after l losing connection I can reconnect without a problem and now the game can be installed.

When I'll wait longer that ~15\20 sec (I do not know exactly how much) transfer will not resume:

Although after waiting addidtional ~30 sec, I can re-install the game.

When I'll add exception pretty fast, there won't be any problems.

So in other words, as long as you are patient, it will be fine?
I have no control over how long the OS will really wait when the connection is blocked. I do know that it might not time out in 30s, although my application-level timeout is 30s...

There might be 2 ways to bypass this problem:

1. Add warning and tutorial how to add firewall exception.
   Here is a tutorial for Win 7, 8 (with images):
   https://www.sevenforums.com/tutorials/542-windows-firewall-add-remove-exception.html.
   This method should also work with Win 10, but not with XP.
   
   Here is a tutorial for all versions (without images):
   http://lexisnexis.custhelp.com/app/...1/~/adding-exceptions-to-the-windows-firewall.
   
   
2. I remember that while installing new software, exception can be added at the end of an installer.
   But this method will required an installer and don't know if it's the right way.

Thanks for the info.

Now (with 25\11\2018) no matter what unsupported media (PC game, CD-Audio, PS1 game) I'll put, I'll get:


Why not:


At least for PS1 games.

It's a bug, which I have just corrected. Please use the new file.

 

[jolek]

So in other words, as long as you are patient, it will be fine?
I have no control over how long the OS will really wait when the connection is blocked. I do know that it might not time out in 30s, although my application-level timeout is 30s...

Not, patient. You have to be quick or add firewall exception before PC client will transfer the game for the first time.

It's a bug, which I have just corrected. Please use the new file.

Thanks, I'll try to test it.

 

[jolek]

It's a bug, which I have just corrected. Please use the new file.

I've tried new build with PS2 DVD Game, PS2 CD Game, PC game, CD-Audio, PS1 Game.
All of these disks when I "hit" Start to Install game shows:

Error
Unsupported disc inserted.
Please insert a Playstation 2 game disc.

 

[Grahf]

Hello, @sp193!

Per your recommendation, I registered here.

I'll check out HDLGameInstaller v0.820 since it seems like you've made a great deal of improvements recently, even though the previous version already worked (nearly) flawless for me.

BTW, if you have some free time to do it, can you please handle that issue with HDDChecker where it boots from an HDD, but gets stuck on the "Now Loading..." screen with most functions?

 

[jolek]

It's a bug, which I have just corrected. Please use the new file.

With version from 28\11\2018 everything should be fine with PS2 DVD Game & PS2 CD Game.
I mean, I can install PS2 game from CD\DVD disc.

When I insert PC game, CD-Audio, PS1 Game, I'm getting:

Error
Unsupported disc inserted.
Please insert a Playstation 2 game disc.

So, now It is working as it should.

Thanks.

However, I still have a problem with additional "!", when I'm editing Full title, OSD title line 1, OSD title 2:
http://www.psx-place.com/threads/hdlgameinstaller.3709/page-3#post-129721.

EDIT: It also happens ("!") when I want to edit installed game (Game options).

Maybe it is some kind of a double click, because "!" it is the first symbol on the virtual keyboard?

 

[sp193]

HDLGameInstaller v0.821 released!

Changelog for v0.821 (as of 2018/12/09):

• Disabled pad input when a PC client is connected, to improve performance.
• Network status will be displayed when a PC client is connected, every 10s.
• NEW 2018/12/09 - Fixed the problem with the soft keyboard not waiting for the circle/cross button to be released before considering the user's intended input.

Changelog for the PC client

• Increased data connection timeout to 80s, to allow the server's connection attempts to be exhausted before the PC client indicates an error.
• Added a prompt to delete the game if it was already installed, when the user adds a game to install.

Downloads/Links
HDLGameInstaller project page (downloads at bottom): https://sites.google.com/view/ysai187/home/projects/hdlgameinstaller
psx-place resource thread: http://www.psx-place.com/threads/hdlgameinstaller.20787

I think this may allow us to get closer to the maximum possible speed of 8.6MB/s. Usually, I get 5.7MB/s due to the network here, but I could reach speeds like 6.9MB/s with this.

I got this idea of disabling pad input from what Jimmikaelkael once commented in the OPL HDLDump server: performance of the server in OPL seemed worse than when it is a standalone product. Since HDLGameInstaller does not actually do much with the pad when the PC is connected, this was not too difficult to do.
wisi first noted that the performance of the SIF was being worsened by the SIF completing its transfer before the IOP can do anything else. And it did get better once before, when I grouped incoming frames together.

Not, patient. You have to be quick or add firewall exception before PC client will transfer the game for the first time.

This is the normal behaviour of TCP, although I cut it short by just closing the connection on the PC side after 30s. So to make things more consistent, I increased this timeout to 80s (since TCP implementations like the ones in Linux and Windows will usually spend something like 75s to connect).

Per your recommendation, I registered here.

Thank you. And, welcome!

I'll check out HDLGameInstaller v0.820 since it seems like you've made a great deal of improvements recently, even though the previous version already worked (nearly) flawless for me.

Thanks.

BTW, if you have some free time to do it, can you please handle that issue with HDDChecker where it boots from an HDD, but gets stuck on the "Now Loading..." screen with most functions?

It should be fine now.

With version from 28\11\2018 everything should be fine with PS2 DVD Game & PS2 CD Game.
I mean, I can install PS2 game from CD\DVD disc.

When I insert PC game, CD-Audio, PS1 Game, I'm getting:


So, now It is working as it should.

Thanks.

I see, thank you.

However, I still have a problem with additional "!", when I'm editing Full title, OSD title line 1, OSD title 2:
http://www.psx-place.com/threads/hdlgameinstaller.3709/page-3#post-129721.

EDIT: It also happens ("!") when I want to edit installed game (Game options).

Thanks for the reminder.

You mentioned this before, but it was not fixed for one reason or anything. I might not have been able to replicate it.
But as it already cost me a lot to re-engineer the UI to support variable-width fonts properly, I think I will want to just leave it like that for at least now.

 

[Grahf]

It should be fine now.

Fixed deadlock that occurred when the logger tries to create a log file on the HDD unit, when HDDChecker is booted from the HDD unit (this is not supported).

Do you mean that creating a log file on an HDD is not supported or that booting from an HDD is not supported?

 

[sp193]

It (HDDChecker) supports getting booted from the HDD unit. It will, however, not generate any log files if you do that.

Unfortunately, that statement by me just was not so clear.

 

[Grahf]

It (HDDChecker) supports getting booted from the HDD unit. It will, however, not generate any log files if you do that.

Unfortunately, that statement by me just was not so clear.

OK, thanks. That makes perfect sense. It needs full access to an HDD when it's doing those tasks, so it can't simultaneously create files on it. I don't really care about log files. I just want to monitor S.M.A.R.T., in case something starts to fail. Occasional disk scans won't hurt either.