Re: curl: (18) transfer closed with outstanding read data remaining

digb150 · ‎02-05-2022

I am trying to download a 6gig zip file over a connection speed of 1Mb/s, which is pretty slow, but from what I have been doing has been consistent. When I get to about 4.1Gb downloaded the dropbox server disconnects and I get the following message "curl: (18) transfer closed with outstanding read data remaining". I have tried a firefox, chrome and IE to see if I get different results but it is always coming out the same. How can I get this zip file? I cannot speed up my connection. I have noticed that each download attempt will disconnect at 1 hour of downloading. Please let me know what I need to do to increase the timeout on the dropBox server end so I can get the zip file locally.

Thanks

XionicFire · ‎02-06-2022

Anything over 2gb on a web connection will probably fail, this is not dropbox its probably your ISP, TCP sockets are not rock solid, they will fail, its just a matter of when, when, the more time you spend on a task the more and more likely that % chance of fail increases.

The only way to get around this is by downloading your file in chunks, sadly using the web browser applet will not cut it, you will need to download the dropbox app for that, it downloads files in 4mb chunks, and if one fails it can resume, the web app does not have this functionality.

Either that or rent an amazon web server somewhere, log in via remote desktop, download the file there, use winrar to chop it up into 6 files of 1gb each, then reupload to dropbox, then on your current pc download the 1gb files one at a time, if one fails redownload, once there tell winrar to extract the file from the multi file parts and it will remake the single 6gb file.

Other than those 2 options i just dont see how else you would be able to do it.

I hope this helps you a little.

Здравко · ‎02-06-2022

@digb150 wrote:
... I get the following message "curl: (18) transfer closed with outstanding read data remaining". ...

Hi @digb150,

Seems you are using curl to download your file. Curl provides option to continue download because of broken connection, for example. I don't know what command options you are using, but if you have something like:

curl -C - -o <your file name> -L https://www.dropbox.com/s/123456asdfghjthghy/<your file name>?dl=1

... then you would be able download your file with resuming download using the same command as many times as need. 😉

Hope this helps.