Error 500 finishing huge file upload (300GiB+)

[pp111]

Hello,

When a huge file (300GiB+) is uploaded using the upload_session API an "Error 500 Internal Server Error" is received when the last part is uploaded (/upload_session/append with close set).

I retry the request but an Error 500 raises again and again. Sometimes, after more than 1 hour retrying the file uploads successfully.

I am using session_type sequential. The same code for smaller files (10GiB) works perfectly.

Are you aware of the problem? Is there any workaround?

[Здравко]

Hi @pp111,

Up border of acceptable file size is 350GiB. If your file doesn't cross this border, then maybe this is some glitch of new bug. It may be useful for support people to have multiple x-dropbox-request-id from your failing requests for debugging. Perform some more such requests and post here their id values.

---

@pp111 wrote:

... received when the last part is uploaded (/upload_session/append with close set).

...

---

If you don't upload files in batch, you don't need to close your sessions during last append. The session will be closed automatically during finalization if not yet. You may try use this way as a workaround.

Good luck.

[pp111]

Thank you.

I confirm that my file is smaller than 350GiB, actually it is exactly 311GiB. And, of course, I have enough free space.

I tried what you suggested and, instead of sending the last part with close set, I directly call upload_session/finish. In this case the last part is uploaded correctly but I get the Error 500 in the upload_session/finish call. I retry the finish call many times, but I get the Error 500 during at least 1 hour after the last part was uploaded.

These are a few x-dropbox-request-id of the failed upload_session/finish calls

14431cf78d2b468f98744e327c85a2b0
80191115a98b49958eea9826d4980f2b
ee6951f1f6b2408b857cd877fb725e6b
30a62bca19c340fa9c5de112a3efd57b
f6b89a0be6694b76b85d2f23ca7c7eed
dffd65f0321a489b809cb650607ca382
a287f969781340adac308b15a53706af
afe05e3d42d744e784908cffd8f042f8
703b6a529fec4beda1f11f4335e2dcca
04a8082ee15d4efa81db292f0cc6dd98
3548b4f5a10d4a8e90f15fbabfd34372
eb2d038ab2bf4c1192606d4f87aebf67
e438955fde704ee79385c500429dee87

Thanks

[Greg-DB]

@pp111 Thanks for the samples! We'll look into it and follow up here once we have any updates on this for you.

[pp111]

Hello,

The issue is still not fixed and it is still not possible or reliable. We keep retrying the upload_session/finish call during hours, but the upload does not finish and we keep getting the error 500.

Do you have new information or a workaround we could use?

Thanks

[Greg-DB]

This is open with engineering, but I don't have an update on it yet. I'll check in with them and follow up here once I have any news.