We're making changes to the Community, so you may have received some notifications - thanks for your patience and welcome back. Learn more here.

Forum Discussion

newdbuser's avatar
newdbuser
Explorer | Level 3
8 years ago

Uploading large files using chunks

Hello: I am using this Python code to upload a large zip file (2-4 GB). I am trying to upload the file in chunks, however I am getting a "dropbox.exceptions.ApiError('some_random_numbers', UploadSessionLookupError(u'incorrect_offset', UploadSessionOffsetError(correct_offset=540)))

 

Here is my code that is uploading the zip file using chunks (from StackOverflow)

 

 

f = open(outputName)
file_size = os.path.getsize(outputName)

CHUNK_SIZE = 10000

if file_size <= CHUNK_SIZE:

	print dbx.files_upload(f.read(), backupPath)

else:

	upload_session_start_result = dbx.files_upload_session_start(f.read(CHUNK_SIZE))
	cursor = dropbox.files.UploadSessionCursor(session_id=upload_session_start_result.session_id,
												   offset=f.tell())
	commit = dropbox.files.CommitInfo(path=backupPath)

	while f.tell() < file_size:
if ((file_size - f.tell()) <= CHUNK_SIZE):
print dbx.files_upload_session_finish(f.read(CHUNK_SIZE), cursor, commit) else: dbx.files_upload_session_append(f.read(CHUNK_SIZE),cursor.session_id,cursor.offset) #dbx.files_upload_session_append_v2(f.read(CHUNK_SIZE), cursor) cursor.offset = f.tell()

 

 I tried v2 for files_upload_session_append as well. Both doesn't work. Also tried different CHUNK_SIZEs, but doesn't work either. Any suggestions?

 

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Staff rankDropbox Staff
    An incorrect_offset error indicates that you're trying to upload to an offset that doesn't match what the server last saw for this upload session. For instance, this can happen if previous request was received and processed successfully but the client did not receive the response, e.g. due to a network error. The correct_offset value tells you where you should resume the upload from.

    In this case, you're using chunk size of 10000 bytes, so it seems you're not actually uploading an entire chunk, since correct_offset is 540.

    To help debug this, I recommend printing out the length of data you're getting out of f.read each time to check that you're uploading as much as you expect to.

About Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

5,875 PostsLatest Activity: 2 years ago
323 Following

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X or Facebook.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!