cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Announcements
We are making some updates so the Community might be down for a few hours on Monday the 11th of November. Apologies for the inconvenience and thank you for your patience. You can find out more here.

Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Uploading large files using chunks

Uploading large files using chunks

newdbuser
Explorer | Level 3

Hello: I am using this Python code to upload a large zip file (2-4 GB). I am trying to upload the file in chunks, however I am getting a "dropbox.exceptions.ApiError('some_random_numbers', UploadSessionLookupError(u'incorrect_offset', UploadSessionOffsetError(correct_offset=540)))

 

Here is my code that is uploading the zip file using chunks (from StackOverflow)

 

 

f = open(outputName)
file_size = os.path.getsize(outputName)

CHUNK_SIZE = 10000

if file_size <= CHUNK_SIZE:

	print dbx.files_upload(f.read(), backupPath)

else:

	upload_session_start_result = dbx.files_upload_session_start(f.read(CHUNK_SIZE))
	cursor = dropbox.files.UploadSessionCursor(session_id=upload_session_start_result.session_id,
												   offset=f.tell())
	commit = dropbox.files.CommitInfo(path=backupPath)

	while f.tell() < file_size:
if ((file_size - f.tell()) <= CHUNK_SIZE):
print dbx.files_upload_session_finish(f.read(CHUNK_SIZE), cursor, commit) else: dbx.files_upload_session_append(f.read(CHUNK_SIZE),cursor.session_id,cursor.offset) #dbx.files_upload_session_append_v2(f.read(CHUNK_SIZE), cursor) cursor.offset = f.tell()

 

 I tried v2 for files_upload_session_append as well. Both doesn't work. Also tried different CHUNK_SIZEs, but doesn't work either. Any suggestions?

 

1 Reply 1

Greg-DB
Dropbox Staff
An incorrect_offset error indicates that you're trying to upload to an offset that doesn't match what the server last saw for this upload session. For instance, this can happen if previous request was received and processed successfully but the client did not receive the response, e.g. due to a network error. The correct_offset value tells you where you should resume the upload from.

In this case, you're using chunk size of 10000 bytes, so it seems you're not actually uploading an entire chunk, since correct_offset is 540.

To help debug this, I recommend printing out the length of data you're getting out of f.read each time to check that you're uploading as much as you expect to.
Need more support?
Who's talking

Top contributors to this post

  • User avatar
    Greg-DB Dropbox Staff
What do Dropbox user levels mean?