You might see that the Dropbox Community team have been busy working on some major updates to the Community itself! So, here is some info on what’s changed, what’s staying the same and what you can expect from the Dropbox Community overall.
Forum Discussion
Robust error handling with Python SDK
I'm writing a file-transfer service that uses the dropbox API via the SDK (8.0.0 release). Most of the time it works, but there have been a couple times where I've encountered errors that I'd like t...
1. The 150 MB limit is technically not an exact limit. It depends on some network conditions which can vary, so even uploads smaller than that can sometimes fail.
It looks like you already found how to change the timeout, but in general we do recommend using chunk sizes smaller than that to begin with. Using a smaller chunk size adds overhead by increasing the number of requests necessary, but it lowers the chance of any call failing, and reduces how much you need to re-upload when one does.
There isn't an optimal chunk size since this will depend on various factors, so you may want to try a few different sizes and see what works best for your app.
2. The connection reset error looks like transient network or infrastructure issues, so you should just have your app automatically catch and retry these.
Upload sessions are good for 48 hours after being created. They aren't invalidated if any particular request after that fails. So, when you get one of these requests, you can just try again with the same upload.
Depending on when during the API call the connection failed, the offset may or may not change. It's probably much more likely that it will fail before all of the data has been uploaded, in which case the offset wouldn't change, so you have the right idea for handling that by re-seeking your file object. (If it does fail after the full request has been received but before your app gets the response, your app will then have the wrong offset. Although unlikely, your app can catch that particular case on the next call, which will fail with an `incorrect_offset` error, containing the actual expected offset.)
Greg-DB
Dropbox Staff
Dropbox Staff1. The 150 MB limit is technically not an exact limit. It depends on some network conditions which can vary, so even uploads smaller than that can sometimes fail.
It looks like you already found how to change the timeout, but in general we do recommend using chunk sizes smaller than that to begin with. Using a smaller chunk size adds overhead by increasing the number of requests necessary, but it lowers the chance of any call failing, and reduces how much you need to re-upload when one does.
There isn't an optimal chunk size since this will depend on various factors, so you may want to try a few different sizes and see what works best for your app.
2. The connection reset error looks like transient network or infrastructure issues, so you should just have your app automatically catch and retry these.
Upload sessions are good for 48 hours after being created. They aren't invalidated if any particular request after that fails. So, when you get one of these requests, you can just try again with the same upload.
Depending on when during the API call the connection failed, the offset may or may not change. It's probably much more likely that it will fail before all of the data has been uploaded, in which case the offset wouldn't change, so you have the right idea for handling that by re-seeking your file object. (If it does fail after the full request has been received but before your app gets the response, your app will then have the wrong offset. Although unlikely, your app can catch that particular case on the next call, which will fail with an `incorrect_offset` error, containing the actual expected offset.)
Great- thanks! Just what I needed to know (i.e. I do not need to create a new session).
If it helps anyone else, here's how I handled that particular exception (in terms of chasing down where one can get the correct offset). I'm not (yet) concerned about other potential reasons for raising the ApiError:
try: client.files_upload_session_append_v2(file_obj.read(DEFAULT_CHUNK_SIZE), cursor) cursor.offset = file_obj.tell() except dropbox.exceptions.ApiError as ex:
if ex.error.is_incorrect_offset(): correct_offset = ex.error.get_incorrect_offset().correct_offset # these methods don't read so naturally cursor.offset = correct_offset file_obj.seek(correct_offset)
# other potential problems here...I suppose there could also be some type-checking in there since I'm not sure ex.error is always an object, nor one that has a `is_incorrect_offset` method.
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
