You might see that the Dropbox Community team have been busy working on some major updates to the Community itself! So, here is some info on what’s changed, what’s staying the same and what you can expect from the Dropbox Community overall.
Forum Discussion
Do I have to wait until /upload_session/finish_batch/check finishes before starting the next batch?
When reading the docs for upload_session/finish_batch it says
For the same account, this route should be executed serially. That means you should not start the next job before current job finishes
I'm unsure whether that means I have to wait for the batch to finish with upload_session/finish_batch/check or not before starting the next batch?
It seems to work if I call upload_session/finish_batch/check in concurrently with opening a new batch.
However it also seems to drop files from the batch sometimes returning "too_many_write_operations" so maybe it isn't supported.
2021/03/19 09:21:03 DEBUG : HTTP REQUEST (req 0xc000f76000)
2021/03/19 09:21:03 DEBUG : POST /2/files/upload_session/finish_batch/check HTTP/1.1
Host: api.dropboxapi.com
User-Agent: rclone/v1.55.0-DEV
Content-Length: 113
Authorization: XXXX
Content-Type: application/json
Accept-Encoding: gzip
{"async_job_id":"dbjid:AABy2FH-VQkY-9lq_BMdfJVJI2_C-vRjliacnw5tIIKeKFitZxUgGsnWlDim84xSqANIbZr2bmqWWWRIPl7Dw_69"}
2021/03/19 09:21:04 DEBUG : HTTP RESPONSE (req 0xc000f76000)
2021/03/19 09:21:04 DEBUG : HTTP/2.0 200 OK
Accept-Encoding: identity,gzip
Cache-Control: no-cache
Content-Type: application/json
Date: Fri, 19 Mar 2021 09:21:03 GMT
Server: envoy
Vary: Accept-Encoding
X-Content-Type-Options: nosniff
X-Dropbox-Request-Id: f1ed7f49b82e4b08b96a6aa47c13b688
X-Dropbox-Response-Origin: far_remote
X-Frame-Options: SAMEORIGIN
X-Server-Response-Time: 63
{".tag": "complete", "entries": [{".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}, {".tag": "failure", "failure": {".tag": "too_many_write_operations"}}]}
It is, of course, a lot faster waiting for multiple batches to complete concurrently!
The documentation and this error are referring to "lock contention" which is an inability to make multiple changes in the same account (or more accurately, namespace) at the same time. I recommend reading the Performance Guide for more information for context, if you haven't already.
So, you should only run one /2/files/upload_session/finish_batch job per namespace at a time. That means waiting until /2/files/upload_session/finish_batch/check returns "complete" before initiating another job via another call to /2/files/upload_session/finish_batch.
You can however start and upload to other upload sessions, even for the same namespace, using /2/files/upload_session/start and /2/files/upload_session/append_v2, at the same time.
This is because the actual change is committed by the job started by /2/files/upload_session/finish_batch. The other calls, /2/files/upload_session/start and /2/files/upload_session/append_v2, don't commit changes; they just upload the data in preparation of being committed later.
- Greg-DB
Dropbox Staff
The documentation and this error are referring to "lock contention" which is an inability to make multiple changes in the same account (or more accurately, namespace) at the same time. I recommend reading the Performance Guide for more information for context, if you haven't already.
So, you should only run one /2/files/upload_session/finish_batch job per namespace at a time. That means waiting until /2/files/upload_session/finish_batch/check returns "complete" before initiating another job via another call to /2/files/upload_session/finish_batch.
You can however start and upload to other upload sessions, even for the same namespace, using /2/files/upload_session/start and /2/files/upload_session/append_v2, at the same time.
This is because the actual change is committed by the job started by /2/files/upload_session/finish_batch. The other calls, /2/files/upload_session/start and /2/files/upload_session/append_v2, don't commit changes; they just upload the data in preparation of being committed later.
Thank you Greg that was extremely clear.
I understand exactly now what I am doing 🙂
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
