We're making changes to the Community, so you may have received some notifications - thanks for your patience and welcome back. Learn more here.
Forum Discussion
/files/save_url has high rate of failure
Hi, I am currently testing using the /files/save_url API endpoint and am experiencing a high rate of failure. I am testing using a set of 6 small image files stored in S3-compatible storage environments, and using their public URLs with the aforementioned API endpoint. In every test run, there is at least one file that fails to be created in Dropbox, and it is a different file from the set each time. A file that fails in test #1 will succeed in tests #2, #3 and fail again in test #4 etc. These image files are standard .jpg files and generally 500kb or less.
Upon checking the status of the API requests using /files/save_url/check_job_status, the failed IDs return the "download_failed" response. I know there is a 5 minute limit on file transfers using /files/save_url, however all these failed IDs are returning their failure response only a few seconds after initiating the API request.
It may be worth mentioning that the API requests to /files/save_url are occurring all together. I don't believe the URLs are being rate-limited on the S3-compatible storage side, as it is tested with multiple environments, including self-hosted storage without such restrictions.
Done some googling, found some users with the same issue, but unable to find anything that resolves the issue.
Some example failed job IDs:
nJekCKfZPnwAAAAAAAAAAQ
aPQH1k_ei7cAAAAAAAAAAQ
Is there a way to resolve this issue?
- Greg-DB
Dropbox Staff
Thanks for the report! It looks like this is hitting an internal constraint. We'll look into what we may be able do about that. In the meantime, to work around this, you may want to add an automatic retry or two (i.e., starting at save_url again for the relevant URL) when this occurs.
Greg-DB wrote:Thanks for the report! It looks like this is hitting an internal constraint. We'll look into what we may be able do about that. In the meantime, to work around this, you may want to add an automatic retry or two (i.e., starting at save_url again for the relevant URL) when this occurs.
Thanks for the response. Unfortunately, our intended use case for our Dropbox API implementation would result in us transferring a greater amount than 6 files at once, so automatic retries would not really be viable at the current rate of failure. Are you able to provide any information on what this internal constraint is? If so, we may be able to work around it.
- Greg-DB
I don't have much more specific information to share, but it does seem that it may be related to issuing multiple jobs at or around the same time, likely resulting in the same sort of lock contention discussed here.
I'll also pass this along as a feature request for a batch version of this call to better support use cases where you need to submit many of these, but I can't promise if or when that might be implemented.
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
