You might see that the Dropbox Community team have been busy working on some major updates to the Community itself! So, here is some info on what’s changed, what’s staying the same and what you can expect from the Dropbox Community overall.
Forum Discussion
Levi2
6 years agoExplorer | Level 3
Internal server error when sharing folders
Hi, is there some kind of limitation with sharing folders? I'm trying to share 5.4k folders and I'm getting
PollError('internal_error', None))
consistently. Sometimes the files share, sometimes they don't. I can manually share them no problem and usually I can share them as one offs in a python REPL using the dropbox sdk.
There are no error details.
My code is functionallt simialr to the snippet below:
import dropbox #censored access_token = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' dbx = dropbox.Dropbox(access_token) def get_entries(folder_result_list): '''using this so we don't try and share the same folder twice''' entries = [] for entry in folder_result_list.entries: if not hasattr(entry,'shared_folder_id') and not entry.shared_folder_id: entries.append(entry) return entries folder_result_list = dbx.files_list_folder(parent_folder,include_mounted_folders=False,limit=2000) entries = get_entries(folder_result_list) while folder_result_list.has_more: folder_result_list = dbx.files_list_folder_continue(folder_result_list.cursor) entries.extend(get_entries(folder_result_list)) #filter for only FolderMetadata so we don't try and share DeletedFolder's or individual files entry_dict = {entry.id:entry for entry in entries if isinstance(entry,dropbox.files.FolderMetadata)} for index,keyval in enumerate(entry_dict.items()): entry_id = keyval[0] entry = keyval[1] try: #forcing async so we get consistent results. checking the job status and doing stuff with the shared_folder_id later on in the script entry_id_to_job_id[entry_id] = dbx.sharing_share_folder(entry.path_lower,force_async=True).get_async_job_id() except dropbox.exceptions.ApiError as e: #do some logging of errors here pass
The full error looks something like this:
ApiError('0ed9d04d8f01449b5dedea33b7445fee', PollError('internal_error', None))
Thanks for the post, and apologies that the error message isn't more helpful in this case. For this `internal_error` case, I looked it up and I can confirm that it's actually due to "lock contention".
That's a result of how Dropbox works on the backend. It's a technical inability to make multiple modifications, such as sharing a folder, at the same time. The simultaneous activity could be coming from your app itself, or elsewhere, e.g., from the user's desktop client. It can come from the same user, or another member of a shared folder. You can find more information about lock contention here:
https://www.dropbox.com/developers/reference/data-ingress-guide
In short, to avoid this error, you should avoid making multiple concurrent state modifications. E.g., don't issue multiple such requests at a time, and use batch endpoints whenever possible. That won't guarantee that you won't run in to this error though, as contention can still come from other sources, so you may also want to implement automatic retries in your app. For this particular scenario of sharing multiple folders, I also recommend adding a delay between the consecutive share operations, which can help in certain cases.
I'll send this along as a request to the team to improve the error reporting.
- Greg-DBDropbox Staff
Thanks for the post, and apologies that the error message isn't more helpful in this case. For this `internal_error` case, I looked it up and I can confirm that it's actually due to "lock contention".
That's a result of how Dropbox works on the backend. It's a technical inability to make multiple modifications, such as sharing a folder, at the same time. The simultaneous activity could be coming from your app itself, or elsewhere, e.g., from the user's desktop client. It can come from the same user, or another member of a shared folder. You can find more information about lock contention here:
https://www.dropbox.com/developers/reference/data-ingress-guide
In short, to avoid this error, you should avoid making multiple concurrent state modifications. E.g., don't issue multiple such requests at a time, and use batch endpoints whenever possible. That won't guarantee that you won't run in to this error though, as contention can still come from other sources, so you may also want to implement automatic retries in your app. For this particular scenario of sharing multiple folders, I also recommend adding a delay between the consecutive share operations, which can help in certain cases.
I'll send this along as a request to the team to improve the error reporting.
- Levi2Explorer | Level 3
Thanks for the links.
This makes sense since I was trying to share thousands of folders. However, I avoided multi-threading and no one else is using this folder since it's a test folder.
It seems that namespace locking is the culprit since all of the folders are in the same name space and I'm using force_asyn=True leaving when the folder actually gets shared up to dropbox.
I know that there are batch endpoints for uploading data, but are there batch endpoints for sharing folders?
- Greg-DBDropbox Staff
No, unfortunatelty there isn't a batch option for sharing folders in particular, but I'll pass this along as a feature request. I can't promise if or when that might be implemented though.
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
5,911 PostsLatest Activity: 2 hours agoIf you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X or Facebook.
For more info on available support options for your Dropbox plan, see this article.
If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!