Dropbox API Creating Shared link settings Error.

[Nellyb43]

I have a program in Python 3.9 which cleans data every morning, creates a final dataset, and then uses the dropbox api to create a shared link with settings (password) for the file, then email that link with the password to those who subscribe to my daily newsletter.

This setup works... but only some of the time. For many months it was completely reliable, but now the creating shared link settings break the whole program.

By some of the time, I mean that the program will work Monday, Tuesday, Wednesday, but break on Thursday.

Luckily, I do get a continuous error, but am writing this because I am unable to find a fix.

Code:

import dropbox
dailyfile_csv = "\Daily Data (2020-12-29 10.07 AM).csv"
dailyfile_dta = "\Daily Data (2020-12-29 10.07 AM).dta"

/Daily Lobbying Data (2020-12-29 10.07 AM).dta
dbx = dropbox.Dropbox(token)
user = dbx.users_get_current_account()
print("Dropbox Account Info: " + str(dbx.users_get_current_account()))
#Should have logged onto dropbox application.


#STEP: Creating randomly generated passwords for the data.
csvpass = str(randompassword())
print("CSV Password = " + csvpass)
dtapass = str(randompassword())
print("DTA Password = " + dtapass)
#Created randomly generated passwords for the data.


#These are link settings for the files which say that the file requires a password, and we set the password equal to the randomly generated passwords above.
link_settings_csv = dropbox.sharing.SharedLinkSettings(
        requested_visibility = dropbox.sharing.RequestedVisibility.password,
        link_password=csvpass)
        #expires = datetime.datetime.now() + datetime.timedelta(days=7))

link_settings_dta = dropbox.sharing.SharedLinkSettings(
        requested_visibility = dropbox.sharing.RequestedVisibility.password,
        link_password=dtapass)
        #expires = datetime.datetime.now() + datetime.timedelta(days=7))
#Setting link settings as requiring password.

time.sleep(15)

#STEP: VERY IMPORTANT. This creates a link for the huge data file either in csv or dta format to be sent in the daily emails.
csv_link = dbx.sharing_create_shared_link_with_settings(dailyfile_csv,settings=link_settings_csv).url
time.sleep(10)
print(csv_link)
time.sleep(7)
dta_link = dbx.sharing_create_shared_link_with_settings(dailyfile_dta,settings=link_settings_dta).url
time.sleep(10)
print(dta_link)
time.sleep(7)
print("Finished creating dropbox links.")

Error Code:

#This happens right after attempting to create the first shared link.
Traceback (most recent call last):
  File "D:\Dropbox\Lobbyingdata.com\Lobbying Data\Daily_Data_Step_3_Dropboxing.py", line 102, in <module>
    csv_link = dbx.sharing_create_shared_link_with_settings(dailyfile_csv,settings=link_settings_csv).url
  File "C:\Users\syxsc\AppData\Roaming\Python\Python39\site-packages\dropbox\base.py", line 3969, in sharing_create_shared_link_with_settings
    r = self.request(
  File "C:\Users\syxsc\AppData\Roaming\Python\Python39\site-packages\dropbox\dropbox_client.py", line 338, in request
    raise ApiError(res.request_id,
dropbox.exceptions.ApiError: ApiError('dd1470bb63b1468b977c6c3fdd30afdc', CreateSharedLinkWithSettingsError('path', LookupError('not_found', None)))

[Greg-DB]

A 'path/not_found' error like this indicates that creating the shared link failed because there was nothing found at the specified path to link to. In this case, based on the error output, this is referring to the path you're supplying in your 'dailyfile_csv' variable.

You'll need to check why there isn't anything at that path. Perhaps you meant to upload something there, but didn't do so successfully. Or, perhaps your 'dailyfile_csv' value is just incorrect. 

[Nellyb43]

So, when running my program I generate the dataset in a folder not in the dropbox app folder, then use shurtil.copyfile to copy that file into the dropbox application folder. 

shutil.copyfile(oldpath,dropbox_app_path)

Then to create dailyfile_csv I do 

dailyfile_csv = newpath_csv_dbx.replace('\\','/')[40:]

This takes something like ""D:\Dropbox\Apps\App Folder Name\Daily Data (2020-12-29).dta" and turns it into "/Daily Data (2020-12-29).dta", which is what the dropbox api wants.

I do indeed see my file "dailyfile_csv" in the application folder - is there anything else that may be going on?

[Nellyb43]

That smiley face is actually:  "40 colon ]"

[Nellyb43]

So I ended up finding a fix for this by having my program wait 60 seconds with time.sleep(60) and somehow the sharing works perfectly again.

Please correct me if I am wrong but I am guessing the sync for the app folder is not instantaneous, especially due to the massive size of the files I attempt to share. Since the sync needs some time, I add a waiting period of a minute in order to ensure that the big file has been transferred and synced from my internal folders to the dropbox app folder. 

[Greg-DB]

Right, uploads are not instantaneous. Given your description, it looks like you're writing to the local filesystem and are relying on the Dropbox desktop client to upload the file to Dropbox. The file needs to be uploaded to Dropbox, not just written to the local filesystem, before sharing_create_shared_link_with_settings will work for it.

So, your API sharing_create_shared_link_with_settings code was apparently running before the file was actually uploaded, which would lead to the error. (This could happen when the upload takes a bit of time, or if the client isn't running, etc.)

You can also use other API methods such as files_get_metadata or files_list_folder/files_list_folder_continue to check what files exist on Dropbox first.

[Nellyb43 · Accepted solution]

Very interesting. Do you recommend I upload the file straight to dropbox using the API over using util to simply copy it over from my local directory?

I arbitrarily chose 60 seconds of time to give dropbox time to fully sync up the file in the application folder. Each file is on average 500MB. Do you recommend I go with a longer length of time just to make sure the sync works with sharing no matter what. One minute here and there is no issue.

[Greg-DB]

You can certainly use the API to upload files instead (files_upload for files smaller than 150 MB, or upload sessions otherwise). That would be helpful in that the code would directly know when the file is completely uploaded.

That's not strictly necessary though. You can use other functionality on the API to detect changes, e.g., to determine when the file is uploaded. Check out the Detecting Changes Guide for more information.

In any case, I wouldn't recommend just waiting some amount of time, as that could be unreliable.

[Nellyb43]

Thank you so much! I have changed my code to allow for uploading of the file from the api rather than just moving it into my file directory. The program is running smoothly now and with no hickups.