Python Dropbox API slow performance

[Sadsyndrome]

Hello

Im not a professional programmer..I recently built an app on python running on raspberry pi for monitoring energy usage at home.

it all works fine, I was using Dropbox app to save photo images temporarily. Images are approx 1mb saved every 30 mins and deleted once I have finished with them.

this worked fine for a few hours until my access token expired. After getting a refresh token to resolve that issue it was all working again. However the next day performance dropped for the simple upload of these image files and even a test text file. The upload takes in region of 3 .5 minutes regardless of size of upload or even reading the directory listing.

I assumed this was th refresh token process so downloaded a new access token, but even using that without refresh token the Dropbox ap still takes 3.5 mins for any action??

Any ideas what may cause this? As I said it was working fine until original access token expired and also for a period after I changed it for refresh token? Would it be rate limiting or something like that?

[Greg-DB]

The access token/refresh token shouldn't have any impact on that. The Dropbox API does have a rate limiting system, but it works by responding with a rate limit error response to any call it needs to reject, not by slowing down the connection speed.

Your connection speed to Dropbox depends on the routing you get between your ISP and our servers, and may be slower than your ISP's rated speeds. Sometimes resetting or retrying your connection gets you a different route and better speeds, but that is outside of our control. Some ISPs also throttle sustained connections so if you see an initial high connection speed followed by lower speeds, that could be the reason. Otherwise, there may be other local factors impacting the processing speed of your code, as well as the speed of your network connection. We can't offer help with your device or network connection though, so you may need to contact your computer and/or network admin, if any, for help with that.

To debug this you may want to step through with a debugger, and/or add more logging. If you do find an issue with a specific Dropbox API call, feel free to show the details here so we can take a look. (Be sure to redact any access token, refresh token, or app secret though.)

[Sadsyndrome]

Thx. I don’t think it’s a network error. I’m transferring data to google sheets with no issue. The upload file process takes 3.5 mins for a single text file 201 bytes. The time does seem to be pretty consistent at approx 3.5 mins regardless of file size. It worked perfectly yesterday. Even getting file metadata takes over 2mins? Is there anywhere I can look at the Dropbox end ? I don’t have an IDE I just use a text editor to knock up my python scripts and not sure where to start with debugging to be honest. I guess I could take a look at network monitor if I remember how! I’ve been out of the IT game for 4 years now and memory is fading:)

here is script and  output. Sorry couldn’t see how to attach these so I put them in this message. It works (slowly) so I assume keys and python syntax ok 

##

import dropbox
from datetime import datetime
mylocpath ='/home/mark/FTP/myfiles/'
locfname='gasmeter.txt'
dboxpath='/meterreadings/'
file_from = mylocpath +locfname
file_to = dboxpath+locfname

dbx = dropbox.Dropbox(
app_key = "mykey",
app_secret = "mysecret",
oauth2_refresh_token = "myrefreshtokeen"
)
f = open(file_from, 'rb')

now = datetime.now()
current_time = now.strftime("%H:%M:%S")
print ("start dbx call Time =", current_time)

dbx.files_upload(f.read(), file_to)

now = datetime.now()
current_time = now.strftime("%H:%M:%S")
print ("end dbx call =", current_time)
print ("done")

##

mark@raspberrypi:~/pwrlog/test $ python dbupl.py
start dbx call Time = 20:10:10
end dbx call = 20:13:32
done
mark@raspberrypi:~/pwrlog/test $

e = 19:25:12
done

[Greg-DB]

The route you get to the Dropbox servers may be different than the route to get to the servers for other services, so a comparison with other services may not be applicable.

Anyway, Dropbox does not offer the ability to debug this from the Dropbox servers. You'd need to do any debugging on your side.

Looking at the code itself, I don't see any issues. I just tried this script myself, plugging in my path and key/secret/refresh token values, and the script ran in 2 seconds to successfully upload a 1 MB file.

I see you're using the Python SDK, so the following code would enable additional debugging output which might be useful:

try:
    import http.client as http_client
except ImportError:
    # Python 2
    import httplib as http_client
http_client.HTTPConnection.debuglevel = 1

import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger().setLevel(logging.DEBUG)
requests_log = logging.getLogger("requests.packages.urllib3")
requests_log.setLevel(logging.DEBUG)
requests_log.propagate = True

import urllib3
urllib3.add_stderr_logger()

[Sadsyndrome]

Hi. Just a quick update in case it is useful.

I reinstalled dropbox using pip install. 3.5 mins download persisted.

rebooted rpi still no joy

tested over a different network ( my smartphone hotspot) same problem on that network

I ran tcpdump and saw the host name api.dropboxapi.com in amongst lots of traffic

I pinged that from rpi but  dns response was ipv6 and no traffic flowed?

I disabled ipv6 on rpi and problem is fixed???

not sure why it worked yesterday?

unless my changes have coincided with  something else?

my api shows 1000 hits a day on analytics, I don’t understand that either???? I love technology:)

thx for your prompt responses