The Community is undergoing a major upgrade. Find out more, ask us questions or share your thoughts here.

Forum Discussion

Opuscope's avatar
Opuscope
Helpful | Level 5
8 years ago

.Net SDK on UWP download stream sends empty buffers on read

Hi Dropbox dev community!

 

We're having a bit of a problem with the .Net SDK on our UWP app : we're using response.GetContentAsStreamAsync() after getting the path of a file on a dropbox account, for progression measurement and having it non blocking for our UI (context : we're on a Hololens inside a Unity app). We used to go for response.GetContentAsByteArrayAsync() before but on bigger files, the hololens memory just doesn't hold up and usually crashes the entire download task. So we went for a stream for progressively downloading and saving the file locally. This is our current code :

 

 

ulong fileSize = response.Response.Size;
int bufferSize = 4 * 1024 * 1024;
var buffer = new byte[bufferSize];
ulong downloadProgress = 0;
using (var stream = await response.GetContentAsStreamAsync())
{
    int byteReadLength = stream.Read(buffer, 0, bufferSize);
    while (byteReadLength > 0)
    {
        using (var outputStream = fileStream.GetOutputStreamAt(downloadProgress))
        {
            using (var writer = new DataWriter(outputStream))
            {
                writer.WriteBytes(buffer);
                await writer.StoreAsync();
                await outputStream.FlushAsync();
            }
        }
        downloadProgress += (ulong)buffer.Length;

        if (downloadProgress > fileSize)
            break;
        if (downloadProgress + (ulong)bufferSize > fileSize)
        {
            bufferSize = (int)(fileSize - downloadProgress);
        }
        byteReadLength = stream.Read(buffer, 0, bufferSize);
    }
}

First things first : on usage, this code seems to work, the streaming seems to happen chunk by chunk of 4mb and the save to file does create an adequately sized result on the hard drive. Keep in mind this runs on a Hololens, the main reason we force the file save during the stream read loop is because we do not have the memory capacity to fully load big (300mb+) files into RAM.

 

But... when trying to open the downloaded file locally, may it be a video, a (really big) text file or a picture, it's always corrupted. And further analysis of the file indicates that :

  • the first 4mb chunk went through correctly
  • the rest is just 0x00 bytes with occasional nonsensical clutter every 100mbs or so

So, we kinda have a few questions:

  • are we doing something wrong?
  • why is there no EOF on stream? We had to do the downloadProgress check stuff to check we weren't getting more data than was expected, because we managed to fill up a Hololens' entire HDD with one download that never ended

Small remark : yes, we know the SDK is not really compatible with Unity, but at this point in the app, the context is full UWP, completely separate from the Unity main thread. Also, it works with the GetContentAsByteArrayAsync() method, only that the Hololens doesn't like getting blasted with so much live memory usage.

 

thank you :)

 

  • We looked into this, and we believe the issue here is due to this possible behavior from Stream.Read:

     

     

    Return Value
    Type: System.Int32
    The total number of bytes read into the buffer. This can be less than the number of bytes requested if that many bytes are not currently available, or zero (0) if the end of the stream has been reached.

     

    That means that byteReadLength in the code snippet here won't always necessarilly be a whole 4 MB. The being the case, it looks like you'll need to make two changes:

    - when you write the data to the file, only write as much as byteReadLength

    - when you increment your downloadProgress, only add as much as byteReadLength

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Staff rankDropbox Staff

    We looked into this, and we believe the issue here is due to this possible behavior from Stream.Read:

     

     

    Return Value
    Type: System.Int32
    The total number of bytes read into the buffer. This can be less than the number of bytes requested if that many bytes are not currently available, or zero (0) if the end of the stream has been reached.

     

    That means that byteReadLength in the code snippet here won't always necessarilly be a whole 4 MB. The being the case, it looks like you'll need to make two changes:

    - when you write the data to the file, only write as much as byteReadLength

    - when you increment your downloadProgress, only add as much as byteReadLength

    • Opuscope's avatar
      Opuscope
      Helpful | Level 5

      it works much better indeed! Thank you!

About Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

5,875 PostsLatest Activity: 56 minutes ago
323 Following

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X or Facebook.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!