We are aware of the issue with the badge emails resending to everyone, we apologise for the inconvenience - learn more here.

Forum Discussion

Sayak B.'s avatar
Sayak B.
Explorer | Level 3
9 years ago

Dropbox API in C# how to reuse the session id of bulk file upload in case file could not be uploaded by any issue.

In case of uploading big file of more than 100 mb, if by some case the internet connection get dropped then how it will be possible to upload the same file from the chunk it has been failed to upload. Please suggest. I have tried to upload by same session id but it never worked. It is getting exception.

  • Sayak B.'s avatar
    Sayak B.
    Explorer | Level 3

    I am using Dropbox .NET library.

    The problem is. I am actually trying to upload my sql server database backup everyday by zipping it to DropBox. The backup files after zip is average 50 mb in size. But the reality is while upload the internet get disconnected several time. So I need to upload the same thing fresh again and again the internet connection issue happens, The database server is in a remote location where internet connection is not so good. The result is that the file could not ever been uploaded because of the said issue, What I am trying to do is to use the upload session id which I am getting after uploading the first chunk. I am keeping it in a xml file and checking if the connection is failed and the file could not been uploaded then restart uploading the same file from the last successful uploaded chunk by using the session id. Below is my code snippet.

     

    async void ConnectDropBox()
    {
    await LoginDropBox();

    var progress = new Progress<ProgressReport>();
    progress.ProgressChanged += (o, e) =>
    {
    Dispatcher.BeginInvoke((Action)(() =>
    {
    pgbarUpload.Maximum = e.TotalSteps;
    pgbarUpload.Value = e.CurrentStep;
    lblStatus.Text = "Uploading slice " + e.CurrentStep.ToString() + " of " + e.TotalSteps;
    File.WriteAllText(AppVariables.xmlFilePath, AppVariables.CurrentUpload.ToXml<xmlUpload>());

    if (e.CurrentStep == e.TotalSteps)
    {
    lblStatus.Text = string.Format("File \"{0}\" uploaded successfully with {1} slices", Path.GetFileName(AppVariables.ZipFilePath), e.TotalSteps);
    btnUpload.IsEnabled = true;
    btnRefresh.IsEnabled = true;
    }
    }));
    };

    try
    {
    await Upload(progress);
    }
    //catch (InvalidOperationException) { }
    catch (Exception ex)
    {
    JHMessageBox.Show(ex.Message, "Exception Occured !", MessageBoxButton.OK, MessageBoxImage.Error);
    }
    }

     

    private async Task Upload(IProgress<ProgressReport> progress)
    {
    try
    {
    const int chunkSize = 128 * 1024;
    bool IsPrevUploadFailed = false;

    using (var stream = new FileStream(AppVariables.ZipFilePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
    {
    int numChunks = (int)Math.Ceiling((double)stream.Length / chunkSize);

    var progressReport = new ProgressReport() { TotalSteps = numChunks };

    byte[] buffer = new byte[chunkSize];
    string sessionId = null;
    await Task.Run(async () =>
    {
    for (var idx = 0; idx < numChunks; idx++)
    {

    if (!AppVariables.LastUpload.IsNull() && !IsPrevUploadFailed)
    {
    if (AppVariables.CurrentUpload.Date == AppVariables.LastUpload.Date &&
    AppVariables.CurrentUpload.FileChecksum == AppVariables.LastUpload.FileChecksum &&
    AppVariables.CurrentUpload.FileName == AppVariables.LastUpload.FileName)
    {
    sessionId = AppVariables.LastUpload.SessionId;
    IsPrevUploadFailed = true;
    }
    }


    var byteRead = stream.Read(buffer, 0, chunkSize);

    using (MemoryStream memStream = new MemoryStream(buffer, 0, byteRead))
    {
    if ((idx == 0 && !IsPrevUploadFailed) || (IsPrevUploadFailed && !AppVariables.LastUpload.IsNull() && AppVariables.LastUpload.Slice == 0))
    {
    var result = await AppVariables.DrpBoxClient.Files.UploadSessionStartAsync(body: memStream);
    sessionId = result.SessionId;
    }

    else
    {
    UploadSessionCursor cursor = new UploadSessionCursor(sessionId, (ulong)(chunkSize * idx));
    if (!IsPrevUploadFailed || (IsPrevUploadFailed && !AppVariables.LastUpload.IsNull()))
    {
    if (idx == numChunks - 1)
    {
    await AppVariables.DrpBoxClient.Files.UploadSessionFinishAsync(cursor, new CommitInfo(Config.DropBoxDirectory + System.IO.Path.GetFileName(AppVariables.ZipFilePath)), memStream);
    }

    else
    {
    await AppVariables.DrpBoxClient.Files.UploadSessionAppendV2Async(cursor, body: memStream);
    }
    }
    }
    }


    AppVariables.CurrentUpload.SessionId = sessionId;
    AppVariables.CurrentUpload.Slice = idx;
    AppVariables.CurrentUpload.FileName = AppVariables.ZipFileName;

    progressReport.CurrentStep = idx + 1;
    progressReport.PercentComplete = (idx + 1) * 100 / progressReport.TotalSteps;
    progress.Report(progressReport);
    }
    });
    stream.Close();
    stream.Dispose();
    }
    }
    catch (Exception ex)
    {
    throw ex;
    }
    }
    }

    public class ProgressReport
    {
    public int PercentComplete { get; set; }
    public int CurrentStep { get; set; }
    public int TotalSteps { get; set; }
    }

    public class xmlUpload
    {
    public DateTime Date { get; set; }
    public string FileChecksum { get; set; }
    public string FileName { get; set; }
    public int Slice { get; set; }
    public string SessionId { get; set; }
    }

     

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Staff rankDropbox Staff

    You can certainly use upload sessions to resume uploads of large files by persisting and re-using the session ID. If a particular request fails, you just need to re-upload that chunk, and not the whole file again.

    Where does your code fail for you though? What exception(s) are you getting? Since you are using the .NET SDK, I do recommend starting from that code sample to get the basic file uploading logic first.

  • Sayak B.'s avatar
    Sayak B.
    Explorer | Level 3

    Basic file upload is done with the code I have provided. That is working fine. I want to keep the file information including the session id that I get after uploading the first chunk. But that generating exception. Following is the exception--

    Dropbox.Api.ApiException`1[Dropbox.Api.Files.UploadSessionLookupError]: incorrect_offset/..
    at Dropbox.Api.DropboxRequestHandler.<Dropbox.Api.Stone.ITransport.SendUploadRequestAsync>d__5`3.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
    at BackupUpload.frmZipNUpload.<>c__DisplayClass5_1.<<Upload>b__0>d.MoveNext() in E:\Jorehaut Tea Limited\Applications\BackupUpload\BackupUpload\Classes\frmZipNUpload.cs:line 234
    --- End of stack trace from previous location where exception was thrown ---
    at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
    at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
    at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
    at BackupUpload.frmZipNUpload.<Upload>d__5.MoveNext()

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Staff rankDropbox Staff

    Thanks! The incorrect_offset error indicates that the server is expecting a different "offset" value (how far along in the file you are in the upload session) than the app is submitting. You can find more information on that in the documentation here:

    https://dropbox.github.io/dropbox-sdk-dotnet/html/T_Dropbox_Api_Files_UploadSessionLookupError_IncorrectOffset.htm

    You can get the expected offset value from UploadSessionOffsetError.CorrectOffset:

    https://dropbox.github.io/dropbox-sdk-dotnet/html/P_Dropbox_Api_Files_UploadSessionOffsetError_CorrectOffset.htm

    You'll want to add some logic to deal with this error (especially since your app will be running from an unreliable connection.)

    • Sayak B.'s avatar
      Sayak B.
      Explorer | Level 3

      Finally I resolved the "incorrect_offset" issue by catching this exception and uploading the next slice. But the thing is after successful upload the file is corrupt. i.e. I am uploading zip file. that is corrupt and can not unzip by windows. What I am doing wrong and why it is happening ? can you please suggest what should be done ?

      • Greg-DB's avatar
        Greg-DB
        Icon for Dropbox Staff rankDropbox Staff

        It sounds like there may be a bug in your uploading logic. It's hard to say what it may be offhand, so I recommend adding some logging to your upload code to see where the issue may be and determine where the incorrect data is being sent up, or if you're sending the data out of order or something.

         

        Have you tried uploading via the sample I linked to though? Does that work correctly for you?

About Dropbox API Support & Feedback

Node avatar for Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

5,877 PostsLatest Activity: 12 months ago
325 Following

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X or Facebook.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!