You might see that the Dropbox Community team have been busy working on some major updates to the Community itself! So, here is some info on what’s changed, what’s staying the same and what you can expect from the Dropbox Community overall.

Forum Discussion

abhishek9851's avatar
abhishek9851
Explorer | Level 3
8 years ago

Java Batch upload via uploadSessionAppendV2

I am trying to batch upload multiple files via uploadSessionAppendV2, but somehow all the files are written to a single file.

Lets say i have two identical files file1.txt, file2.txt , each with size 2575.

Here is my code

 

List<UploadSessionFinishArg> entries = new ArrayList<>();
String sessionId = null;
long offset = 0;

// upload and get session id for first file
File file = new File("file1.txt");
InputStream in = new FileInputStream("file1.txt");
String sessionId =  getCleint().files().uploadSessionStart().uploadAndFinish(in).getSessionId();
offset =  file.length(); // 2575
UploadSessionCursor cursor = new UploadSessionCursor(sessionId,  offset);
CommitInfo commitInfo = new CommitInfo("/uploads/file1.txt", WriteMode.OVERWRITE, false, new Date(), false);
UploadSessionFinishArg arg = new UploadSessionFinishArg(cursor, commitInfo);

// second file
file = new File("file1.txt");
try (InputStream in = new FileInputStream("file2.txt")) {
    boolean close = true;
    getCleint().files().uploadSessionAppendV2(cursor, close).uploadAndFinish(in);
    offset += file.length(); //5150
    cursor = new UploadSessionCursor(sessionId,  offset);

    CommitInfo commitInfo = new CommitInfo("/uploads/file2.txt", WriteMode.OVERWRITE, false, new Date(), false);
    UploadSessionFinishArg arg = new UploadSessionFinishArg(cursor, commitInfo);
    entries.add(arg);

} catch (Exception e) {
    e.printStackTrace();
}


//now batch commit 
LaunchEmptyResult result = getCleint().files().uploadSessionFinishBatch(entries);

while ( getCleint().files().uploadSessionFinishBatchCheck(result.getAsyncJobIdValue()).isInProgress()) {
    try {
        Thread.sleep(1000);
    } catch (InterruptedException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

UploadSessionFinishBatchJobStatus status = getCleint(). files().uploadSessionFinishBatchCheck(result.getAsyncJobIdValue());
System.out.println(status.toString());

  

The problem is, all the data from both files is commited to /uploads/file2.txt, and file1.txt is not created at all.

This is the response i get, correct_offset":5150 for the first file, which is confusing.

 

{
    ".tag": "complete",
    "entries": [{
        ".tag": "failure",
        "failure": {
            ".tag": "lookup_failed",
            "lookup_failed": {
                ".tag": "incorrect_offset",
                "correct_offset": 5150
            }
        }
    }, {
        ".tag": "success",
        ".tag": "file",
        "name": "file2.txt",
        "id": "id:amNxvd7HFlAAAAAAAAAAvA",
        "client_modified": "2017-01-13T23:56:11Z",
        "server_modified": "2017-01-13T23:56:11Z",
        "rev": "1695120345e",
        "size": 5150,
        "path_lower": "/uploads/file2.txt",
        "path_display": "/uploads/file2.txt"
    }]
}
  • It looks like you're only calling uploadSessionStart once. You'll need to call that once per file. That is, a single "upload session" is used to upload a single file. You can use uploadSessionFinishBatch to finish multiple different upload sessions at once though.
  • abhishek9851's avatar
    abhishek9851
    Explorer | Level 3

    Cant figure it out how to delete the post, so i ll just post the answer about how to upload files in parallel. 

     

    UploadSessionCursor cursor = null;
    
    for (int i = 1; i <= 100; i++) {
        try {
    
            file = localFiles.get(i);
            remoteFileName = remoteFilePath + "/" + file.getName();
    
            currentPath = file.getAbsolutePath();
            try (InputStream in = new FileInputStream(currentPath)) {
                sessionId = getCleint().files().uploadSessionStart(true).uploadAndFinish(in).getSessionId();
    
                offset = file.length();
                cursor = new UploadSessionCursor(sessionId, offset);
                System.out.println("uploaded " + currentPath + " offset" + offset);
    
                CommitInfo commitInfo = new CommitInfo(remoteFileName, WriteMode.OVERWRITE, false, new Date(), false);
                UploadSessionFinishArg arg = new UploadSessionFinishArg(cursor, commitInfo);
                entries.add(arg);
            } catch (Exception e) {
                e.printStackTrace();
            }
    
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
    try {
    
        System.out.println("Batch entries commit");
    
        LaunchEmptyResult result = getCleint().files().uploadSessionFinishBatch(entries);
    
        while (getCleint().files().uploadSessionFinishBatchCheck(result.getAsyncJobIdValue()).isInProgress()) {
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    
        UploadSessionFinishBatchJobStatus status = getCleint().files().uploadSessionFinishBatchCheck(result.getAsyncJobIdValue());
        System.out.println(status.toString());
    
    } catch (Exception e) {
        e.printStackTrace();
    }

     

  • Greg-DB's avatar
    Greg-DB
    Icon for Dropbox Staff rankDropbox Staff
    It looks like you're only calling uploadSessionStart once. You'll need to call that once per file. That is, a single "upload session" is used to upload a single file. You can use uploadSessionFinishBatch to finish multiple different upload sessions at once though.

About Dropbox API Support & Feedback

Node avatar for Dropbox API Support & Feedback

Find help with the Dropbox API from other developers.

5,910 PostsLatest Activity: 3 days ago
333 Following

If you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X or Facebook.

For more info on available support options for your Dropbox plan, see this article.

If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!