You might see that the Dropbox Community team have been busy working on some major updates to the Community itself! So, here is some info on what’s changed, what’s staying the same and what you can expect from the Dropbox Community overall.
Forum Discussion
michal_k
9 years agoExplorer | Level 4
[Java SDK V2] How to do batch upload?
Hello, I am trying to make Batch upload work by using JAVA SDK v2. I read this documentation and tried to perform a batch upload. There is a general algorithm, which should use direct HTTP ca...
- 9 years ago
That looks like a bug that was fixed in version 2.1.2. Please make sure you're using the latest version of the SDK, and let me know if you're still seeing that.
michal_k
Explorer | Level 4
I modified the sample into an Unit Test, did the changes you advised, but now I keep getting this error as a result of uploadSessionFinishBatchCheck.
uploadSessionFinishBatchJobStatus={ ".tag" : "complete", "entries" : [ { ".tag" : "failure", "failure" : { ".tag" : "lookup_failed", "lookup_failed" : "not_closed" } } ] }
Here is the code, sorry for the messy state:
@Test public void chunkedUploadFiles() throws Exception { // Adjust the chunk size based on your network speed and reliability. Larger chunk sizes will // result in fewer network requests, which will be faster. But if an error occurs, the entire // chunk will be lost and have to be re-uploaded. Use a multiple of 4MiB for your chunk size. // final long CHUNKED_UPLOAD_CHUNK_SIZE = 8L << 20; // 8MiB final long CHUNKED_UPLOAD_CHUNK_SIZE = 15; //in Bytes final int CHUNKED_UPLOAD_MAX_ATTEMPTS = 5; final String FILE_UPLOADED_BY_PARTS = "FileUploadedByParts.txt"; String dropboxPath = getDropboxPath(FILE_UPLOADED_BY_PARTS); File localFile = createTempFileWithContent(FILE_UPLOADED_BY_PARTS, "The content of the file uploaded by parts 01"); //44B of text long size = localFile.length(); long uploaded = 0L; DbxException thrown = null; // Chunked uploads have 3 phases, each of which can accept uploaded bytes: // // (1) Start: initiate the upload and get an upload session ID // (2) Append: upload chunks of the file to append to our session // (3) Finish: commit the upload and close the session // // We track how many bytes we uploaded to determine which phase we should be in. String sessionId = null; for (int i = 0; i < CHUNKED_UPLOAD_MAX_ATTEMPTS; ++i) { if (i > 0) { System.out.printf("Retrying chunked upload (%d / %d attempts)\n", i + 1, CHUNKED_UPLOAD_MAX_ATTEMPTS); } try (InputStream in = new FileInputStream(localFile)) { // if this is a retry, make sure seek to the correct offset in.skip(uploaded); // (1) Start if (sessionId == null) { sessionId = tested.files().uploadSessionStart() .uploadAndFinish(in, CHUNKED_UPLOAD_CHUNK_SIZE) .getSessionId(); uploaded += CHUNKED_UPLOAD_CHUNK_SIZE; printProgress(uploaded, size); } UploadSessionCursor cursor = new UploadSessionCursor(sessionId, uploaded); // (2) Append while ((size - uploaded) > CHUNKED_UPLOAD_CHUNK_SIZE) { tested.files().uploadSessionAppendV2(cursor) .uploadAndFinish(in, CHUNKED_UPLOAD_CHUNK_SIZE); uploaded += CHUNKED_UPLOAD_CHUNK_SIZE; printProgress(uploaded, size); cursor = new UploadSessionCursor(sessionId, uploaded); } final long remaining = size - uploaded; if ((remaining) > 0) { tested.files().uploadSessionAppendV2(cursor) .uploadAndFinish(in, remaining); uploaded += remaining; printProgress(uploaded, size); cursor = new UploadSessionCursor(sessionId, uploaded); } // (3) Finish CommitInfo commitInfo = CommitInfo.newBuilder(dropboxPath) .withMode(WriteMode.ADD) .withClientModified(new Date(localFile.lastModified())) .build(); UploadSessionFinishArg entry = new UploadSessionFinishArg(cursor, commitInfo); final String asyncJobIdValue = tested.files().uploadSessionFinishBatch(newArrayList(entry)).getAsyncJobIdValue(); while (tested.files().uploadSessionFinishBatchCheck(asyncJobIdValue).isInProgress()) { Thread.sleep(1000); } final UploadSessionFinishBatchJobStatus uploadSessionFinishBatchJobStatus = tested.files().uploadSessionFinishBatchCheck( asyncJobIdValue); System.out.println("uploadSessionFinishBatchJobStatus=" + uploadSessionFinishBatchJobStatus.toStringMultiline()); return; } catch (RetryException ex) { thrown = ex; // RetryExceptions are never automatically retried by the client for uploads. Must // catch this exception even if DbxRequestConfig.getMaxRetries() > 0. sleepQuietly(ex.getBackoffMillis()); continue; } catch (NetworkIOException ex) { thrown = ex; // network issue with Dropbox (maybe a timeout?) try again continue; } catch (UploadSessionLookupErrorException ex) { if (ex.errorValue.isIncorrectOffset()) { thrown = ex; // server offset into the stream doesn't match our offset (uploaded). Seek to // the expected offset according to the server and try again. uploaded = ex.errorValue .getIncorrectOffsetValue() .getCorrectOffset(); continue; } else { // Some other error occurred, give up. System.err.println("Error uploading to Dropbox: " + ex.getMessage()); System.exit(1); return; } } catch (UploadSessionFinishErrorException ex) { if (ex.errorValue.isLookupFailed() && ex.errorValue.getLookupFailedValue().isIncorrectOffset()) { thrown = ex; // server offset into the stream doesn't match our offset (uploaded). Seek to // the expected offset according to the server and try again. uploaded = ex.errorValue .getLookupFailedValue() .getIncorrectOffsetValue() .getCorrectOffset(); continue; } else { // some other error occurred, give up. System.err.println("Error uploading to Dropbox: " + ex.getMessage()); System.exit(1); return; } } catch (DbxException ex) { System.err.println("Error uploading to Dropbox: " + ex.getMessage()); System.exit(1); return; } catch (IOException ex) { System.err.println("Error reading from file \"" + localFile + "\": " + ex.getMessage()); System.exit(1); return; } } // if we made it here, then we must have run out of attempts System.err.println("Maxed out upload attempts to Dropbox. Most recent error: " + thrown.getMessage()); System.exit(1); } private static void printProgress(long uploaded, long size) { System.out.printf("Uploaded %12d / %12d bytes (%5.2f%%)\n", uploaded, size, 100 * (uploaded / (double) size)); } private static void sleepQuietly(long millis) { try { Thread.sleep(millis); } catch (InterruptedException ex) { // just exit System.err.println("Error uploading to Dropbox: interrupted during backoff."); System.exit(1); } }
michal_k
9 years agoExplorer | Level 4
I checked the Javadoc and I think, that the problem was in this statement:
tested.files().uploadSessionAppendV2(cursor).uploadAndFinish(in, remaining);
There was the close argument missing, so I think, that correct statement should be this:
tested.files().uploadSessionAppendV2(cursor, true).uploadAndFinish(in, remaining);
The file is now uploaded to Dropbox, however I have a problem with following statement:
while (tested.files().uploadSessionFinishBatchCheck(asyncJobIdValue).isInProgress()) { Thread.sleep(1000); }
I get this exception:
java.lang.NoSuchMethodError: com.fasterxml.jackson.core.JsonParseException.<init>(Lcom/fasterxml/jackson/core/JsonParser;Ljava/lang/String;)V at com.dropbox.core.stone.StoneSerializer.expectField(StoneSerializer.java:84) at com.dropbox.core.v2.files.UploadSessionFinishBatchResultEntry$Serializer.deserialize(UploadSessionFinishBatchResultEntry.java:253) at com.dropbox.core.v2.files.UploadSessionFinishBatchResultEntry$Serializer.deserialize(UploadSessionFinishBatchResultEntry.java:205) at com.dropbox.core.stone.StoneSerializers$ListSerializer.deserialize(StoneSerializers.java:265) at com.dropbox.core.stone.StoneSerializers$ListSerializer.deserialize(StoneSerializers.java:244) at com.dropbox.core.v2.files.UploadSessionFinishBatchResult$Serializer.deserialize(UploadSessionFinishBatchResult.java:124) at com.dropbox.core.v2.files.UploadSessionFinishBatchJobStatus$Serializer.deserialize(UploadSessionFinishBatchJobStatus.java:236) at com.dropbox.core.v2.files.UploadSessionFinishBatchJobStatus$Serializer.deserialize(UploadSessionFinishBatchJobStatus.java:190) at com.dropbox.core.stone.StoneSerializer.deserialize(StoneSerializer.java:66) at com.dropbox.core.v2.DbxRawClientV2$1.execute(DbxRawClientV2.java:104) at com.dropbox.core.v2.DbxRawClientV2.executeRetriable(DbxRawClientV2.java:256) at com.dropbox.core.v2.DbxRawClientV2.rpcStyle(DbxRawClientV2.java:97) at com.dropbox.core.v2.files.DbxUserFilesRequests.uploadSessionFinishBatchCheck(DbxUserFilesRequests.java:1565) at com.dropbox.core.v2.files.DbxUserFilesRequests.uploadSessionFinishBatchCheck(DbxUserFilesRequests.java:1592) ...nothing interesting here
When I put breakpoint on the line StoneSerializer.java:84, I found out, that there are two problems, probably bugs.
First is, that StoneSerializer is not able to parse the response and second problem is the usage of new JsonParseException, where first argument should be message, not an instance of the JsonParser.
The actual exception message, that is suppressed by this incorrect usage in my case should be:
expected field 'success', but was: 'name'
About Dropbox API Support & Feedback
Find help with the Dropbox API from other developers.
5,912 PostsLatest Activity: 9 hours agoIf you need more help you can view your support options (expected response time for an email or ticket is 24 hours), or contact us on X or Facebook.
For more info on available support options for your Dropbox plan, see this article.
If you found the answer to your question in this Community thread, please 'like' the post to say thanks and to let us know it was useful!