htran1 commented on a change in pull request #2868: GOBBLIN-1025: Add retry for
PK-Chuking iterator
URL: https://github.com/apache/incubator-gobblin/pull/2868#discussion_r367159350
##########
File path:
gobblin-salesforce/src/main/java/org/apache/gobblin/salesforce/SalesforceRecordIterator.java
##########
@@ -99,7 +103,40 @@ private void fulfillCurrentRecord() {
}
}
- private void resetCurrentRecordStatus() {
+ private InputStreamCSVReader reopenStreamWithRetry(ResultStruct
resultStruct, int seekLineNumber, int retryNumber, int count) {
Review comment:
Can you refactor the logic in
`SalesforceExtractor.reinitializeBufferedReader` and use it here? That has an
additional validation to make sure that after skipping we arrived at the
expected location based on the last record read before the failure.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
