rdblue commented on a change in pull request #1957:
URL: https://github.com/apache/iceberg/pull/1957#discussion_r546003880
##########
File path:
spark/src/main/java/org/apache/iceberg/spark/source/BaseDataReader.java
##########
@@ -77,16 +78,30 @@
}
public boolean next() throws IOException {
- while (true) {
- if (currentIterator.hasNext()) {
- this.current = currentIterator.next();
- return true;
- } else if (tasks.hasNext()) {
- this.currentIterator.close();
- this.currentIterator = open(tasks.next());
+ try {
+ while (true) {
+ if (currentIterator.hasNext()) {
+ this.current = currentIterator.next();
+ return true;
+ } else if (tasks.hasNext()) {
+ this.currentIterator.close();
+ this.currentTask = tasks.next();
+ this.currentIterator = open(currentTask);
+ } else {
+ this.currentIterator.close();
+ return false;
+ }
+ }
+ } catch (IOException | RuntimeException e) {
+ if (currentTask == null || currentTask.isDataTask()) {
+ throw e;
} else {
- this.currentIterator.close();
- return false;
+ String message = String.format("Error reading file: %s",
getInputFile(currentTask).location());
+ if (e instanceof IOException) {
+ throw new IOException(message, e);
+ } else {
+ throw new RuntimeException(message, e);
+ }
Review comment:
The main problem I have with this is that it alters the exception by
discarding its type. Callers can no longer catch exceptions and handle errors.
For example, a caller might catch a `SocketException` and retry, but let an
`EOFException` propagate.
Is there another way to attach the data? What about logging the file path
and exception instead of modifying the exception?
Another option is to create a new exception and attach it as suppressed,
then throw the original unmodified.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
