[
https://issues.apache.org/jira/browse/PARQUET-383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16531536#comment-16531536
]
ASF GitHub Bot commented on PARQUET-383:
----------------------------------------
rgruener opened a new pull request #502: PARQUET-383: Add config for
propagating errors when writing metadata files
URL: https://github.com/apache/parquet-mr/pull/502
Allow for the option to not just eat errors when writing the metadata files.
The code isn't as clean as it could be but I was optimizing for making sure it
was 100% backwards compatible. The code would potentially be cleaner if we
rethrow a runtime exception in the catches of the original method though that
is not the most correct way to handle this.
Also I think it would be nice to not have another explicit config but rather
propagate errors if the user explicitly set the job summary level (meaning it
wasnt just the default) though that has the potential to break people who have
it explicitly set but do not expect it to ever error out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> ParquetOutputCommitter should propagate errors when writing metadata files
> --------------------------------------------------------------------------
>
> Key: PARQUET-383
> URL: https://issues.apache.org/jira/browse/PARQUET-383
> Project: Parquet
> Issue Type: Improvement
> Reporter: Alex Levenson
> Priority: Minor
> Labels: pull-request-available
>
> There's a lot of different ways the output committer can fail, or fail to
> rollback after failing to write metadata files. We should decide whether
> metadata files are required, and fatal (I think that's reasonable if the user
> asked for them), and propagate without squashing exceptions.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
