steveloughran commented on PR #1139: URL: https://github.com/apache/parquet-mr/pull/1139#issuecomment-1976469040
quick updatge on this: in https://issues.apache.org/jira/browse/HADOOP-19098 I'm going to declare that overlapping ranges triggers IllegalArgumentException everywhere; to date only s3a threw exceptions. For the parquet PR I will * add the same safety checks so that it is consistent with older releases * with tests Now, what do people think about fallbacks? we've been using this and to date parquet hasn't ever issued an overlapping request, but there's still the future to think about. How about, when hadoop vector IO requests fail during parquet-side range checks, there's something logged (level?) and fallback to classic PositionedReadable. That way, jobs will always succeed the way they used to. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
