anujmodi2021 commented on PR #8153: URL: https://github.com/apache/hadoop/pull/8153#issuecomment-3723523348
> @anujmodi2021 I am trying to propose a single optimised implementation of an input stream across cloud implementations, as I think we all need this kind of logic. Ideally I want to get to a place where 80% of the logic is shared in a common layer, and then we only implement cloud specific clients to actually make the requests separately. > > There is some consensus to move the shared logic into the parquet-java repo: https://lists.apache.org/thread/nbksq32cs8h1ldj8762y6wh9zzp8gqx6 , and some buy-in from the team at google. I'll be following up on this in the new year. > > Would be great to get your thoughts and if your team would also like to collaborate on this. Thanks for heads up @ahmarsuhail This sounds like a good plan to me as well. We will surely keep a close eye on the updates on this thread and try to contribute to make things better in best way possible. With this change we are not chaning how ABFS handles parquet file though. This just improves the infra and add capability for future improvements to be plugged in seemlessly. We will surely help address any gaps in ABFS to make things better for the common ground you are gearing up to improve. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
