anujmodi2021 commented on PR #6270: URL: https://github.com/apache/hadoop/pull/6270#issuecomment-1873804668
> ok, you need to merge in trunk and it'll be good to go. > > Can you also add as a comment what the explicit text you want in the commit message. I'm thinking something like > > ``` > HADOOP-17347. ABFS: read/cache footer with fs.azure.footer.read.request.size > > The option fs.azure.footer.read.request.size sets the size of the footer to > read and cache; the default value of 524288 is good for parquet > > Contributed by ... > ``` > > side issue, I'm actually thinking of adding cross-stream caching in s3a fs, so that threads working on different row chunks can share footers. Some small LRU cache, with IOStats collection to help measure its value. > > thoughts? Thanks for the review @steveloughran . I have resolved merge conflicts and validated the changes by running the test suite again. The commit message you mentioned seems good. Slight changes can be done as following: ``` HADOOP-17347. ABFS: read/cache footer with fs.azure.footer.read.request.size The option fs.azure.footer.read.request.size sets the size of the footer to read and cache; the default value of 524288 is good for most analytic workloads running on parquet, ORC etc file formats. Contributed by ... ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
