umehrot2 commented on pull request #2064: URL: https://github.com/apache/hudi/pull/2064#issuecomment-698068512
> @umehrot2 I have updated the RFC doc with [details on how to test RFC-15](https://cwiki.apache.org/confluence/display/HUDI/RFC+-+15%3A+HUDI+File+Listing+and+Query+Planning+Improvements#RFC15:HUDIFileListingandQueryPlanningImprovements-TestPlan). Please take a look and let me know if I can help in any way. @prashantwason I missed your earlier pings on this PR. We will start the testing of this PR with S3. Looking at the testing details you provided, I am a bit confused, and have a couple of questions: - Do we need to directly use `HoodieWriteClient` from `spark-shell` to be able to test this ? Can't we directly use `hudi datasource` or `delta streamer` for testing with following options set to true while writing: `hoodie.metadata.file.listings.enable`, `hoodie.metadata.file.listings.verify` ? - Also for testing query performance using `spark datasource`, `spark-sql`, `hive` and `presto` I am assuming it will detect that metadata table is present and automatically use that for getting the list ? Will review the PR this week as well to understand the implementation details. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
