yuanlihan opened a new pull request #1569: Enable Partition Discovery When Loading Data from Parquet File URL: https://github.com/apache/incubator-doris/pull/1569 Currently, we support loading data from parquet file, but can not parse partition columns in the path of parquet file and can not recursively list all files under base path of input. This patch is able to discover and infer partitioning information under the base path of input like in Spark. It recursively list all the files under the base path and parse partition columns base on the base path if needed. This patch parse partition columns at BrokerScanNode.java and save parsing result of each file path as a property of TBrokerRangeDesc, then parquet_reader of BE can read the value of specified partition column.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org For additional commands, e-mail: dev-h...@doris.apache.org
