mapleFU commented on issue #38818: URL: https://github.com/apache/arrow/issues/38818#issuecomment-1825196595
it depend on the impl of filter, I may dive into it later. Currently I found: nodict: ``` Write: ["0", "0", ...] The column will build a dict. dict: ["0"] indices: [0, 0, ... 0] min: "0" max: "0" ``` dict: ``` Write: dict: ["0", "1", ..."9999"], indices: [0, ..., 0] The column will reuse dict. dict: ["0", "1", ..."9999"] indices: [0, ..., 0] min: "0" max: "0" ``` I think the filter might try prune the RowGroup, the min-max is right, so they'll prune the same data. But when reading, the dict will use more time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
