JingsongLi commented on PR #477: URL: https://github.com/apache/paimon-rust/pull/477#issuecomment-4954639816
The dedicated bloom/vector table will incorrectly crop data. A normal Parquet file only writes some fields, but 'writable' equals' None '; When the statistics cover all 'normal fields',' value_dats_coms' is also 'None'. The reading end therefore performs position mapping according to the complete table schema. Tested the schema '[payload (blob), a, b]' and data 'a=100, b=0', filtered the statistics that 'a>50' would cause misreading of 'b', and cut the file to 'finalization files=0'. The Java implementation will write 'normalColumnNames' to' writable Cols'. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
