[
https://issues.apache.org/jira/browse/HUDI-3486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Guo reassigned HUDI-3486:
-------------------------------
Assignee: Ethan Guo
> Value and null count in metadata col stats from metadata table are wrong
> ------------------------------------------------------------------------
>
> Key: HUDI-3486
> URL: https://issues.apache.org/jira/browse/HUDI-3486
> Project: Apache Hudi
> Issue Type: Bug
> Components: metadata
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.11.0
>
> Attachments: Screen Shot 2022-02-22 at 16.07.32.png
>
>
> Deltastreamer continuous mode writing MOR table, async compaction, cleaner
> enabled
> Column stats validation fails after about 10 delta commits as below. The
> only mismatched fields are nullCount and valueCount in
> HoodieColumnRangeMetadata.
> (Below screenshot showing diff of col stats: left: from metadata table,
> right: from base files (ground truth))
> !Screen Shot 2022-02-22 at 16.07.32.png|width=981,height=225!
> {code:java}
> 22/02/22 14:18:27 ERROR HoodieMetadataTableValidator: Validation of column
> stats from metadata table for partition 2022/2/1 failed. Column stats from
> metadata table: [HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_commit_seqno',
> minValue=20220221113858130_7_300214, maxValue=20220221115708846_7_5831739,
> nullCount=0, valueCount=290126, totalSize=893085,
> totalUncompressedSize=8963480}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_commit_time',
> minValue=20220221113858130, maxValue=20220221115708846, nullCount=0,
> valueCount=290126, totalSize=81703, totalUncompressedSize=118411},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_file_name',
>
> minValue=0e1a2619-234c-4810-a972-df38e1be44ab-0_7-132-1259_20220221114105472.parquet,
>
> maxValue=0e1a2619-234c-4810-a972-df38e1be44ab-0_7-84-815_20220221113956665.parquet,
> nullCount=0, valueCount=290126, totalSize=81824,
> totalUncompressedSize=117875}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_partition_path',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=290126,
> totalSize=876, totalUncompressedSize=586}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_record_key',
> minValue=000-4000-0921-e08d5cd6-8143-9db2e9964bcd,
> maxValue=007-4fff-f389-fc6afef2-b313-03bc4c43019a, nullCount=0,
> valueCount=290126, totalSize=6272686, totalUncompressedSize=12766098},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='decimalField',
> minValue=1.1456013E-4, maxValue=0.99988973, nullCount=0, valueCount=290126,
> totalSize=544516, totalUncompressedSize=548752},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='key',
> minValue=000-4000-0921-e08d5cd6-8143-9db2e9964bcd,
> maxValue=007-4fff-f389-fc6afef2-b313-03bc4c43019a, nullCount=0,
> valueCount=290126, totalSize=6272686, totalUncompressedSize=12766098},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='longField',
> minValue=-9219537056511747588, maxValue=9223251540117729269, nullCount=0,
> valueCount=290126, totalSize=588561, totalUncompressedSize=588620},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='partition',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=290126,
> totalSize=876, totalUncompressedSize=586}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='round',
> minValue=0, maxValue=8, nullCount=0, valueCount=290126, totalSize=81655,
> totalUncompressedSize=118256}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='textField',
>
> minValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
>
> maxValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
> nullCount=0, valueCount=290126, totalSize=3085, totalUncompressedSize=2180},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='ts',
> minValue=1643951583449, maxValue=1643951723138, nullCount=0,
> valueCount=290126, totalSize=605298, totalUncompressedSize=812594},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_commit_seqno',
> minValue=20220221115206973_17_5083814,
> maxValue=20220221120030453_14_6820795, nullCount=108549, valueCount=0,
> totalSize=313191, totalUncompressedSize=3473790},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_commit_time',
> minValue=20220221115206973, maxValue=20220221120030453, nullCount=108549,
> valueCount=0, totalSize=13672, totalUncompressedSize=18949},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_file_name',
>
> minValue=399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-440-4091_20220221115531106.parquet,
>
> maxValue=399637df-7275-4e7f-a0d2-8b79363ddafd-0_17-390-3580_20220221115206973.parquet,
> nullCount=108549, valueCount=0, totalSize=14110,
> totalUncompressedSize=19335}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_partition_path',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=108549, valueCount=0,
> totalSize=380, totalUncompressedSize=253}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_record_key',
> minValue=007-4000-4b77-1d2a3db1-b9c9-d66be48153cd,
> maxValue=010-4fff-f17e-5ccb8125-a33b-ac5d41abc447, nullCount=108549,
> valueCount=0, totalSize=2347317, totalUncompressedSize=4776378},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='decimalField',
> minValue=1.6975403E-4, maxValue=0.99988973, nullCount=108549, valueCount=0,
> totalSize=202011, totalUncompressedSize=204770},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='key',
> minValue=007-4000-4b77-1d2a3db1-b9c9-d66be48153cd,
> maxValue=010-4fff-f17e-5ccb8125-a33b-ac5d41abc447, nullCount=108549,
> valueCount=0, totalSize=2347317, totalUncompressedSize=4776378},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='longField',
> minValue=-9219537056511747588, maxValue=9223251540117729269,
> nullCount=108549, valueCount=0, totalSize=232857,
> totalUncompressedSize=232682}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='partition',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=108549, valueCount=0,
> totalSize=380, totalUncompressedSize=253}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='round',
> minValue=7, maxValue=10, nullCount=108549, valueCount=0, totalSize=13637,
> totalUncompressedSize=18879}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='textField',
>
> minValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
>
> maxValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
> nullCount=108549, valueCount=0, totalSize=1216, totalUncompressedSize=959},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='ts',
> minValue=1643951716466, maxValue=1643951788013, nullCount=108549,
> valueCount=0, totalSize=218937, totalUncompressedSize=299435}]Column stats
> from base files: [HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_commit_seqno',
> minValue=20220221113858130_7_300214, maxValue=20220221115708846_7_5831739,
> nullCount=0, valueCount=290126, totalSize=893085,
> totalUncompressedSize=8963480}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_commit_time',
> minValue=20220221113858130, maxValue=20220221115708846, nullCount=0,
> valueCount=290126, totalSize=81703, totalUncompressedSize=118411},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_file_name',
>
> minValue=0e1a2619-234c-4810-a972-df38e1be44ab-0_7-132-1259_20220221114105472.parquet,
>
> maxValue=0e1a2619-234c-4810-a972-df38e1be44ab-0_7-84-815_20220221113956665.parquet,
> nullCount=0, valueCount=290126, totalSize=81824,
> totalUncompressedSize=117875}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_partition_path',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=290126,
> totalSize=876, totalUncompressedSize=586}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_record_key',
> minValue=000-4000-0921-e08d5cd6-8143-9db2e9964bcd,
> maxValue=007-4fff-f389-fc6afef2-b313-03bc4c43019a, nullCount=0,
> valueCount=290126, totalSize=6272686, totalUncompressedSize=12766098},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='decimalField',
> minValue=1.1456013E-4, maxValue=0.99988973, nullCount=0, valueCount=290126,
> totalSize=544516, totalUncompressedSize=548752},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='key',
> minValue=000-4000-0921-e08d5cd6-8143-9db2e9964bcd,
> maxValue=007-4fff-f389-fc6afef2-b313-03bc4c43019a, nullCount=0,
> valueCount=290126, totalSize=6272686, totalUncompressedSize=12766098},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='longField',
> minValue=-9219537056511747588, maxValue=9223251540117729269, nullCount=0,
> valueCount=290126, totalSize=588561, totalUncompressedSize=588620},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='partition',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=290126,
> totalSize=876, totalUncompressedSize=586}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='round',
> minValue=0, maxValue=8, nullCount=0, valueCount=290126, totalSize=81655,
> totalUncompressedSize=118256}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='textField',
>
> minValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
>
> maxValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
> nullCount=0, valueCount=290126, totalSize=3085, totalUncompressedSize=2180},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='ts',
> minValue=1643951583449, maxValue=1643951723138, nullCount=0,
> valueCount=290126, totalSize=605298, totalUncompressedSize=812594},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_commit_seqno',
> minValue=20220221115206973_17_5083814,
> maxValue=20220221120030453_14_6820795, nullCount=0, valueCount=108549,
> totalSize=313191, totalUncompressedSize=3473790},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_commit_time',
> minValue=20220221115206973, maxValue=20220221120030453, nullCount=0,
> valueCount=108549, totalSize=13672, totalUncompressedSize=18949},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_file_name',
>
> minValue=399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-440-4091_20220221115531106.parquet,
>
> maxValue=399637df-7275-4e7f-a0d2-8b79363ddafd-0_17-390-3580_20220221115206973.parquet,
> nullCount=0, valueCount=108549, totalSize=14110,
> totalUncompressedSize=19335}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_partition_path',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=108549,
> totalSize=380, totalUncompressedSize=253}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_record_key',
> minValue=007-4000-4b77-1d2a3db1-b9c9-d66be48153cd,
> maxValue=010-4fff-f17e-5ccb8125-a33b-ac5d41abc447, nullCount=0,
> valueCount=108549, totalSize=2347317, totalUncompressedSize=4776378},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='decimalField',
> minValue=1.6975403E-4, maxValue=0.99988973, nullCount=0, valueCount=108549,
> totalSize=202011, totalUncompressedSize=204770},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='key',
> minValue=007-4000-4b77-1d2a3db1-b9c9-d66be48153cd,
> maxValue=010-4fff-f17e-5ccb8125-a33b-ac5d41abc447, nullCount=0,
> valueCount=108549, totalSize=2347317, totalUncompressedSize=4776378},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='longField',
> minValue=-9219537056511747588, maxValue=9223251540117729269, nullCount=0,
> valueCount=108549, totalSize=232857, totalUncompressedSize=232682},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='partition',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=108549,
> totalSize=380, totalUncompressedSize=253}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='round',
> minValue=7, maxValue=10, nullCount=0, valueCount=108549, totalSize=13637,
> totalUncompressedSize=18879}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='textField',
>
> minValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
>
> maxValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
> nullCount=0, valueCount=108549, totalSize=1216, totalUncompressedSize=959},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='ts',
> minValue=1643951716466, maxValue=1643951788013, nullCount=0,
> valueCount=108549, totalSize=218937, totalUncompressedSize=299435}]
> 22/02/22 14:18:27 ERROR HoodieMetadataTableValidator: Metadata table
> validation failed for 2022/2/1 due to HoodieValidationException
> org.apache.hudi.exception.HoodieValidationException: Validation of column
> stats from metadata table for partition 2022/2/1 failed. Column stats from
> metadata table: [HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_commit_seqno',
> minValue=20220221113858130_7_300214, maxValue=20220221115708846_7_5831739,
> nullCount=0, valueCount=290126, totalSize=893085,
> totalUncompressedSize=8963480}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_commit_time',
> minValue=20220221113858130, maxValue=20220221115708846, nullCount=0,
> valueCount=290126, totalSize=81703, totalUncompressedSize=118411},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_file_name',
>
> minValue=0e1a2619-234c-4810-a972-df38e1be44ab-0_7-132-1259_20220221114105472.parquet,
>
> maxValue=0e1a2619-234c-4810-a972-df38e1be44ab-0_7-84-815_20220221113956665.parquet,
> nullCount=0, valueCount=290126, totalSize=81824,
> totalUncompressedSize=117875}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_partition_path',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=290126,
> totalSize=876, totalUncompressedSize=586}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_record_key',
> minValue=000-4000-0921-e08d5cd6-8143-9db2e9964bcd,
> maxValue=007-4fff-f389-fc6afef2-b313-03bc4c43019a, nullCount=0,
> valueCount=290126, totalSize=6272686, totalUncompressedSize=12766098},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='decimalField',
> minValue=1.1456013E-4, maxValue=0.99988973, nullCount=0, valueCount=290126,
> totalSize=544516, totalUncompressedSize=548752},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='key',
> minValue=000-4000-0921-e08d5cd6-8143-9db2e9964bcd,
> maxValue=007-4fff-f389-fc6afef2-b313-03bc4c43019a, nullCount=0,
> valueCount=290126, totalSize=6272686, totalUncompressedSize=12766098},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='longField',
> minValue=-9219537056511747588, maxValue=9223251540117729269, nullCount=0,
> valueCount=290126, totalSize=588561, totalUncompressedSize=588620},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='partition',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=290126,
> totalSize=876, totalUncompressedSize=586}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='round',
> minValue=0, maxValue=8, nullCount=0, valueCount=290126, totalSize=81655,
> totalUncompressedSize=118256}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='textField',
>
> minValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
>
> maxValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
> nullCount=0, valueCount=290126, totalSize=3085, totalUncompressedSize=2180},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='ts',
> minValue=1643951583449, maxValue=1643951723138, nullCount=0,
> valueCount=290126, totalSize=605298, totalUncompressedSize=812594},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_commit_seqno',
> minValue=20220221115206973_17_5083814,
> maxValue=20220221120030453_14_6820795, nullCount=108549, valueCount=0,
> totalSize=313191, totalUncompressedSize=3473790},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_commit_time',
> minValue=20220221115206973, maxValue=20220221120030453, nullCount=108549,
> valueCount=0, totalSize=13672, totalUncompressedSize=18949},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_file_name',
>
> minValue=399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-440-4091_20220221115531106.parquet,
>
> maxValue=399637df-7275-4e7f-a0d2-8b79363ddafd-0_17-390-3580_20220221115206973.parquet,
> nullCount=108549, valueCount=0, totalSize=14110,
> totalUncompressedSize=19335}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_partition_path',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=108549, valueCount=0,
> totalSize=380, totalUncompressedSize=253}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_record_key',
> minValue=007-4000-4b77-1d2a3db1-b9c9-d66be48153cd,
> maxValue=010-4fff-f17e-5ccb8125-a33b-ac5d41abc447, nullCount=108549,
> valueCount=0, totalSize=2347317, totalUncompressedSize=4776378},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='decimalField',
> minValue=1.6975403E-4, maxValue=0.99988973, nullCount=108549, valueCount=0,
> totalSize=202011, totalUncompressedSize=204770},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='key',
> minValue=007-4000-4b77-1d2a3db1-b9c9-d66be48153cd,
> maxValue=010-4fff-f17e-5ccb8125-a33b-ac5d41abc447, nullCount=108549,
> valueCount=0, totalSize=2347317, totalUncompressedSize=4776378},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='longField',
> minValue=-9219537056511747588, maxValue=9223251540117729269,
> nullCount=108549, valueCount=0, totalSize=232857,
> totalUncompressedSize=232682}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='partition',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=108549, valueCount=0,
> totalSize=380, totalUncompressedSize=253}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='round',
> minValue=7, maxValue=10, nullCount=108549, valueCount=0, totalSize=13637,
> totalUncompressedSize=18879}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='textField',
>
> minValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
>
> maxValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
> nullCount=108549, valueCount=0, totalSize=1216, totalUncompressedSize=959},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='ts',
> minValue=1643951716466, maxValue=1643951788013, nullCount=108549,
> valueCount=0, totalSize=218937, totalUncompressedSize=299435}]Column stats
> from base files: [HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_commit_seqno',
> minValue=20220221113858130_7_300214, maxValue=20220221115708846_7_5831739,
> nullCount=0, valueCount=290126, totalSize=893085,
> totalUncompressedSize=8963480}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_commit_time',
> minValue=20220221113858130, maxValue=20220221115708846, nullCount=0,
> valueCount=290126, totalSize=81703, totalUncompressedSize=118411},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_file_name',
>
> minValue=0e1a2619-234c-4810-a972-df38e1be44ab-0_7-132-1259_20220221114105472.parquet,
>
> maxValue=0e1a2619-234c-4810-a972-df38e1be44ab-0_7-84-815_20220221113956665.parquet,
> nullCount=0, valueCount=290126, totalSize=81824,
> totalUncompressedSize=117875}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_partition_path',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=290126,
> totalSize=876, totalUncompressedSize=586}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='_hoodie_record_key',
> minValue=000-4000-0921-e08d5cd6-8143-9db2e9964bcd,
> maxValue=007-4fff-f389-fc6afef2-b313-03bc4c43019a, nullCount=0,
> valueCount=290126, totalSize=6272686, totalUncompressedSize=12766098},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='decimalField',
> minValue=1.1456013E-4, maxValue=0.99988973, nullCount=0, valueCount=290126,
> totalSize=544516, totalUncompressedSize=548752},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='key',
> minValue=000-4000-0921-e08d5cd6-8143-9db2e9964bcd,
> maxValue=007-4fff-f389-fc6afef2-b313-03bc4c43019a, nullCount=0,
> valueCount=290126, totalSize=6272686, totalUncompressedSize=12766098},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='longField',
> minValue=-9219537056511747588, maxValue=9223251540117729269, nullCount=0,
> valueCount=290126, totalSize=588561, totalUncompressedSize=588620},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='partition',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=290126,
> totalSize=876, totalUncompressedSize=586}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='round',
> minValue=0, maxValue=8, nullCount=0, valueCount=290126, totalSize=81655,
> totalUncompressedSize=118256}, HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='textField',
>
> minValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
>
> maxValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
> nullCount=0, valueCount=290126, totalSize=3085, totalUncompressedSize=2180},
> HoodieColumnRangeMetadata{filePath
> ='0e1a2619-234c-4810-a972-df38e1be44ab-0_7-472-4237_20220221115708846.parquet'columnName='ts',
> minValue=1643951583449, maxValue=1643951723138, nullCount=0,
> valueCount=290126, totalSize=605298, totalUncompressedSize=812594},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_commit_seqno',
> minValue=20220221115206973_17_5083814,
> maxValue=20220221120030453_14_6820795, nullCount=0, valueCount=108549,
> totalSize=313191, totalUncompressedSize=3473790},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_commit_time',
> minValue=20220221115206973, maxValue=20220221120030453, nullCount=0,
> valueCount=108549, totalSize=13672, totalUncompressedSize=18949},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_file_name',
>
> minValue=399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-440-4091_20220221115531106.parquet,
>
> maxValue=399637df-7275-4e7f-a0d2-8b79363ddafd-0_17-390-3580_20220221115206973.parquet,
> nullCount=0, valueCount=108549, totalSize=14110,
> totalUncompressedSize=19335}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_partition_path',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=108549,
> totalSize=380, totalUncompressedSize=253}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='_hoodie_record_key',
> minValue=007-4000-4b77-1d2a3db1-b9c9-d66be48153cd,
> maxValue=010-4fff-f17e-5ccb8125-a33b-ac5d41abc447, nullCount=0,
> valueCount=108549, totalSize=2347317, totalUncompressedSize=4776378},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='decimalField',
> minValue=1.6975403E-4, maxValue=0.99988973, nullCount=0, valueCount=108549,
> totalSize=202011, totalUncompressedSize=204770},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='key',
> minValue=007-4000-4b77-1d2a3db1-b9c9-d66be48153cd,
> maxValue=010-4fff-f17e-5ccb8125-a33b-ac5d41abc447, nullCount=0,
> valueCount=108549, totalSize=2347317, totalUncompressedSize=4776378},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='longField',
> minValue=-9219537056511747588, maxValue=9223251540117729269, nullCount=0,
> valueCount=108549, totalSize=232857, totalUncompressedSize=232682},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='partition',
> minValue=2022/2/1, maxValue=2022/2/1, nullCount=0, valueCount=108549,
> totalSize=380, totalUncompressedSize=253}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='round',
> minValue=7, maxValue=10, nullCount=0, valueCount=108549, totalSize=13637,
> totalUncompressedSize=18879}, HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='textField',
>
> minValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
>
> maxValue=abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz|abcdefghijklmnopqrstuvwxyz,
> nullCount=0, valueCount=108549, totalSize=1216, totalUncompressedSize=959},
> HoodieColumnRangeMetadata{filePath
> ='399637df-7275-4e7f-a0d2-8b79363ddafd-0_14-561-5208_20220221120030453.parquet'columnName='ts',
> minValue=1643951716466, maxValue=1643951788013, nullCount=0,
> valueCount=108549, totalSize=218937, totalUncompressedSize=299435}] {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
