Aleksandr Efimov created IMPALA-15458:
-----------------------------------------
Summary: HBO: input_rows of a sampled scan counts whole partitions
Key: IMPALA-15458
URL: https://issues.apache.org/jira/browse/IMPALA-15458
Project: IMPALA
Issue Type: Improvement
Components: Frontend
Reporter: Aleksandr Efimov
{{HdfsScanNode.appendScanInputStats()}} sets {{input_rows}} to
{{getNumInputRows()}}, the sum of {{numRows}} over
{{getSampledOrRawPartitions()}}. For a sampled scan ({{TABLESAMPLE}}) these are
all partitions with at least one sampled file. When only some files of a
partition are sampled, {{input_rows}} is higher than the number of rows the
scan reads.
IMPALA-15394 puts sampled scans under their own HBO key ({{<SAMPLE:percent>}}),
so for now this only affects how runs of sampled scans match each other. With a
correct {{input_rows}} (for example, scaled by the share of sampled bytes), HBO
could use the rows of a sampled run to estimate the full scan, and the
{{<SAMPLE>}} marker could be removed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
