paul-rogers commented on a change in pull request #1955: DRILL-7491: Incorrect
count() returned for complex types in parquet
URL: https://github.com/apache/drill/pull/1955#discussion_r366671949
##########
File path:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/base/AbstractGroupScanWithMetadata.java
##########
@@ -167,29 +167,46 @@ public boolean isMatchAllMetadata() {
*/
@Override
public long getColumnValueCount(SchemaPath column) {
- long tableRowCount, colNulls;
- Long nulls;
ColumnStatistics<?> columnStats =
getTableMetadata().getColumnStatistics(column);
- ColumnStatistics<?> nonInterestingColStats = null;
- if (columnStats == null) {
- nonInterestingColStats =
getNonInterestingColumnsMetadata().getColumnStatistics(column);
- }
+ ColumnStatistics<?> nonInterestingColStats = columnStats == null
+ ? getNonInterestingColumnsMetadata().getColumnStatistics(column) :
null;
+ long tableRowCount;
if (columnStats != null) {
tableRowCount =
TableStatisticsKind.ROW_COUNT.getValue(getTableMetadata());
} else if (nonInterestingColStats != null) {
tableRowCount =
TableStatisticsKind.ROW_COUNT.getValue(getNonInterestingColumnsMetadata());
Review comment:
Having a hard time understanding this. If a column is uninteresing, we get
the row count from the non-interesting columns metadata. Seems round-about.
Should we get the row count from the table itself? That is, the indirection
through non-interesting columns to get table metadata seems awkward.
Also, if we do have column stats, we get the row count from the table
metadata. This raises the question: by column value count, do we mean NDV
(number of distinct values)? Otherwise, the column value count for top-level
columns is defined as the same as the row count whether the column is
interesting or not.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
