maytasm commented on code in PR #13612:
URL: https://github.com/apache/druid/pull/13612#discussion_r1066523891
##########
extensions-core/parquet-extensions/example/compat/nested_array_struct.json:
##########
@@ -45,7 +45,8 @@
},
"dimensionsSpec": {
"dimensions": [
- "i32_dec"
+ "i32_dec",
+ "primitive"
Review Comment:
Some of our existing tests (such as
CompatParquetInputTest#testReadNestedArrayStruct) read parquet using this
configs. Reading is done via the RecordReader which used to read more than
needed before this PR. For example, in
CompatParquetInputTest#testReadNestedArrayStruct we expect the InputRow we read
to have `primitive` column. However, the spec (before this change) does not
need/require/consume the `primitive` column (`primitive` is not in the metric,
dimension, transform, etc). To fix the test, we can either make it so that the
spec requires the `primitive` (i.e. adding `primitive` to dimensionSpec) or
remove the assert check on `primitive` column. Basically, our Reader is smarter
and only read what is needed breaking some existing tests.
##########
extensions-core/parquet-extensions/example/compat/nested_array_struct.json:
##########
@@ -45,7 +45,8 @@
},
"dimensionsSpec": {
"dimensions": [
- "i32_dec"
+ "i32_dec",
+ "primitive"
Review Comment:
Some of our existing tests (such as
CompatParquetInputTest#testReadNestedArrayStruct) read parquet using this
configs. Reading is done via the RecordReader which used to read more than
needed before this PR. For example, in
CompatParquetInputTest#testReadNestedArrayStruct we expect the InputRow we read
to have `primitive` column. However, the spec (before this change) does not
need/require/consume the `primitive` column (`primitive` is not in the metric,
dimension, transform, etc). To fix the test, we can either make it so that the
spec requires the `primitive` (i.e. adding `primitive` to dimensionSpec) or
remove the assert check on `primitive` column. Basically, our Reader is more
efficient and only read what is needed breaking some existing tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
