[
https://issues.apache.org/jira/browse/DRILL-8416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706924#comment-17706924
]
James Turton commented on DRILL-8416:
-------------------------------------
I can't say a lot for certain yet but I think that the conditions needed to
reveal the bug are not often present in real world Parquet files. Which is not
to say that the Parquet files are invalid, just that I think that their empty
dictionary pages are uncommon.
> Memory leak during quering parquet file containing null timestamps
> ------------------------------------------------------------------
>
> Key: DRILL-8416
> URL: https://issues.apache.org/jira/browse/DRILL-8416
> Project: Apache Drill
> Issue Type: Bug
> Components: Storage - Parquet
> Affects Versions: 1.21.0
> Reporter: Matthias Rosenthaler
> Priority: Major
> Fix For: 1.21.1
>
> Attachments: example.parquet, meta_steps.parquet
>
>
> If I try to query (
> {code:java}
> SELECT * FROM
> `hdfs.data`.`./v2/meta_steps/me-2023-03-20-13-15-30-inv230021-kontrollsystemf39st9qrx20-03-2/meta_steps.parquet`{code}
> ) the following parquet file which is stored on hadoop file system I am
> getting the following error:
> {code:java}
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (64)
> Allocator(op:0:0:1:ParquetRowGroupScan) 1000000/64/34688/10000000000
> (res/actual/peak/limit){code}
> Everything is working fine with drill version 1.19.
> If I select only columns without NULL values, the query also works in 1.21.0:
> {code:java}
> SELECT `name`,`type` FROM
> `hdfs.data`.`./v2/meta_steps/me-2023-03-20-13-15-30-inv230021-kontrollsystemf39st9qrx20-03-2/meta_steps.parquet`{code}
> Generated a new example.parquet with pyarrow 8.0.0 and a float column with
> NULL valuues and the same error happened.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
