This is an automated email from the ASF dual-hosted git repository.
viirya pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 5bfe46d4 fix: `ColumnReader.loadVector` should initiate
`CometDictionary` after re-import arrays (#473)
5bfe46d4 is described below
commit 5bfe46d4f1841985a85b74dc8e8ceaf10a8e3266
Author: Liang-Chi Hsieh <[email protected]>
AuthorDate: Sat May 25 14:05:02 2024 -0700
fix: `ColumnReader.loadVector` should initiate `CometDictionary` after
re-import arrays (#473)
---
.../main/java/org/apache/comet/parquet/ColumnReader.java | 14 +++++++-------
1 file changed, 7 insertions(+), 7 deletions(-)
diff --git a/common/src/main/java/org/apache/comet/parquet/ColumnReader.java
b/common/src/main/java/org/apache/comet/parquet/ColumnReader.java
index 46fd87f6..50991d5b 100644
--- a/common/src/main/java/org/apache/comet/parquet/ColumnReader.java
+++ b/common/src/main/java/org/apache/comet/parquet/ColumnReader.java
@@ -230,15 +230,15 @@ public class ColumnReader extends AbstractColumnReader {
// return plain vector.
currentVector = cometVector;
return currentVector;
- } else if (dictionary == null) {
- // There is dictionary from native side but the Java side dictionary
hasn't been
- // initialized yet.
- Dictionary arrowDictionary =
dictionaryProvider.lookup(dictionaryEncoding.getId());
- CometPlainVector dictionaryVector =
- new CometPlainVector(arrowDictionary.getVector(), useDecimal128,
isUuid);
- dictionary = new CometDictionary(dictionaryVector);
}
+ // We should already re-initiate `CometDictionary` here because
`Data.importVector` API will
+ // release the previous dictionary vector and create a new one.
+ Dictionary arrowDictionary =
dictionaryProvider.lookup(dictionaryEncoding.getId());
+ CometPlainVector dictionaryVector =
+ new CometPlainVector(arrowDictionary.getVector(), useDecimal128,
isUuid);
+ dictionary = new CometDictionary(dictionaryVector);
+
currentVector =
new CometDictionaryVector(
cometVector, dictionary, dictionaryProvider, useDecimal128,
false, isUuid);
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
