widening primitive conversion in parquet dictionary

[Swapnil Chougule]

Hi

Is there widening primitive conversion support in parquet dictionary?

I could see only same type methods are implemented in dictionary
https://github.com/apache/parquet-mr/blob/master/parquet-column/src/main/java/org/apache/parquet/column/values/dictionary/PlainValuesDictionary.java

I came across a case where long data needs to be read as double.
PlainLongDictionary is being created for same. This dictionary has only
implementation for 'decodeToLong'.
Can we have 'decodeToDouble' implentation as well here? (as long to double
is widening primitive conversion). Same scenarios can be replicated for
other supported(widening primitive conversion) types.

Thanks,
Swapnil

Overridden methods of dictionary

[Swapnil Chougule]

Hi Folks,

Abstract class Dictionary contains methods:
public Binary decodeToBinary(int id)
public int decodeToInt(int id)
public long decodeToLong(int id)
public float decodeToFloat(int id)
public double decodeToDouble(int id)
public boolean decodeToBoolean(int id)

These are subsequently overridden in respective dictionary implementation
like

PlainLongDictionary overrides "decodeToLong" method only
PlainIntegerDictionary overrides "decodeToInt" method only
& so on

Can we support type upcasting here ?
PlainLongDictionary overrides "decodeToLong" & "decodeToDouble" methods
PlainIntegerDictionary overrides "decodeToInt", "decodeToLong" &
"decodeToDouble" methods

Type up casting is valid use case.
It also needs some changes in ValidTypeMap.java &
SchemaCompatibilityValidator.java for Filter predicate.

Can parquet support this type upcasting feature? I came across such
scenario in one of my use case.

Thanks,
Swapnil