Hi Swapnil, I don't think it is a good idea for Parquet to implement casting at this level. Parquet should return the data that was stored and make it possible for higher-level components, like record materialization, to coerce types.
rb On Mon, Feb 25, 2019 at 11:48 AM Swapnil Chougule <[email protected]> wrote: > Hi Folks, > > Abstract class Dictionary contains methods: > public Binary decodeToBinary(int id) > public int decodeToInt(int id) > public long decodeToLong(int id) > public float decodeToFloat(int id) > public double decodeToDouble(int id) > public boolean decodeToBoolean(int id) > > These are subsequently overridden in respective dictionary implementation > like > > PlainLongDictionary overrides "decodeToLong" method only > PlainIntegerDictionary overrides "decodeToInt" method only > & so on > > Can we support type upcasting here ? > PlainLongDictionary overrides "decodeToLong" & "decodeToDouble" methods > PlainIntegerDictionary overrides "decodeToInt", "decodeToLong" & > "decodeToDouble" methods > > Type up casting is valid use case. > It also needs some changes in ValidTypeMap.java & > SchemaCompatibilityValidator.java for Filter predicate. > > Can parquet support this type upcasting feature? I came across such > scenario in one of my use case. > > Thanks, > Swapnil > -- Ryan Blue Software Engineer Netflix
