+1 for rollback. Thanks, Dongjoon.
On Mon, Oct 21, 2019 at 11:35 Owen O'Malley <[email protected]> wrote: > All, > TimestampColumnVector is awkwardly defined with both the millis since > 1970 and nanos with in a second. Hive's use is such that it doesn't matter > if the millis have the last three digits set or not. Spark however does > care. > > In ORC-306, we inadvertently changed the behavior of the ORC reader to set > the lower 3 digits of the millis. Previously it always had zeros. > > In SPARK-24322, they added compensating code such that they now depend on > the last three digits being non-zero. > > In ORC-546, we changed the semantics to the previous behavior (pre ORC-306) > to always have zeros. ORC-546 was released in 1.6.0 and was scheduled for > 1.5.7. > > Since ORC-546 requires additional changes in Spark and given that ORC 1.6.0 > isn't widely used yet, I'd like to roll back ORC-546. > > Concerns? > > .. Owen >
