szlta opened a new pull request #3248:
URL: https://github.com/apache/iceberg/pull/3248
Currently there are 3 struct-like data holder types that ORC integrates with
in Iceberg: Record (Iceberg), RowData (Flink) and InternalRow (Spark).
While there's an OrcRowWriter<T> implementation for all these 3 types, only
the first two relies on OrcValueWriter<?>'s, while Spark uses SparkValueWriters
and that has a different API ("int ordinal" is part of nonNullWrite method).
Also there's no common abstract StuctWriter<?> class which would be similar
to how Parquet/Avro writes are implemented (e.g. see
https://github.com/apache/iceberg/blob/master/parquet/src/main/java/org/apache/iceberg/parquet/ParquetValueWriters.java#L539
)
Due to all that, the current ORC write code has a lot of code duplication,
and fits less into Iceberg, where the write code paths for Parquet and Avro are
much more in harmony.
Thus I propose to bring ORC writes up to the standard too, as we can already
see some effects of this shortcoming in this PR:
https://github.com/apache/iceberg/pull/2935 where a common StructWriter<?> type
could help implement positional delete writer creation for ORC.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
