tustvold commented on code in PR #1746:
URL: https://github.com/apache/arrow-rs/pull/1746#discussion_r883107978
##########
parquet/src/arrow/arrow_writer.rs:
##########
@@ -341,26 +341,24 @@ fn write_leaf(
column: &ArrayRef,
levels: LevelInfo,
) -> Result<i64> {
- let indices = levels.filter_array_indices();
- // Slice array according to computed offset and length
- let column = column.slice(levels.offset, levels.length);
+ // TODO: Avoid filtering if no need
Review Comment:
This is actually a hold over from an intermediate version of this PR, you
can only avoid doing the "take" operation if there are no-nulls which is
relatively rare.
##########
parquet/src/arrow/arrow_writer.rs:
##########
@@ -341,26 +341,24 @@ fn write_leaf(
column: &ArrayRef,
levels: LevelInfo,
) -> Result<i64> {
- let indices = levels.filter_array_indices();
- // Slice array according to computed offset and length
- let column = column.slice(levels.offset, levels.length);
+ // TODO: Avoid filtering if no need
Review Comment:
This is actually a hold over from an intermediate version of this PR, you
can only avoid doing the "take" operation if there are no-nulls which is
relatively rare, will remove
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
