nevi-me opened a new pull request #8792:
URL: https://github.com/apache/arrow/pull/8792
save progress (11/11/2020)
save progress
Integrating level calculations in writer
Some tests are failing, still have a long way to go
fix lints
save progress
I'm nearly able to reproduce a `<struct<struct<primitive>>`
I'm writing one level too high for nulls, so my null counts differ.
Fixing this should result in nested struct roundtrip for the fully
nullable case.
Currently failing tests:
```rust
failures:
arrow::arrow_writer::tests::arrow_writer_2_level_struct
arrow::arrow_writer::tests::arrow_writer_complex
arrow::levels::tests::test_calculate_array_levels_2
arrow::levels::tests::test_calculate_array_levels_nested_list
arrow::levels::tests::test_calculate_one_level_2
```
They are mainly failing because we don't roundtrip lists correctly
save progress 19/20-11-2020
Structs that have nulls are working (need to revert non-null logic)
TODOs that need addressing later on
save progress
- Focused more on nested structs.
- Confident that writes are now fine
- Found issue with struct logical comparison, blocks this work
add failing arrow struct array test
a bit of cleanup for failing tests
Also document why dictionary test is failing
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
