askoa opened a new pull request, #3447:
URL: https://github.com/apache/arrow-rs/pull/3447
# Which issue does this PR close?
Closes #3408
# Rationale for this change
# What changes are included in this PR?
The issue is not completely fixed. I added a test `fallback_flush_data_page`
and marked it as `ignore` as its failing. I included the difference before and
after change. We can see from the diff that, before the change, the values are
garbage after 32 (which is the page size). After change, there is an issue
between 33-39. The values match after 39.
As I am not acquainted with parquet format, it might take some time for me
to analyze this. If anyone else want to analyze then feel free to go ahead.
cc @tustvold @alamb
difference before change:
```
running 1 test
thread 'arrow::arrow_writer::tests::fallback_flush_data_page' panicked at
'assertion failed: `(left == right)`
left: `["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11",
"12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24",
"25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37",
"38", "39", "40", "41", "42", "43", "44", "45", "46", "47", "48", "49", "50",
"51", "52", "53", "54", "55", "56", "57", "58", "59", "60", "61", "62"]`,
right: `["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11",
"12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24",
"25", "26", "27", "28", "29", "30", "31", "0", "01", "02", "03", "04", "05",
"06", "07", "89", "81", "80", "81", "82", "83", "84", "85", "0", "01", "23",
"24", "25", "26", "27", "28", "29", "21", "20", "21", "23", "24", "25"]`',
parquet/src/arrow/arrow_writer/mod.rs:1887:21
```
difference after change:
```
running 1 test
thread 'arrow::arrow_writer::tests::fallback_flush_data_page' panicked at
'assertion failed: `(left == right)`
left: `["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11",
"12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24",
"25", "26", "27", "28", "29", "30", "31", "32", "33", "34", "35", "36", "37",
"38", "39", "40", "41", "42", "43", "44", "45", "46", "47", "48", "49", "50",
"51", "52", "53", "54", "55", "56", "57", "58", "59", "60", "61", "62"]`,
right: `["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11",
"12", "13", "14", "15", "16", "17", "18", "19", "20", "21", "22", "23", "24",
"25", "26", "27", "28", "29", "30", "31", "2", "23", "24", "25", "26", "27",
"28", "29", "40", "41", "42", "43", "44", "45", "46", "47", "48", "49", "50",
"51", "52", "53", "54", "55", "56", "57", "58", "59", "60", "61", "62"]`',
parquet/src/arrow/arrow_writer/mod.rs:1889:21
```
# Are there any user-facing changes?
<!--
If there are user-facing changes then we may require documentation to be
updated before approving the PR.
-->
<!---
If there are any breaking changes to public APIs, please add the `breaking
change` label.
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
