adamreeve commented on PR #102:
URL: https://github.com/apache/parquet-testing/pull/102#issuecomment-3981171655
Thanks @hsiang-c. It looks like the new data is very close to what's in the
existing files but the int96 timestamp values are different. I get differences
for that column if I try to use the C++ test utilities:
```
Expected equality of these values:
expected_column_data.values
Which is: { 12-byte object <00-00 00-00 01-00 00-00 02-00 00-00>,
12-byte object <01-00 00-00 02-00 00-00 03-00 00-00>, 12-byte object <02-00
00-00 03-00 00-00 04-00 00-00>, 12-byte object <03-00 00-00 04-00 00-00 05-00
00-00>, 12-byte object <04-00 00-00 05-00 00-00 06-00 00-00>, 12-byte object
<05-00 00-00 06-00 00-00 07-00 00-00>, 12-byte object <06-00 00-00 07-00 00-00
08-00 00-00>, 12-byte object <07-00 00-00 08-00 00-00 09-00 00-00>, 12-byte
object <08-00 00-00 09-00 00-00 0A-00 00-00>, 12-byte object <09-00 00-00 0A-00
00-00 0B-00 00-00>, 12-byte object <0A-00 00-00 0B-00 00-00 0C-00 00-00>,
12-byte object <0B-00 00-00 0C-00 00-00 0D-00 00-00>, 12-byte object <0C-00
00-00 0D-00 00-00 0E-00 00-00>, 12-byte object <0D-00 00-00 0E-00 00-00 0F-00
00-00>, 12-byte object <0E-00 00-00 0F-00 00-00 10-00 00-00>, 12-byte object
<0F-00 00-00 10-00 00-00 11-00 00-00>, 12-byte object <10-00 00-00 11-00 00-00
12-00 00-00>, 12-byte object <11-00 00-00 12-00 00-00 13-00 00-00>, 12-byte
object <12-00 00-00 13-00 00-00 14-00 00-00>, 12-byte object <13-00 00-00
14-00 00-00 15-00 00-00>, 12-byte object <14-00 00-00 15-00 00-00 16-00 00-00>,
12-byte object <15-00 00-00 16-00 00-00 17-00 00-00>, 12-byte object <16-00
00-00 17-00 00-00 18-00 00-00>, 12-byte object <17-00 00-00 18-00 00-00 19-00
00-00>, 12-byte object <18-00 00-00 19-00 00-00 1A-00 00-00>, 12-byte object
<19-00 00-00 1A-00 00-00 1B-00 00-00>, 12-byte object <1A-00 00-00 1B-00 00-00
1C-00 00-00>, 12-byte object <1B-00 00-00 1C-00 00-00 1D-00 00-00>, 12-byte
object <1C-00 00-00 1D-00 00-00 1E-00 00-00>, 12-byte object <1D-00 00-00 1E-00
00-00 1F-00 00-00>, 12-byte object <1E-00 00-00 1F-00 00-00 20-00 00-00>,
12-byte object <1F-00 00-00 20-00 00-00 21-00 00-00>, ... }
read_col_data.values
Which is: { 12-byte object <00-00 00-00 00-00 00-00 01-00 00-00>,
12-byte object <00-00 00-00 02-00 00-00 00-00 00-00>, 12-byte object <03-00
00-00 00-00 00-00 04-00 00-00>, 12-byte object <00-00 00-00 05-00 00-00 00-00
00-00>, 12-byte object <06-00 00-00 00-00 00-00 07-00 00-00>, 12-byte object
<00-00 00-00 08-00 00-00 00-00 00-00>, 12-byte object <09-00 00-00 00-00 00-00
0A-00 00-00>, 12-byte object <00-00 00-00 0B-00 00-00 00-00 00-00>, 12-byte
object <0C-00 00-00 00-00 00-00 0D-00 00-00>, 12-byte object <00-00 00-00 0E-00
00-00 00-00 00-00>, 12-byte object <0F-00 00-00 00-00 00-00 10-00 00-00>,
12-byte object <00-00 00-00 11-00 00-00 00-00 00-00>, 12-byte object <12-00
00-00 00-00 00-00 13-00 00-00>, 12-byte object <00-00 00-00 14-00 00-00 00-00
00-00>, 12-byte object <15-00 00-00 00-00 00-00 16-00 00-00>, 12-byte object
<00-00 00-00 17-00 00-00 00-00 00-00>, 12-byte object <18-00 00-00 00-00 00-00
19-00 00-00>, 12-byte object <00-00 00-00 1A-00 00-00 00-00 00-00>, 12-byte
object <1B-00 00-00 00-00 00-00 1C-00 00-00>, 12-byte object <00-00 00-00
1D-00 00-00 00-00 00-00>, 12-byte object <1E-00 00-00 00-00 00-00 1F-00 00-00>,
12-byte object <00-00 00-00 20-00 00-00 00-00 00-00>, 12-byte object <21-00
00-00 00-00 00-00 22-00 00-00>, 12-byte object <00-00 00-00 23-00 00-00 00-00
00-00>, 12-byte object <24-00 00-00 00-00 00-00 25-00 00-00>, 12-byte object
<00-00 00-00 26-00 00-00 00-00 00-00>, 12-byte object <27-00 00-00 00-00 00-00
28-00 00-00>, 12-byte object <00-00 00-00 29-00 00-00 00-00 00-00>, 12-byte
object <2A-00 00-00 00-00 00-00 2B-00 00-00>, 12-byte object <00-00 00-00 2C-00
00-00 00-00 00-00>, 12-byte object <2D-00 00-00 00-00 00-00 2E-00 00-00>,
12-byte object <00-00 00-00 2F-00 00-00 00-00 00-00>, ... }
```
I can't see where this comes from in the Java code, it looks like it's not
part of `generateLinearData`?
On the C++ side this column is generated by `GenerateSampleData<Int96Type>`:
https://github.com/apache/arrow/blob/6e5d143a4a1c45d9d3a5cc8b26397dab721f2331/cpp/src/parquet/encryption/test_encryption_util.cc#L135
And can you also please add a note about the encryption keys used to the
README
(https://github.com/apache/parquet-testing/blob/master/data/README.md#encrypted-files).
I've pushed a branch with my changes to C++ parquet to
https://github.com/apache/arrow/compare/main...adamreeve:arrow:test_aes256_keys
(probably needs some refactoring before I could make a PR with this, but it's
enough to test the files).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
