jorisvandenbossche commented on a change in pull request #11724:
URL: https://github.com/apache/arrow/pull/11724#discussion_r752050016
##########
File path: python/pyarrow/tests/parquet/test_basic.py
##########
@@ -352,6 +352,37 @@ def test_byte_stream_split(use_legacy_dataset):
use_legacy_dataset=use_legacy_dataset)
+@parametrize_legacy_dataset
+def test_col_encoding(use_legacy_dataset):
+ arr_float = pa.array(list(map(float, range(100))))
+ arr_int = pa.array(list(map(int, range(100))))
+ mixed_table = pa.Table.from_arrays([arr_float, arr_int],
+ names=['a', 'b'])
+
+ # Check NONE col_encoding.
+ _check_roundtrip(mixed_table, expected=mixed_table, use_dictionary=False,
+ col_encoding=None, use_legacy_dataset=use_legacy_dataset)
+
+ # Check "BYTE_STREAM_SPLIT" for column 'a' and "PLAIN" col_encoding for
+ # column 'b'.
+ _check_roundtrip(mixed_table, expected=mixed_table, use_dictionary=False,
+ col_encoding={'a': "BYTE_STREAM_SPLIT", 'b': "PLAIN"},
+ use_legacy_dataset=use_legacy_dataset)
+
+ # Check "RLE" for column 'a' and "BYTE_STREAM_SPLIT" col_encoding for
+ # column 'b'.
+ _check_roundtrip(mixed_table, expected=mixed_table,
+ use_byte_stream_split=['a'],
+ col_encoding={'a': "RLE", 'b': "BYTE_STREAM_SPLIT"},
Review comment:
I think the description in ARROW-13781 ("supercede") might be about that
with this new keyword, the existing `use_btye_stream_split` keyword is no
longer needed since you can achieve the same with `col_encoding`. And indeed
specifying it through `col_encoding` will ignore what was passed to
`use_byte_stream_split`. So for users there is no need to specify both, in
which case I would explicitly disallow that (or at least raise a warning), to
avoid confusion.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
