wgtmac commented on code in PR #55449:
URL: https://github.com/apache/arrow/pull/55449#discussion_r4520443739
##########
cpp/src/parquet/properties.h:
##########
@@ -375,6 +375,19 @@ struct PARQUET_EXPORT CdcOptions {
int norm_level = 0;
};
+/// EXPERIMENTAL: Configuration for ALP encoding.
+struct PARQUET_EXPORT AlpConfig {
+ /// Values per encoded vector. Must be a power of two in [8, 32768].
+ int32_t vector_size = 1024;
+
+ /// Target number of non-null input values examined before caching the
preset.
+ /// Must be positive.
+ int64_t sampling_window_size = 122880;
Review Comment:
https://github.com/apache/arrow/pull/48345#discussion_r3148214213 mentioned
this value to the ALP paper. I think the reason is that the ALP paper uses
DuckDB for experiment whose default parquet row group size is 122880. (cc
@prtkgaur)
I think this should be an implementation-specific tuning choice because the
appropriate sampling window depends on the data so I’ve kept the existing
default and made it configurable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
