kbendick commented on code in PR #5435:
URL: https://github.com/apache/iceberg/pull/5435#discussion_r938117606
##########
parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java:
##########
@@ -231,6 +241,42 @@ private WriteBuilder createContextFunc(
return this;
}
+ private void setBloomFilterConfig(
+ Context context, BiConsumer<String, Boolean> withBloomFilterEnabled) {
+ Splitter dotSplitter = Splitter.on('.');
+ Joiner dotJoiner = Joiner.on(".");
+ context
+ .columnBloomFilterEnabled()
+ .forEach(
+ (columnPath, value) -> {
+ // check the column exists and support bloom filter
+ Types.NestedField field =
schema.caseInsensitiveFindField(columnPath);
+ if (field == null) {
+ LOG.warn(
+ "Column: {} is not found in the schema: {}, ignore it.",
columnPath, schema);
+ return;
+ }
+ Type fieldType = field.type();
+ // doesn't support boolean, list, map, struct type
+ if (!fieldType.isPrimitiveType()
+ || fieldType.typeId().equals(Type.TypeID.BOOLEAN)) {
+ LOG.warn(
+ "The type: {} of column: {} is not supported for parquet
bloom filter, ignore it",
+ fieldType,
+ columnPath);
+ return;
Review Comment:
Nit / non-blocking question: Is this something we can prevent the user from
doing? That might not be worth it though.
Id also format it as `Invalid bloom filter column configuration: field `%s`
is of type '%s` which parquet does not have bloom filter support for. This
column configuration will be ignored.`. I am a fan of a full constant string at
the front for searching logs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
