kbendick commented on code in PR #5435:
URL: https://github.com/apache/iceberg/pull/5435#discussion_r938117606
##########
parquet/src/main/java/org/apache/iceberg/parquet/Parquet.java:
##########
@@ -231,6 +241,42 @@ private WriteBuilder createContextFunc(
return this;
}
+ private void setBloomFilterConfig(
+ Context context, BiConsumer<String, Boolean> withBloomFilterEnabled) {
+ Splitter dotSplitter = Splitter.on('.');
+ Joiner dotJoiner = Joiner.on(".");
+ context
+ .columnBloomFilterEnabled()
+ .forEach(
+ (columnPath, value) -> {
+ // check the column exists and support bloom filter
+ Types.NestedField field =
schema.caseInsensitiveFindField(columnPath);
+ if (field == null) {
+ LOG.warn(
+ "Column: {} is not found in the schema: {}, ignore it.",
columnPath, schema);
+ return;
+ }
+ Type fieldType = field.type();
+ // doesn't support boolean, list, map, struct type
+ if (!fieldType.isPrimitiveType()
+ || fieldType.typeId().equals(Type.TypeID.BOOLEAN)) {
+ LOG.warn(
+ "The type: {} of column: {} is not supported for parquet
bloom filter, ignore it",
+ fieldType,
+ columnPath);
+ return;
Review Comment:
Nit / non-blocking question: Is this something we can prevent the user from
doing? That might not be worth it though.
Id also format it as `Invalid bloom filter column configuration: field `%s`
is of type '%s' which cannot benefit from a bloom filter. This column
configuration will be ignored.`. I am a fan of a full constant string at the
front for searching logs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
