rambleraptor commented on code in PR #15795:
URL: https://github.com/apache/iceberg/pull/15795#discussion_r3365545168
##########
data/src/test/java/org/apache/iceberg/data/BaseFormatModelTests.java:
##########
@@ -130,6 +133,56 @@ protected boolean supportsBatchReads() {
@TempDir private File tableDir;
+ /**
+ * Returns the schema with columns whose types are unsupported by this
engine removed. The base
+ * implementation accepts every column; engines that cannot handle certain
types override this
+ * method, typically by calling {@link #excludeColumnsContaining(Schema,
Set)}.
+ */
+ protected Schema filterUnsupported(Schema schema) {
+ return schema;
+ }
+
+ /**
+ * Returns {@code schema} with every top-level column whose type tree
contains any of {@code
+ * unsupportedTypeIds} removed.
+ */
+ protected static Schema excludeColumnsContaining(
+ Schema schema, Set<Type.TypeID> unsupportedTypeIds) {
+ List<Types.NestedField> kept =
+ schema.columns().stream()
+ .filter(
+ col ->
+ TypeUtil.find(col.type(), t ->
unsupportedTypeIds.contains(t.typeId())) == null)
+ .toList();
+ return kept.size() == schema.columns().size() ? schema : new Schema(kept);
+ }
+
+ private Schema supportedSchema(DataGenerator generator) {
+ Schema filtered = filterUnsupported(generator.schema());
+ assumeThat(filtered.columns())
+ .as("All columns of %s are unsupported by this engine", generator)
+ .isNotEmpty();
+ return filtered;
+ }
+
+ private static List<Record> project(List<Record> records, Schema
targetSchema) {
Review Comment:
Yep, added it to a couple.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
