norrishuang commented on code in PR #4314:
URL: https://github.com/apache/flink-cdc/pull/4314#discussion_r2929963064
##########
flink-cdc-connect/flink-cdc-pipeline-connectors/flink-cdc-pipeline-connector-iceberg/src/test/java/org/apache/flink/cdc/connectors/iceberg/sink/IcebergDataSinkFactoryTest.java:
##########
@@ -111,6 +111,48 @@ void testPrefixRequireOption() {
Assertions.assertThat(dataSink).isInstanceOf(IcebergDataSink.class);
}
+ @Test
+ void testCreateGlueCatalogDataSink() {
+ DataSinkFactory sinkFactory =
+ FactoryDiscoveryUtils.getFactoryByIdentifier("iceberg",
DataSinkFactory.class);
+
Assertions.assertThat(sinkFactory).isInstanceOf(IcebergDataSinkFactory.class);
+
+ Configuration conf =
+ Configuration.fromMap(
+ ImmutableMap.<String, String>builder()
+ .put("catalog.properties.type", "glue")
+ .put("catalog.properties.warehouse",
"s3://my-bucket/warehouse")
+ .put("catalog.properties.io-impl",
"org.apache.iceberg.aws.s3.S3FileIO")
+ .put("catalog.properties.client.region",
"us-east-1")
+ .put("catalog.properties.glue.skip-archive",
"true")
+ .build());
+ DataSink dataSink =
+ sinkFactory.createDataSink(
+ new FactoryHelper.DefaultContext(
+ conf, conf,
Thread.currentThread().getContextClassLoader()));
+ Assertions.assertThat(dataSink).isInstanceOf(IcebergDataSink.class);
+ }
+
+ @Test
+ void testCreateGlueCatalogWithCatalogImpl() {
+ DataSinkFactory sinkFactory =
+ FactoryDiscoveryUtils.getFactoryByIdentifier("iceberg",
DataSinkFactory.class);
+
Assertions.assertThat(sinkFactory).isInstanceOf(IcebergDataSinkFactory.class);
+
+ Configuration conf =
+ Configuration.fromMap(
+ ImmutableMap.<String, String>builder()
+ .put("catalog.properties.catalog-impl",
"org.apache.iceberg.aws.glue.GlueCatalog")
+ .put("catalog.properties.warehouse",
"s3://my-bucket/warehouse")
+ .put("catalog.properties.io-impl",
"org.apache.iceberg.aws.s3.S3FileIO")
+ .build());
+ DataSink dataSink =
+ sinkFactory.createDataSink(
+ new FactoryHelper.DefaultContext(
+ conf, conf,
Thread.currentThread().getContextClassLoader()));
+ Assertions.assertThat(dataSink).isInstanceOf(IcebergDataSink.class);
+ }
Review Comment:
`iceberg-aws` is declared with provided scope (supplied by EMR at runtime),
so GlueCatalog is not on the test classpath. Even if we added it as a test
dependency, GlueCatalog.initialize() would try to connect to AWS Glue, which
would fail in CI without valid AWS credentials.
These Glue tests are intentionally scoped to verify that the factory
correctly parses Glue catalog configurations. Actual catalog construction and
connectivity should be covered by integration tests with proper AWS access.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
