rajarshisarkar commented on a change in pull request #4099:
URL: https://github.com/apache/iceberg/pull/4099#discussion_r807853727
##########
File path:
aws/src/integration/java/org/apache/iceberg/aws/glue/TestGlueCatalogTable.java
##########
@@ -364,4 +367,75 @@ public void testColumnCommentsAndParameters() {
);
Assert.assertEquals("Columns do not match", expectedColumns,
actualColumns);
}
+
+ @Test
+ public void testRegisterTable() {
+ String namespace = createNamespace();
+ String tableName = getRandomName();
+ createTable(namespace, tableName);
+ Table table = glueCatalog.loadTable(TableIdentifier.of(namespace,
tableName));
+ DataFile dataFile = DataFiles.builder(partitionSpec)
+ .withPath("/path/to/data-a.parquet")
+ .withFileSizeInBytes(10)
+ .withRecordCount(1)
+ .build();
+ table.newAppend().appendFile(dataFile).commit();
+ table.refresh();
+ long v1SnapshotId = table.currentSnapshot().snapshotId();
+ String v1MetadataLocation = ((BaseTable)
table).operations().current().metadataFileLocation();
+ table.newDelete().deleteFile(dataFile).commit();
+ table.refresh();
Review comment:
I think we can avoid the refresh calls.
##########
File path: aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java
##########
@@ -431,6 +433,39 @@ protected boolean isValidIdentifier(TableIdentifier
tableIdentifier) {
IcebergToGlueConverter.isValidTableName(tableIdentifier.name());
}
+ @Override
+ public org.apache.iceberg.Table registerTable(TableIdentifier identifier,
String metadataFileLocation) {
+ Preconditions.checkArgument(isValidIdentifier(identifier),
+ "Table identifier to register is invalid: " + identifier);
+ Preconditions.checkArgument(metadataFileLocation != null &&
!metadataFileLocation.isEmpty(),
+ "Cannot register an empty metadata file location as a table");
+
+ Map<String, String> tableParameters = ImmutableMap.of(
+ BaseMetastoreTableOperations.TABLE_TYPE_PROP,
+
BaseMetastoreTableOperations.ICEBERG_TABLE_TYPE_VALUE.toLowerCase(Locale.ENGLISH),
+ BaseMetastoreTableOperations.METADATA_LOCATION_PROP,
+ metadataFileLocation);
+
+ TableInput tableInput = TableInput.builder()
+ .name(IcebergToGlueConverter.getTableName(identifier))
+ .tableType(GlueTableOperations.GLUE_EXTERNAL_TABLE_TYPE)
+ .parameters(tableParameters)
+ .build();
+
+ try {
+ glue.createTable(CreateTableRequest.builder()
+ .databaseName(IcebergToGlueConverter.getDatabaseName(identifier))
+ .tableInput(tableInput)
Review comment:
Should we save `awsProperties.glueCatalogId()` as well?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
