[
https://issues.apache.org/jira/browse/GOBBLIN-1786?focusedWorklogId=848480&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-848480
]
ASF GitHub Bot logged work on GOBBLIN-1786:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 02/Mar/23 01:45
Start Date: 02/Mar/23 01:45
Worklog Time Spent: 10m
Work Description: phet commented on code in PR #3643:
URL: https://github.com/apache/gobblin/pull/3643#discussion_r1122516199
##########
gobblin-data-management/src/test/java/org/apache/gobblin/data/management/copy/iceberg/IcebergDatasetTest.java:
##########
@@ -187,7 +188,7 @@ public void
testGetFilePathsDoesNotSwallowDestFileSystemException() throws IOExc
MockFileSystemBuilder sourceFsBuilder = new
MockFileSystemBuilder(SRC_FS_URI);
FileSystem sourceFs = sourceFsBuilder.build();
- IcebergDataset icebergDataset = new IcebergDataset(testDbName,
testTblName, icebergTable, new Properties(), sourceFs);
+ IcebergDataset icebergDataset = new IcebergDataset(testDbName,
testTblName, icebergTable, SRC_CATALOG_URI, new Properties(), sourceFs);
Review Comment:
in the suggestion above, this additional new param goes away
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/iceberg/IcebergDatasetFinder.java:
##########
@@ -92,6 +103,18 @@ public Iterator<IcebergDataset> getDatasetsIterator()
throws IOException {
protected IcebergDataset createIcebergDataset(String dbName, String tblName,
IcebergCatalog icebergCatalog, Properties properties, FileSystem fs) {
IcebergTable icebergTable = icebergCatalog.openTable(dbName, tblName);
- return new IcebergDataset(dbName, tblName, icebergTable, properties, fs);
+ return new IcebergDataset(dbName, tblName, icebergTable,
icebergCatalog.getCatalogUri(), properties, fs);
Review Comment:
seems worthwhile to keep around the `IcebergCatalog.getCatalogUri()` method,
but rather than calling here, why not have that catalog pass its URI to the
`IcebergTable`, as the former is the very one to create the latter?
subsequently `IcebergDataset` will call `IcebergTable.getDatasetDescriptor`,
w/o needing to pass in a URI arg.
Issue Time Tracking
-------------------
Worklog Id: (was: 848480)
Time Spent: 6h (was: 5h 50m)
> Support Other Catalog Types for Iceberg Distcp
> ----------------------------------------------
>
> Key: GOBBLIN-1786
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1786
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Meeth Gala
> Priority: Major
> Time Spent: 6h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
