phet commented on code in PR #3560:
URL: https://github.com/apache/gobblin/pull/3560#discussion_r977062551
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/iceberg/IcebergDatasetFinder.java:
##########
@@ -40,18 +39,20 @@
* and creates a {@link IcebergDataset} for each one.
*/
@Slf4j
-@AllArgsConstructor
public class IcebergDatasetFinder implements
IterableDatasetFinder<IcebergDataset> {
public static final String ICEBERG_DATASET_PREFIX =
DatasetConstants.PLATFORM_ICEBERG + ".dataset";
public static final String ICEBERG_HIVE_CATALOG_METASTORE_URI_KEY =
ICEBERG_DATASET_PREFIX + ".hive.metastore.uri";
public static final String ICEBERG_DB_NAME = ICEBERG_DATASET_PREFIX +
".database.name";
public static final String ICEBERG_TABLE_NAME = ICEBERG_DATASET_PREFIX +
".table.name";
- private String dbName;
- private String tblName;
private final Properties properties;
- protected final FileSystem fs;
+ protected final FileSystem sourceFs;
+
+ public IcebergDatasetFinder(FileSystem fs, Properties properties) {
Review Comment:
note: equivalent to `@lombok.RequiredArgsConstructor`
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/copy/iceberg/IcebergDataset.java:
##########
@@ -185,31 +185,28 @@ protected Map<Path, FileStatus>
getFilePathsToFileStatus() throws IOException {
for (String pathString : pathsToCopy) {
Path path = new Path(pathString);
- result.put(path, this.fs.getFileStatus(path));
+ result.put(path, this.sourceFs.getFileStatus(path));
}
return result;
}
- DatasetDescriptor getSourceDataset() {
- return getDatasetDescriptor(sourceMetastoreURI);
+ DatasetDescriptor getSourceDataset(FileSystem sourceFs) {
+ return getDatasetDescriptor(sourceMetastoreURI, sourceFs);
}
- DatasetDescriptor getDestinationDataset() {
- return getDatasetDescriptor(targetMetastoreURI);
+ DatasetDescriptor getDestinationDataset(FileSystem targetFs) {
Review Comment:
shouldn't these either be `@VisibleForTesting` or `protected`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@gobblin.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
