[
https://issues.apache.org/jira/browse/GOBBLIN-1877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kip Kohn updated GOBBLIN-1877:
------------------------------
Description:
Manifest-based copy (FS distcp) shouldn't presume the manifest files to be on
the same FileSystem used as the I/O source to copy from. Instead allow
optional config to provide an alternative solely for reading manifest files
(enumerating filepaths).
Proposed config: `gobblin.copy.manifestBased.read.fs.uri`
(after `gobblin.copy.manifestBased.manifest.location`, and
`source.filebased.fs.uri`)
was:Manifest-based copy (FS distcp) shouldn't presume the manifest files to
be on the same FileSystem as used as the copy-source. Instead allow optional
config to provide an alternative solely for reading manifest files (enumerating
filepaths).
> Manifest-Based Distcp - Enable specifying an alt. FS solely for reading
> manifests
> ---------------------------------------------------------------------------------
>
> Key: GOBBLIN-1877
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1877
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-core
> Reporter: Kip Kohn
> Assignee: Abhishek Tiwari
> Priority: Minor
>
> Manifest-based copy (FS distcp) shouldn't presume the manifest files to be on
> the same FileSystem used as the I/O source to copy from. Instead allow
> optional config to provide an alternative solely for reading manifest files
> (enumerating filepaths).
> Proposed config: `gobblin.copy.manifestBased.read.fs.uri`
> (after `gobblin.copy.manifestBased.manifest.location`, and
> `source.filebased.fs.uri`)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
