[
https://issues.apache.org/jira/browse/GOBBLIN-1559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Urmi Mustafi updated GOBBLIN-1559:
----------------------------------
Description:
Currently wildcard does not work with GaaS paths due to complications with path
finding. See FSDatasetDescriptor.isPathContaining, if the path is a glob it
will just return false. The reason for this is that it is not really possible
to calculate if one path glob pattern "contains"/ is a subset of another glob
pattern.
But for our self serve edges, we usually just accept any path by defining
{code:java}
gobblin.flow.edge.input.dataset.descriptor.0.path=${gobblin.flow.input.dataset.descriptor.path}{code}
We should be able to add a special case in this method, where glob patterns are
accepted if the paths match exactly, and that should allow glob patterns as
input paths for the self serve edges.
was:Glob patterns should be accepted if the paths match exactly, and that
should allow glob patterns as input paths for the self serve edges.
> Support wildcard for input paths
> --------------------------------
>
> Key: GOBBLIN-1559
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1559
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
>
> Currently wildcard does not work with GaaS paths due to complications with
> path finding. See FSDatasetDescriptor.isPathContaining, if the path is a glob
> it will just return false. The reason for this is that it is not really
> possible to calculate if one path glob pattern "contains"/ is a subset of
> another glob pattern.
> But for our self serve edges, we usually just accept any path by defining
> {code:java}
> gobblin.flow.edge.input.dataset.descriptor.0.path=${gobblin.flow.input.dataset.descriptor.path}{code}
> We should be able to add a special case in this method, where glob patterns
> are accepted if the paths match exactly, and that should allow glob patterns
> as input paths for the self serve edges.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
