Joseph Allemandou created GOBBLIN-1558:
------------------------------------------
Summary: Make BasePublisher record detailed folders in state when
parent-folder doesn't exist
Key: GOBBLIN-1558
URL: https://issues.apache.org/jira/browse/GOBBLIN-1558
Project: Apache Gobblin
Issue Type: Improvement
Components: gobblin-core
Reporter: Joseph Allemandou
Assignee: Abhishek Tiwari
When publishing to non-existing parent directory (first run of job type for
instance), the BaseDataPublisher doesn't recurse into written folders but moves
the top written folder at once (line 429). While being correct, this prevent
the state's PUBLISHER_DIRS value to contain the detailed published folder.
The update consists of a new method
{code:java}
addWriterOutputToNewDir{code}
that mimic the already existing
{code:java}
addWriterOutputToExistingDir{code}
and is applied when the parent folder is not present, making the PUBLISHER_DIRS
value being filled with detailed folder value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
