[
https://issues.apache.org/jira/browse/GOBBLIN-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Allemandou updated GOBBLIN-1558:
---------------------------------------
Description:
It's currently not possible to overwrite the behavior of the publisher when
publishing to non-existing parent directory (first run of job type for
instance). This ticket is about extracting a func tion in the the
BaseDataPublisher to make behavior possible to override.
Extracted new method
{code:java}
addWriterOutputToNewDir{code}
that goes with the already existing
{code:java}
addWriterOutputToExistingDir{code}
The goal of this ticket is to NOT change the behavior of the current code.
was:
When publishing to non-existing parent directory (first run of job type for
instance), the BaseDataPublisher doesn't recurse into written folders but moves
the top written folder at once (line 429). While being correct, this prevent
the state's PUBLISHER_DIRS value to contain the detailed published folder.
The update consists of a new method
{code:java}
addWriterOutputToNewDir{code}
that mimic the already existing
{code:java}
addWriterOutputToExistingDir{code}
and is applied when the parent folder is not present, making the PUBLISHER_DIRS
value being filled with detailed folder value.
Summary: Make it possible to overwrite BaseDataPublisher behavior when
parent-folder doesn't exist (was: Make BasePublisher record detailed folders
in state when parent-folder doesn't exist)
> Make it possible to overwrite BaseDataPublisher behavior when parent-folder
> doesn't exist
> -----------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1558
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1558
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-core
> Reporter: Joseph Allemandou
> Assignee: Abhishek Tiwari
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It's currently not possible to overwrite the behavior of the publisher when
> publishing to non-existing parent directory (first run of job type for
> instance). This ticket is about extracting a func tion in the the
> BaseDataPublisher to make behavior possible to override.
> Extracted new method
> {code:java}
> addWriterOutputToNewDir{code}
> that goes with the already existing
> {code:java}
> addWriterOutputToExistingDir{code}
> The goal of this ticket is to NOT change the behavior of the current code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
