[
https://issues.apache.org/jira/browse/GOBBLIN-1558?focusedWorklogId=661137&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661137
]
ASF GitHub Bot logged work on GOBBLIN-1558:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 06/Oct/21 19:06
Start Date: 06/Oct/21 19:06
Worklog Time Spent: 10m
Work Description: jobar opened a new pull request #3409:
URL: https://github.com/apache/gobblin/pull/3409
Dear Gobblin maintainers,
This is my second PR, I hope I'm doing it in a way that works for you folks
:)
### JIRA
- [ ] My PR addresses the following [Gobblin
JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references
them in the PR title.
- https://issues.apache.org/jira/browse/GOBBLIN-1558
### Description
- [ ] Make BaseDataPublisher publish to new folders in a way that preserve
detailed subfolder information in PUBLISHER_DIRS state value. In previous
version, when publishing to non-existing parent directory (first run of job
type for instance), the BaseDataPublisher wouldn't recurse into written folders
but moves the top written folder at once, leading to the state's PUBLISHER_DIRS
value containing the top-level written folder, not the contain the detailed
published folder.
### Tests
- [ ] Not yet done, will come soon.
### Commits
- [ ] My commits all reference JIRA issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
2. Subject is limited to 50 characters
3. Subject does not end with a period
4. Subject uses the imperative mood ("add", not "adding")
5. Body wraps at 72 characters
6. Body explains "what" and "why", not "how"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 661137)
Remaining Estimate: 0h
Time Spent: 10m
> Make BasePublisher record detailed folders in state when parent-folder
> doesn't exist
> ------------------------------------------------------------------------------------
>
> Key: GOBBLIN-1558
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1558
> Project: Apache Gobblin
> Issue Type: Improvement
> Components: gobblin-core
> Reporter: Joseph Allemandou
> Assignee: Abhishek Tiwari
> Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When publishing to non-existing parent directory (first run of job type for
> instance), the BaseDataPublisher doesn't recurse into written folders but
> moves the top written folder at once (line 429). While being correct, this
> prevent the state's PUBLISHER_DIRS value to contain the detailed published
> folder.
> The update consists of a new method
> {code:java}
> addWriterOutputToNewDir{code}
> that mimic the already existing
> {code:java}
> addWriterOutputToExistingDir{code}
> and is applied when the parent folder is not present, making the
> PUBLISHER_DIRS value being filled with detailed folder value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
