[
https://issues.apache.org/jira/browse/GOBBLIN-1952?focusedWorklogId=889337&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-889337
]
ASF GitHub Bot logged work on GOBBLIN-1952:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 07/Nov/23 21:19
Start Date: 07/Nov/23 21:19
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3822:
URL: https://github.com/apache/gobblin/pull/3822#discussion_r1385600647
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/spec/JobExecutionPlan.java:
##########
@@ -112,10 +112,10 @@ private static JobSpec buildJobSpec(FlowSpec flowSpec,
Config jobConfig, Long fl
// job names are assumed to be unique within a dag.
int hash = flowInputPath.hashCode();
Review Comment:
We can't reuse that hash because it prevents collisions where flows with the
same jobnames within edges. Since we hash this string, it helps the second hash
be unique as well to not run into the same scenario
Issue Time Tracking
-------------------
Worklog Id: (was: 889337)
Time Spent: 1.5h (was: 1h 20m)
> GaaS JobNames with long lengths cause issues with HDFS folders
> --------------------------------------------------------------
>
> Key: GOBBLIN-1952
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1952
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-service
> Reporter: William Lo
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> Gobblin-as-a-Service creates jobnames using flowgroups flownames, edges, and
> jobnames from the template. However, this tends to create a very long string
> which then causes issues in Gobblin job when creating files that use the
> jobname to create working directories or state stores. Although there has
> been previous code that shortens job name lengths, we want to further
> increase this by being more aggressive with the maximum length of the jobname
> to reduce the odds of exceeding 255 chars (max length of HDFS component)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
