Will-Lo commented on code in PR #3514:
URL: https://github.com/apache/gobblin/pull/3514#discussion_r884037127
##########
gobblin-service/src/main/java/org/apache/gobblin/service/modules/spec/JobExecutionPlan.java:
##########
@@ -108,8 +109,13 @@ private static JobSpec buildJobSpec(FlowSpec flowSpec,
Config jobConfig, Long fl
// Modify the job name to include the flow group, flow name, edge id,
and a random string to avoid collisions since
// job names are assumed to be unique within a dag.
- jobName = Joiner.on(JOB_NAME_COMPONENT_SEPARATION_CHAR).join(flowGroup,
flowName, jobName, edgeId, flowInputPath.hashCode());
-
+ int hash = flowInputPath.hashCode();
+ jobName = Joiner.on(JOB_NAME_COMPONENT_SEPARATION_CHAR).join(flowGroup,
flowName, jobName, edgeId, hash);
+ // jobNames are commonly used as a directory name, which is limited to
255 characters
+ if (jobName.length() >= MAX_JOB_NAME_LENGTH) {
+ // shorten job length to be 128 characters (flowGroup) + hashCode
length
+ jobName =
Joiner.on(JOB_NAME_COMPONENT_SEPARATION_CHAR).join(flowGroup, hash);
Review Comment:
I think adding some property of the flowName is good (handles the scenario
of a job in the same flowGroup with the same input path running into
conflicts), but instead of an arbitrary number of characters from the flowName
I'll hash the flowName instead, I think with that it'll be cover all cases
(that I can think of right now at least :) )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
