Will-Lo commented on a change in pull request #3339:
URL: https://github.com/apache/gobblin/pull/3339#discussion_r678521720
##########
File path:
gobblin-modules/gobblin-http/src/main/java/org/apache/gobblin/utils/HttpUtils.java
##########
@@ -143,7 +143,7 @@ public static URI buildURI(String urlTemplate, Map<String,
String> keys, Map<Str
*/
public static void updateStatusType(ResponseStatus status, int statusCode,
Set<String> errorCodeWhitelist) {
if (statusCode >= 300 & statusCode < 500) {
- List<String> whitelist = new ArrayList<>();
Review comment:
We can have a new PR to include all of it throughout the codebase, good
call out!
##########
File path:
gobblin-service/src/main/java/org/apache/gobblin/service/modules/flow/FlowGraphPath.java
##########
@@ -142,26 +142,26 @@ private static boolean
isNodeForkable(DagNode<JobExecutionPlan> dagNode) {
*/
private Dag<JobExecutionPlan> convertHopToDag(FlowEdgeContext
flowEdgeContext, Config sysConfig)
throws SpecNotFoundException, JobTemplate.TemplateException,
URISyntaxException {
- FlowTemplate flowTemplate = flowEdgeContext.getEdge().getFlowTemplate();
- DatasetDescriptor inputDatasetDescriptor =
flowEdgeContext.getInputDatasetDescriptor();
- DatasetDescriptor outputDatasetDescriptor =
flowEdgeContext.getOutputDatasetDescriptor();
- Config mergedConfig = flowEdgeContext.getMergedConfig();
- SpecExecutor specExecutor = flowEdgeContext.getSpecExecutor();
-
- List<JobExecutionPlan> jobExecutionPlans = new ArrayList<>();
- Map<String, String> templateToJobNameMap = new HashMap<>();
-
- //Get resolved job configs from the flow template
- List<Config> resolvedJobConfigs =
flowTemplate.getResolvedJobConfigs(mergedConfig, inputDatasetDescriptor,
outputDatasetDescriptor);
- //Iterate over each resolved job config and convert the config to a
JobSpec.
- for (Config resolvedJobConfig : resolvedJobConfigs) {
- JobExecutionPlan jobExecutionPlan = new
JobExecutionPlan.Factory().createPlan(flowSpec, resolvedJobConfig,
specExecutor, flowExecutionId, sysConfig);
- jobExecutionPlans.add(jobExecutionPlan);
- templateToJobNameMap.put(getJobTemplateName(jobExecutionPlan),
jobExecutionPlan.getJobSpec().getConfig().getString(
- ConfigurationKeys.JOB_NAME_KEY));
- }
- updateJobDependencies(jobExecutionPlans, templateToJobNameMap);
- return new JobExecutionPlanDagFactory().createDag(jobExecutionPlans);
+ FlowTemplate flowTemplate = flowEdgeContext.getEdge().getFlowTemplate();
Review comment:
Yeah it was slightly off here, the autoformat fixed it when i modified
the mile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
