[
https://issues.apache.org/jira/browse/GOBBLIN-1496?focusedWorklogId=630683&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630683
]
ASF GitHub Bot logged work on GOBBLIN-1496:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 28/Jul/21 17:44
Start Date: 28/Jul/21 17:44
Worklog Time Spent: 10m
Work Description: Will-Lo commented on a change in pull request #3339:
URL: https://github.com/apache/gobblin/pull/3339#discussion_r678521720
##########
File path:
gobblin-modules/gobblin-http/src/main/java/org/apache/gobblin/utils/HttpUtils.java
##########
@@ -143,7 +143,7 @@ public static URI buildURI(String urlTemplate, Map<String,
String> keys, Map<Str
*/
public static void updateStatusType(ResponseStatus status, int statusCode,
Set<String> errorCodeWhitelist) {
if (statusCode >= 300 & statusCode < 500) {
- List<String> whitelist = new ArrayList<>();
Review comment:
We can have a new PR to include all of it throughout the codebase, good
call out!
##########
File path:
gobblin-service/src/main/java/org/apache/gobblin/service/modules/flow/FlowGraphPath.java
##########
@@ -142,26 +142,26 @@ private static boolean
isNodeForkable(DagNode<JobExecutionPlan> dagNode) {
*/
private Dag<JobExecutionPlan> convertHopToDag(FlowEdgeContext
flowEdgeContext, Config sysConfig)
throws SpecNotFoundException, JobTemplate.TemplateException,
URISyntaxException {
- FlowTemplate flowTemplate = flowEdgeContext.getEdge().getFlowTemplate();
- DatasetDescriptor inputDatasetDescriptor =
flowEdgeContext.getInputDatasetDescriptor();
- DatasetDescriptor outputDatasetDescriptor =
flowEdgeContext.getOutputDatasetDescriptor();
- Config mergedConfig = flowEdgeContext.getMergedConfig();
- SpecExecutor specExecutor = flowEdgeContext.getSpecExecutor();
-
- List<JobExecutionPlan> jobExecutionPlans = new ArrayList<>();
- Map<String, String> templateToJobNameMap = new HashMap<>();
-
- //Get resolved job configs from the flow template
- List<Config> resolvedJobConfigs =
flowTemplate.getResolvedJobConfigs(mergedConfig, inputDatasetDescriptor,
outputDatasetDescriptor);
- //Iterate over each resolved job config and convert the config to a
JobSpec.
- for (Config resolvedJobConfig : resolvedJobConfigs) {
- JobExecutionPlan jobExecutionPlan = new
JobExecutionPlan.Factory().createPlan(flowSpec, resolvedJobConfig,
specExecutor, flowExecutionId, sysConfig);
- jobExecutionPlans.add(jobExecutionPlan);
- templateToJobNameMap.put(getJobTemplateName(jobExecutionPlan),
jobExecutionPlan.getJobSpec().getConfig().getString(
- ConfigurationKeys.JOB_NAME_KEY));
- }
- updateJobDependencies(jobExecutionPlans, templateToJobNameMap);
- return new JobExecutionPlanDagFactory().createDag(jobExecutionPlans);
+ FlowTemplate flowTemplate = flowEdgeContext.getEdge().getFlowTemplate();
Review comment:
Yeah it was slightly off here, the autoformat fixed it when i modified
the mile
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 630683)
Time Spent: 40m (was: 0.5h)
> Predefine map and array sizes where possible for GaaS
> -----------------------------------------------------
>
> Key: GOBBLIN-1496
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1496
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: William Lo
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> GaaS has many scenarios with sparse/empty maps and lists. When initializing
> new data structures, we should initialize them to a certain size if we know
> the elements beforehand.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
