[
https://issues.apache.org/jira/browse/SQOOP-1496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14141538#comment-14141538
]
Abraham Elmahrek commented on SQOOP-1496:
-----------------------------------------
+1 Thanks Veena!
> Sqoop2: Revisit/Refactor the SubmissionEngine/ExecutionEngine APIs
> ------------------------------------------------------------------
>
> Key: SQOOP-1496
> URL: https://issues.apache.org/jira/browse/SQOOP-1496
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Attachments: SQOOP-1496.patch, Screen Shot 2014-09-08 at 2.35.06
> PM.png
>
>
> Here are the things this ticket should address
> 1. Refactor the submit() in JobManager ( it is way too long for any good unit
> testing to happen)
> 2. Revisit the Execution Engine and Submission Engine relationship in the
> code.
> ExecutionEngine api should probably not create a submissionRequest. It should
> be the inverse. The Submission Engine is an abstraction of a JobTracker/
> YARN/ Mesos/ OOzie. It will create a job request for the execution engine
> while it submits the job into the execution engine
> a) Suggest renaming SubmissionEngine to a JobSubmissionManager.
>
> b) SubmissionRequest is a JobRunRequest. See the java doc on the current code
> SubmissionRequest, it should already hint us that this object holds the job
> context to aid the execution engine
> c) Change the apis in Execution Engine/ SubmissionManager to reflect the
> same, i.e rename prepareSubmissionRequest to prepareJobRequest
> d) The Submission term is overloaded in the code base. In one case it is used
> to represent the JobRuns( history of the jobs) and in another place it is the
> submission engine such as JobTracker/YARN/ OOzie. The act of submitting
> creates a submission record. So submission is an artifact of
> completing//failing a submit request. Prior to that it is still a job request.
> YARN is an example of a submission engine and it can do much more than
> submissions, so at some point we should revisit and outline what the
> responsibilities of the submission engine API should be,
> e) define other responsibilities of the submission engine. For instance, if
> we are to add more monitoring/ tracking into how the job execution is
> progressing, we should be able to add those apis to this entity. There is
> more room for improvement for this api
> 3. Lastly add unit test for the JobManager!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
