[
https://issues.apache.org/jira/browse/SQOOP-1879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14245527#comment-14245527
]
Veena Basavaraj edited comment on SQOOP-1879 at 12/14/14 2:15 AM:
------------------------------------------------------------------
[~jarcec] thanks for the notes, I will fix it , you are right I will fix the
counters.
and also we should fix the update methods in JobManager to not make multiple
calls for getJob. Since I copied the code from there, that pattern is seen here
too. It is 5 times more calls to getJob than what we need.
I made a comment in SQOOP-1878 regarding multiple calls, since I became aware
that this can be very expensive in a real cluster running other processes
Correction. I should have looked at the latest revision in the patch ( I have
linked the RB) and if you see the comment, it explains why those things such
as error and counters are not set, there is no running job.
{code}
// jobs in local mode are complete at this point and do not have a running job
288
if (!isLocal()) {
289
request.getJobSubmission().setError(error(externalJobId));
290
request.getJobSubmission().setProgress(progress(externalJobId));
291
request.getJobSubmission().setCounters(counters(externalJobId));
292
request.getJobSubmission().setStatus(status(externalJobId));
293
}
294
{code}
was (Author: vybs):
[~jarcec] thanks for the notes, I will fix it , you are right I will fix the
counters.
and also we should fix the update methods in JobManager to not make multiple
calls for getJob. Since I copied the code from there, that pattern is seen here
too. It is 5 times more calls to getJob than what we need.
I made a comment in SQOOP-1878 regarding multiple calls, since I became aware
that this can be very expensive in a real cluster running other processes
Correction. I should have looked at the latest patch and if you see the
comment, it explains why those things such as error and counters are not set,
there is no running job.
// jobs in local mode are complete at this point and do not have a running job
288
if (!isLocal()) {
289
request.getJobSubmission().setError(error(externalJobId));
290
request.getJobSubmission().setProgress(progress(externalJobId));
291
request.getJobSubmission().setCounters(counters(externalJobId));
292
request.getJobSubmission().setStatus(status(externalJobId));
293
}
294
re
> Submission Engine does not set all details on SubmissionRecord in Local mode
> ----------------------------------------------------------------------------
>
> Key: SQOOP-1879
> URL: https://issues.apache.org/jira/browse/SQOOP-1879
> Project: Sqoop
> Issue Type: Sub-task
> Reporter: Veena Basavaraj
> Assignee: Veena Basavaraj
> Attachments: SQOOP-1879-v1.patch
>
>
> {code}
> // If we're in local mode than wait on completion. Local job runner do
> not
> // seems to be exposing API to get previously submitted job which makes
> // other methods of the submission engine quite useless.
> if(isLocal()) {
> job.waitForCompletion(true);
> } else {
> job.submit();
> }
> {code}
> the above code says that the job.waitForCompletion(true) is blocking and
> hence the other submisison engine apis are useless in this local mode.
> But we dont set the all attributes of the job run into the submission record
> in this case
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
