[jira] [Commented] (HIVE-16433) Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
[
https://issues.apache.org/jira/browse/HIVE-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968060#comment-15968060
]
Jimmy Xiang commented on HIVE-16433:
Ok, thanks for the explanation. +1
> Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
> ---
>
> Key: HIVE-16433
> URL: https://issues.apache.org/jira/browse/HIVE-16433
> Project: Hive
> Issue Type: Bug
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: HIVE-16433.000.patch
>
>
> Not nullify variable {{rj}} to avoid NPE due to race condition in ExecDriver.
> currently {{rj}} is set to null in ExecDriver.shutdown which is called from
> other thread for query cancellation. It can happen at any time. There is a
> potential race condition, the {{rj}} is still accessed after shutdown is
> called. For example: if the following code is executed right after
> ExecDriver.shutdown is called.
> {code}
> this.jobID = rj.getJobID();
> updateStatusInQueryDisplay();
> returnVal = jobExecHelper.progress(rj, jc, ctx);
> {code}
> Currently the purpose of nullifying {{rj}} is mainly to make sure
> {{rj.killJob()}} is only called once.
> I will add a flag {{jobKilled}} to make sure {{rj.killJob()}} will be only
> called once.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16433) Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
[
https://issues.apache.org/jira/browse/HIVE-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15968046#comment-15968046
]
zhihai xu commented on HIVE-16433:
--
Thanks for the review [~jxiang]! Currently I didn't have the stack trace, but
it is a potential race condition since user can cancel the query at any time,
ExecDriver.shutdown may be called at any time to set variable {{rj}} to null.
Yes, I plan to use isTaskShutdown function from hook to find out whether
ExecDriver is shutdown by user, so we can better monitor the query from outside.
> Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
> ---
>
> Key: HIVE-16433
> URL: https://issues.apache.org/jira/browse/HIVE-16433
> Project: Hive
> Issue Type: Bug
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: HIVE-16433.000.patch
>
>
> Not nullify variable {{rj}} to avoid NPE due to race condition in ExecDriver.
> currently {{rj}} is set to null in ExecDriver.shutdown which is called from
> other thread for query cancellation. It can happen at any time. There is a
> potential race condition, the {{rj}} is still accessed after shutdown is
> called. For example: if the following code is executed right after
> ExecDriver.shutdown is called.
> {code}
> this.jobID = rj.getJobID();
> updateStatusInQueryDisplay();
> returnVal = jobExecHelper.progress(rj, jc, ctx);
> {code}
> Currently the purpose of nullifying {{rj}} is mainly to make sure
> {{rj.killJob()}} is only called once.
> I will add a flag {{jobKilled}} to make sure {{rj.killJob()}} will be only
> called once.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16433) Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
[
https://issues.apache.org/jira/browse/HIVE-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967948#comment-15967948
]
Jimmy Xiang commented on HIVE-16433:
By the way, is the new method isTaskShutdown used any place?
> Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
> ---
>
> Key: HIVE-16433
> URL: https://issues.apache.org/jira/browse/HIVE-16433
> Project: Hive
> Issue Type: Bug
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: HIVE-16433.000.patch
>
>
> Not nullify variable {{rj}} to avoid NPE due to race condition in ExecDriver.
> currently {{rj}} is set to null in ExecDriver.shutdown which is called from
> other thread for query cancellation. It can happen at any time. There is a
> potential race condition, the {{rj}} is still accessed after shutdown is
> called. For example: if the following code is executed right after
> ExecDriver.shutdown is called.
> {code}
> this.jobID = rj.getJobID();
> updateStatusInQueryDisplay();
> returnVal = jobExecHelper.progress(rj, jc, ctx);
> {code}
> Currently the purpose of nullifying {{rj}} is mainly to make sure
> {{rj.killJob()}} is only called once.
> I will add a flag {{jobKilled}} to make sure {{rj.killJob()}} will be only
> called once.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16433) Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
[
https://issues.apache.org/jira/browse/HIVE-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967940#comment-15967940
]
Jimmy Xiang commented on HIVE-16433:
Can you attach a stacktrace to show the NPE problem?
> Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
> ---
>
> Key: HIVE-16433
> URL: https://issues.apache.org/jira/browse/HIVE-16433
> Project: Hive
> Issue Type: Bug
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: HIVE-16433.000.patch
>
>
> Not nullify variable {{rj}} to avoid NPE due to race condition in ExecDriver.
> currently {{rj}} is set to null in ExecDriver.shutdown which is called from
> other thread for query cancellation. It can happen at any time. There is a
> potential race condition, the {{rj}} is still accessed after shutdown is
> called. For example: if the following code is executed right after
> ExecDriver.shutdown is called.
> {code}
> this.jobID = rj.getJobID();
> updateStatusInQueryDisplay();
> returnVal = jobExecHelper.progress(rj, jc, ctx);
> {code}
> Currently the purpose of nullifying {{rj}} is mainly to make sure
> {{rj.killJob()}} is only called once.
> I will add a flag {{jobKilled}} to make sure {{rj.killJob()}} will be only
> called once.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-16433) Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
[
https://issues.apache.org/jira/browse/HIVE-16433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15967219#comment-15967219
]
zhihai xu commented on HIVE-16433:
--
Test
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
succeeded in my local build.
All these test failures are not related to my patch.
> Not nullify variable "rj" to avoid NPE due to race condition in ExecDriver.
> ---
>
> Key: HIVE-16433
> URL: https://issues.apache.org/jira/browse/HIVE-16433
> Project: Hive
> Issue Type: Bug
>Reporter: zhihai xu
>Assignee: zhihai xu
>Priority: Minor
> Attachments: HIVE-16433.000.patch
>
>
> Not nullify variable {{rj}} to avoid NPE due to race condition in ExecDriver.
> currently {{rj}} is set to null in ExecDriver.shutdown which is called from
> other thread for query cancellation. It can happen at any time. There is a
> potential race condition, the rj is still accessed after shutdown is called.
> For example: if the following code is executed right after
> ExecDriver.shutdown is called.
> {code}
> this.jobID = rj.getJobID();
> updateStatusInQueryDisplay();
> returnVal = jobExecHelper.progress(rj, jc, ctx);
> {code}
> Currently the purpose of nullifying rj is mainly to make sure
> {{rj.killJob()}} is only called once.
> I will add a flag jobKilled to make sure {{rj.killJob()}} will be only called
> once.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
