[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22412:
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
merged into master. Thank you [~xiepengjie] for fixing this and [~belugabehr]
for reviewing the changes!
> StatsUtils throw NPE when explain
> -
>
> Key: HIVE-22412
> URL: https://issues.apache.org/jira/browse/HIVE-22412
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.2.1, 2.0.0, 3.0.0
>Reporter: xiepengjie
>Assignee: xiepengjie
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22412.patch
>
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> The demo like this:
> {code:java}
> drop table if exists explain_npe_map;
> drop table if exists explain_npe_array;
> drop table if exists explain_npe_struct;
> create table explain_npe_map( c1 map );
> create table explain_npe_array ( c1 array );
> create table explain_npe_struct ( c1 struct );
> -- error
> set hive.cbo.enable=false;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;
> -- correct
> set hive.cbo.enable=true;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;{code}
>
> if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
> not.
> {code:java}
> hive> drop table if exists explain_npe_map;
> OK
> Time taken: 0.063 seconds
> hive> drop table if exists explain_npe_array;
> OK
> Time taken: 0.035 seconds
> hive> drop table if exists explain_npe_struct;
> OK
> Time taken: 0.015 seconds
> hive>
> > create table explain_npe_map( c1 map );
> OK
> Time taken: 0.584 seconds
> hive> create table explain_npe_array ( c1 array );
> OK
> Time taken: 0.216 seconds
> hive> create table explain_npe_struct ( c1 struct );
> OK
> Time taken: 0.17 seconds
> hive>
> > set hive.cbo.enable=false;
> hive> explain select c1 from explain_npe_map where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_array where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_struct where c1 is null;
> FAILED: RuntimeException Error invoking signature method
> hive>
> > set hive.cbo.enable=true;
> hive> explain select c1 from explain_npe_map where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_map
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: map)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_array where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_array
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: array)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_struct where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_struct
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
>
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-22412:
--
Labels: pull-request-available (was: )
> StatsUtils throw NPE when explain
> -
>
> Key: HIVE-22412
> URL: https://issues.apache.org/jira/browse/HIVE-22412
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.2.1, 2.0.0, 3.0.0
>Reporter: xiepengjie
>Assignee: xiepengjie
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22412.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The demo like this:
> {code:java}
> drop table if exists explain_npe_map;
> drop table if exists explain_npe_array;
> drop table if exists explain_npe_struct;
> create table explain_npe_map( c1 map );
> create table explain_npe_array ( c1 array );
> create table explain_npe_struct ( c1 struct );
> -- error
> set hive.cbo.enable=false;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;
> -- correct
> set hive.cbo.enable=true;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;{code}
>
> if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
> not.
> {code:java}
> hive> drop table if exists explain_npe_map;
> OK
> Time taken: 0.063 seconds
> hive> drop table if exists explain_npe_array;
> OK
> Time taken: 0.035 seconds
> hive> drop table if exists explain_npe_struct;
> OK
> Time taken: 0.015 seconds
> hive>
> > create table explain_npe_map( c1 map );
> OK
> Time taken: 0.584 seconds
> hive> create table explain_npe_array ( c1 array );
> OK
> Time taken: 0.216 seconds
> hive> create table explain_npe_struct ( c1 struct );
> OK
> Time taken: 0.17 seconds
> hive>
> > set hive.cbo.enable=false;
> hive> explain select c1 from explain_npe_map where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_array where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_struct where c1 is null;
> FAILED: RuntimeException Error invoking signature method
> hive>
> > set hive.cbo.enable=true;
> hive> explain select c1 from explain_npe_map where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_map
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: map)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_array where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_array
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: array)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_struct where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_struct
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: struct)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken:
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Attachment: (was: HIVE-22412.patch)
> StatsUtils throw NPE when explain
> -
>
> Key: HIVE-22412
> URL: https://issues.apache.org/jira/browse/HIVE-22412
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.2.1, 2.0.0, 3.0.0
>Reporter: xiepengjie
>Assignee: xiepengjie
>Priority: Major
> Attachments: HIVE-22412.patch
>
>
> The demo like this:
> {code:java}
> drop table if exists explain_npe_map;
> drop table if exists explain_npe_array;
> drop table if exists explain_npe_struct;
> create table explain_npe_map( c1 map );
> create table explain_npe_array ( c1 array );
> create table explain_npe_struct ( c1 struct );
> -- error
> set hive.cbo.enable=false;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;
> -- correct
> set hive.cbo.enable=true;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;{code}
>
> if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
> not.
> {code:java}
> hive> drop table if exists explain_npe_map;
> OK
> Time taken: 0.063 seconds
> hive> drop table if exists explain_npe_array;
> OK
> Time taken: 0.035 seconds
> hive> drop table if exists explain_npe_struct;
> OK
> Time taken: 0.015 seconds
> hive>
> > create table explain_npe_map( c1 map );
> OK
> Time taken: 0.584 seconds
> hive> create table explain_npe_array ( c1 array );
> OK
> Time taken: 0.216 seconds
> hive> create table explain_npe_struct ( c1 struct );
> OK
> Time taken: 0.17 seconds
> hive>
> > set hive.cbo.enable=false;
> hive> explain select c1 from explain_npe_map where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_array where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_struct where c1 is null;
> FAILED: RuntimeException Error invoking signature method
> hive>
> > set hive.cbo.enable=true;
> hive> explain select c1 from explain_npe_map where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_map
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: map)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_array where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_array
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: array)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_struct where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_struct
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: struct)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 2.932 seconds, Fetched: 20 row(s)
> hive>
> {code}
> ms error like:
> for map:
> {code:java}
>
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Attachment: HIVE-22412.patch
Status: Patch Available (was: Open)
> StatsUtils throw NPE when explain
> -
>
> Key: HIVE-22412
> URL: https://issues.apache.org/jira/browse/HIVE-22412
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.0.0, 1.2.1
>Reporter: xiepengjie
>Assignee: xiepengjie
>Priority: Major
> Attachments: HIVE-22412.patch, HIVE-22412.patch
>
>
> The demo like this:
> {code:java}
> drop table if exists explain_npe_map;
> drop table if exists explain_npe_array;
> drop table if exists explain_npe_struct;
> create table explain_npe_map( c1 map );
> create table explain_npe_array ( c1 array );
> create table explain_npe_struct ( c1 struct );
> -- error
> set hive.cbo.enable=false;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;
> -- correct
> set hive.cbo.enable=true;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;{code}
>
> if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
> not.
> {code:java}
> hive> drop table if exists explain_npe_map;
> OK
> Time taken: 0.063 seconds
> hive> drop table if exists explain_npe_array;
> OK
> Time taken: 0.035 seconds
> hive> drop table if exists explain_npe_struct;
> OK
> Time taken: 0.015 seconds
> hive>
> > create table explain_npe_map( c1 map );
> OK
> Time taken: 0.584 seconds
> hive> create table explain_npe_array ( c1 array );
> OK
> Time taken: 0.216 seconds
> hive> create table explain_npe_struct ( c1 struct );
> OK
> Time taken: 0.17 seconds
> hive>
> > set hive.cbo.enable=false;
> hive> explain select c1 from explain_npe_map where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_array where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_struct where c1 is null;
> FAILED: RuntimeException Error invoking signature method
> hive>
> > set hive.cbo.enable=true;
> hive> explain select c1 from explain_npe_map where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_map
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: map)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_array where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_array
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: array)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_struct where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_struct
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: struct)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 2.932 seconds, Fetched: 20 row(s)
> hive>
> {code}
> ms
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Attachment: HIVE-22412.patch
> StatsUtils throw NPE when explain
> -
>
> Key: HIVE-22412
> URL: https://issues.apache.org/jira/browse/HIVE-22412
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.2.1, 2.0.0, 3.0.0
>Reporter: xiepengjie
>Assignee: xiepengjie
>Priority: Major
> Attachments: HIVE-22412.patch
>
>
> The demo like this:
> {code:java}
> drop table if exists explain_npe_map;
> drop table if exists explain_npe_array;
> drop table if exists explain_npe_struct;
> create table explain_npe_map( c1 map );
> create table explain_npe_array ( c1 array );
> create table explain_npe_struct ( c1 struct );
> -- error
> set hive.cbo.enable=false;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;
> -- correct
> set hive.cbo.enable=true;
> explain select c1 from explain_npe_map where c1 is null;
> explain select c1 from explain_npe_array where c1 is null;
> explain select c1 from explain_npe_struct where c1 is null;{code}
>
> if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
> not.
> {code:java}
> hive> drop table if exists explain_npe_map;
> OK
> Time taken: 0.063 seconds
> hive> drop table if exists explain_npe_array;
> OK
> Time taken: 0.035 seconds
> hive> drop table if exists explain_npe_struct;
> OK
> Time taken: 0.015 seconds
> hive>
> > create table explain_npe_map( c1 map );
> OK
> Time taken: 0.584 seconds
> hive> create table explain_npe_array ( c1 array );
> OK
> Time taken: 0.216 seconds
> hive> create table explain_npe_struct ( c1 struct );
> OK
> Time taken: 0.17 seconds
> hive>
> > set hive.cbo.enable=false;
> hive> explain select c1 from explain_npe_map where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_array where c1 is null;
> FAILED: NullPointerException null
> hive> explain select c1 from explain_npe_struct where c1 is null;
> FAILED: RuntimeException Error invoking signature method
> hive>
> > set hive.cbo.enable=true;
> hive> explain select c1 from explain_npe_map where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_map
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: map)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_array where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_array
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: array)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
> hive> explain select c1 from explain_npe_struct where c1 is null;
> OK
> STAGE DEPENDENCIES:
> Stage-0 is a root stageSTAGE PLANS:
> Stage: Stage-0
> Fetch Operator
> limit: -1
> Processor Tree:
> TableScan
> alias: explain_npe_struct
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Filter Operator
> predicate: false (type: boolean)
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
> stats: NONE
> Select Operator
> expressions: c1 (type: struct)
> outputColumnNames: _col0
> Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL
> Column stats: NONE
> ListSinkTime taken: 2.932 seconds, Fetched: 20 row(s)
> hive>
> {code}
> ms error like:
> for map:
> {code:java}
>
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Description:
The demo like this:
{code:java}
drop table if exists explain_npe_map;
drop table if exists explain_npe_array;
drop table if exists explain_npe_struct;
create table explain_npe_map( c1 map );
create table explain_npe_array ( c1 array );
create table explain_npe_struct ( c1 struct );
-- error
set hive.cbo.enable=false;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;
-- correct
set hive.cbo.enable=true;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;{code}
if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
not.
{code:java}
hive> drop table if exists explain_npe_map;
OK
Time taken: 0.063 seconds
hive> drop table if exists explain_npe_array;
OK
Time taken: 0.035 seconds
hive> drop table if exists explain_npe_struct;
OK
Time taken: 0.015 seconds
hive>
> create table explain_npe_map( c1 map );
OK
Time taken: 0.584 seconds
hive> create table explain_npe_array ( c1 array );
OK
Time taken: 0.216 seconds
hive> create table explain_npe_struct ( c1 struct );
OK
Time taken: 0.17 seconds
hive>
> set hive.cbo.enable=false;
hive> explain select c1 from explain_npe_map where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_array where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_struct where c1 is null;
FAILED: RuntimeException Error invoking signature method
hive>
> set hive.cbo.enable=true;
hive> explain select c1 from explain_npe_map where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_map
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: map)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_array where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_array
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: array)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_struct where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_struct
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: struct)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 2.932 seconds, Fetched: 20 row(s)
hive>
{code}
ms error like:
for map:
{code:java}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1045)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:931)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOfVariableLengthTypes(StatsUtils.java:869)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:526)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:223)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124)
at
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Description:
The demo like this:
{code:java}
drop table if exists explain_npe_map;
drop table if exists explain_npe_array;
drop table if exists explain_npe_struct;
create table explain_npe_map( c1 map );
create table explain_npe_array ( c1 array );
create table explain_npe_struct ( c1 struct );
-- error
set hive.cbo.enable=false;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;
-- correct
set hive.cbo.enable=true;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;{code}
if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
not.
{code:java}
hive> drop table if exists explain_npe_map;
OK
Time taken: 0.063 seconds
hive> drop table if exists explain_npe_array;
OK
Time taken: 0.035 seconds
hive> drop table if exists explain_npe_struct;
OK
Time taken: 0.015 seconds
hive>
> create table explain_npe_map( c1 map );
OK
Time taken: 0.584 seconds
hive> create table explain_npe_array ( c1 array );
OK
Time taken: 0.216 seconds
hive> create table explain_npe_struct ( c1 struct );
OK
Time taken: 0.17 seconds
hive>
> set hive.cbo.enable=false;
hive> explain select c1 from explain_npe_map where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_array where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_struct where c1 is null;
FAILED: RuntimeException Error invoking signature method
hive>
> set hive.cbo.enable=true;
hive> explain select c1 from explain_npe_map where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_map
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: map)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_array where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_array
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: array)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_struct where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_struct
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: struct)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 2.932 seconds, Fetched: 20 row(s)
hive>
{code}
ms error like:
for map:
{code:java}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1045)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:931)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOfVariableLengthTypes(StatsUtils.java:869)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:526)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:223)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124)
at
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Description:
The demo like this:
{code:java}
drop table if exists explain_npe_map;
drop table if exists explain_npe_array;
drop table if exists explain_npe_struct;
create table explain_npe_map( c1 map );
create table explain_npe_array ( c1 array );
create table explain_npe_struct ( c1 struct );
-- error
set hive.cbo.enable=false;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;
-- correct
set hive.cbo.enable=true;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;{code}
if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
not.
{code:java}
hive> drop table if exists explain_npe_map;
OK
Time taken: 0.063 seconds
hive> drop table if exists explain_npe_array;
OK
Time taken: 0.035 seconds
hive> drop table if exists explain_npe_struct;
OK
Time taken: 0.015 seconds
hive>
> create table explain_npe_map( c1 map );
OK
Time taken: 0.584 seconds
hive> create table explain_npe_array ( c1 array );
OK
Time taken: 0.216 seconds
hive> create table explain_npe_struct ( c1 struct );
OK
Time taken: 0.17 seconds
hive>
> set hive.cbo.enable=false;
hive> explain select c1 from explain_npe_map where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_array where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_struct where c1 is null;
FAILED: RuntimeException Error invoking signature method
hive>
> set hive.cbo.enable=true;
hive> explain select c1 from explain_npe_map where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_map
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: map)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_array where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_array
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: array)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_struct where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_struct
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: struct)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 2.932 seconds, Fetched: 20 row(s)
hive>
{code}
ms error like:
for map:
{code:java}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1045)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:931)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOfVariableLengthTypes(StatsUtils.java:869)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:526)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:223)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124)
at
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Description:
The demo like this:
{code:java}
drop table if exists explain_npe_map;
drop table if exists explain_npe_array;
drop table if exists explain_npe_struct;
create table explain_npe_map( c1 map );
create table explain_npe_array ( c1 array );
create table explain_npe_struct ( c1 struct );
-- error
set hive.cbo.enable=false;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;
-- correct
set hive.cbo.enable=true;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;{code}
if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
not.
{code:java}
hive> drop table if exists explain_npe_map;
OK
Time taken: 0.063 seconds
hive> drop table if exists explain_npe_array;
OK
Time taken: 0.035 seconds
hive> drop table if exists explain_npe_struct;
OK
Time taken: 0.015 seconds
hive>
> create table explain_npe_map( c1 map );
OK
Time taken: 0.584 seconds
hive> create table explain_npe_array ( c1 array );
OK
Time taken: 0.216 seconds
hive> create table explain_npe_struct ( c1 struct );
OK
Time taken: 0.17 seconds
hive>
> set hive.cbo.enable=false;
hive> explain select c1 from explain_npe_map where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_array where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_struct where c1 is null;
FAILED: RuntimeException Error invoking signature method
hive>
> set hive.cbo.enable=true;
hive> explain select c1 from explain_npe_map where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_map
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: map)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_array where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_array
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: array)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_struct where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_struct
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: struct)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 2.932 seconds, Fetched: 20 row(s)
hive>
{code}
ms error like:
for map:
{code:java}
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1045)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:931)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOfVariableLengthTypes(StatsUtils.java:869)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:526)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:223)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124)
at
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Description:
The demo like this:
{code:java}
drop table if exists explain_npe_map;
drop table if exists explain_npe_array;
drop table if exists explain_npe_struct;
create table explain_npe_map( c1 map );
create table explain_npe_array ( c1 array );
create table explain_npe_struct ( c1 struct );
-- error
set hive.cbo.enable=false;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;
-- correct
set hive.cbo.enable=true;
explain select c1 from explain_npe_map where c1 is null;
explain select c1 from explain_npe_array where c1 is null;
explain select c1 from explain_npe_struct where c1 is null;{code}
if the conf 'hive.cbo.enable' set false , NPE will be thrown ; otherwise will
not.
{code:java}
hive> drop table if exists explain_npe_map;
OK
Time taken: 0.063 seconds
hive> drop table if exists explain_npe_array;
OK
Time taken: 0.035 seconds
hive> drop table if exists explain_npe_struct;
OK
Time taken: 0.015 seconds
hive>
> create table explain_npe_map( c1 map );
OK
Time taken: 0.584 seconds
hive> create table explain_npe_array ( c1 array );
OK
Time taken: 0.216 seconds
hive> create table explain_npe_struct ( c1 struct );
OK
Time taken: 0.17 seconds
hive>
> set hive.cbo.enable=false;
hive> explain select c1 from explain_npe_map where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_array where c1 is null;
FAILED: NullPointerException null
hive> explain select c1 from explain_npe_struct where c1 is null;
FAILED: RuntimeException Error invoking signature method
hive>
> set hive.cbo.enable=true;
hive> explain select c1 from explain_npe_map where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_map
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: map)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.593 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_array where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_array
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: array)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 1.969 seconds, Fetched: 20 row(s)
hive> explain select c1 from explain_npe_struct where c1 is null;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stageSTAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: explain_npe_struct
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Filter Operator
predicate: false (type: boolean)
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
Select Operator
expressions: c1 (type: struct)
outputColumnNames: _col0
Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column
stats: NONE
ListSinkTime taken: 2.932 seconds, Fetched: 20 row(s)
hive>
{code}
ms error like:
{code:java}
2019-10-10 09:11:52,670 ERROR ql.Driver (SessionState.java:printError(1068)) -
FAILED: NullPointerException null
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1045)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:931)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOfVariableLengthTypes(StatsUtils.java:869)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:526)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:223)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136)
at
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Affects Version/s: 1.2.1
2.0.0
3.0.0
> StatsUtils throw NPE when explain
> -
>
> Key: HIVE-22412
> URL: https://issues.apache.org/jira/browse/HIVE-22412
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 1.2.1, 2.0.0, 3.0.0
>Reporter: xiepengjie
>Assignee: xiepengjie
>Priority: Major
>
> The demo like this:
> {code:java}
> set hive.cbo.enable=false;
> create table explain_npe ( c1 map );
> explain select c1 from explain_npe where c1 is null;
> create table explain_npe_1 ( c1 array );
> explain select c1 from explain_npe_1 where c1 is null;{code}
> error like:
> {code:java}
> 2019-10-10 09:11:52,670 ERROR ql.Driver (SessionState.java:printError(1068))
> - FAILED: NullPointerException null
> java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1045)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:931)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOfVariableLengthTypes(StatsUtils.java:869)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:526)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:223)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136)
> at
> org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$TableScanStatsRule.process(StatsRulesProcFactory.java:111)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:95)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:79)
> at
> org.apache.hadoop.hive.ql.lib.PreOrderWalker.walk(PreOrderWalker.java:56)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:110)
> at
> org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)
> at
> org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:192)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10205)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:210)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:425)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1153)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1206)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22412) StatsUtils throw NPE when explain
[
https://issues.apache.org/jira/browse/HIVE-22412?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22412:
--
Description:
The demo like this:
{code:java}
set hive.cbo.enable=false;
create table explain_npe ( c1 map );
explain select c1 from explain_npe where c1 is null;
create table explain_npe_1 ( c1 array );
explain select c1 from explain_npe_1 where c1 is null;{code}
error like:
{code:java}
2019-10-10 09:11:52,670 ERROR ql.Driver (SessionState.java:printError(1068)) -
FAILED: NullPointerException null
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1045)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:931)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOfVariableLengthTypes(StatsUtils.java:869)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:526)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:223)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:124)
at
org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$TableScanStatsRule.process(StatsRulesProcFactory.java:111)
at
org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:95)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:79)
at
org.apache.hadoop.hive.ql.lib.PreOrderWalker.walk(PreOrderWalker.java:56)
at
org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:110)
at
org.apache.hadoop.hive.ql.optimizer.stats.annotation.AnnotateWithStatistics.transform(AnnotateWithStatistics.java:78)
at
org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:192)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10205)
at
org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:210)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at
org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:425)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:309)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1153)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1206)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1082)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
{code}
was:
The demo like this:
{code:java}
create table explain_npe ( c1 map );
explain select c1 from explain_npe where c1 is null;{code}
error like:
{code:java}
2019-10-10 09:11:52,670 ERROR ql.Driver (SessionState.java:printError(1068)) -
FAILED: NullPointerException null
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfMap(StatsUtils.java:1045)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getSizeOfComplexTypes(StatsUtils.java:931)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.getAvgColLenOfVariableLengthTypes(StatsUtils.java:869)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.estimateRowSizeFromSchema(StatsUtils.java:526)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:223)
at
org.apache.hadoop.hive.ql.stats.StatsUtils.collectStatistics(StatsUtils.java:136)
at
