[
https://issues.apache.org/jira/browse/HIVE-15468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15902226#comment-15902226
]
Teddy Choi edited comment on HIVE-15468 at 3/9/17 12:30 AM:
------------------------------------------------------------
[~sershe], thank you. But the following message doesn't match to your
explanation. It's not vectorized because of Select expression for SELECT
operator: struct is not supported. Do you have some idea?
{noformat}
POSTHOOK: query: EXPLAIN VECTORIZATION EXPRESSION
SELECT strct, mp, lst FROM orc_create_complex
POSTHOOK: type: QUERY
Explain
PLAN VECTORIZATION:
enabled: true
enabledConditionsMet: [hive.vectorized.execution.enabled IS true]
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: orc_create_complex
Statistics: Num rows: 3 Data size: 3432 Basic stats: COMPLETE
Column stats: NONE
Select Operator
expressions: strct (type: struct<a:string,b:string>), mp
(type: map<string,string>), lst (type: array<string>)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 3 Data size: 3432 Basic stats:
COMPLETE Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 3 Data size: 3432 Basic stats:
COMPLETE Column stats: NONE
table:
input format:
org.apache.hadoop.mapred.SequenceFileInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde:
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: llap
LLAP IO: all inputs
Map Vectorization:
enabled: true
enabledConditionsMet:
hive.vectorized.use.vectorized.input.format IS true
inputFileFormats:
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
notVectorizedReason: Select expression for SELECT operator:
Data type struct<a:string,b:string> of Column[strct] not supported
vectorized: false
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
{noformat}
was (Author: teddy.choi):
[~sershe], thank you. But the following message doesn't match to your
explanation. It's not vectorized because of Select expression for SELECT
operator: struct is not supported. Do you have some idea?
{noformat}
Explain
PLAN VECTORIZATION:
enabled: true
enabledConditionsMet: [hive.vectorized.execution.enabled IS true]
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 depends on stages: Stage-1
STAGE PLANS:
Stage: Stage-1
Tez
#### A masked pattern was here ####
Vertices:
Map 1
Map Operator Tree:
TableScan
alias: orc_create_complex
Statistics: Num rows: 3 Data size: 3432 Basic stats: COMPLETE
Column stats: NONE
Select Operator
expressions: strct (type: struct<a:string,b:string>), mp
(type: map<string,string>), lst (type: array<string>)
outputColumnNames: _col0, _col1, _col2
Statistics: Num rows: 3 Data size: 3432 Basic stats:
COMPLETE Column stats: NONE
File Output Operator
compressed: false
Statistics: Num rows: 3 Data size: 3432 Basic stats:
COMPLETE Column stats: NONE
table:
input format:
org.apache.hadoop.mapred.SequenceFileInputFormat
output format:
org.apache.hadoop.hive.ql.io.HiveSequenceFileOutputFormat
serde:
org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe
Execution mode: llap
LLAP IO: all inputs
Map Vectorization:
enabled: true
enabledConditionsMet:
hive.vectorized.use.vectorized.input.format IS true
inputFileFormats:
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
notVectorizedReason: Select expression for SELECT operator:
Data type struct<a:string,b:string> of Column[strct] not supported
vectorized: false
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
ListSink
{noformat}
> Enhance the vectorized execution engine to support complex types
> ----------------------------------------------------------------
>
> Key: HIVE-15468
> URL: https://issues.apache.org/jira/browse/HIVE-15468
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
> Reporter: Chao Sun
> Assignee: Teddy Choi
>
> Currently Hive's vectorized execution engine only supports scalar types, as
> documented here:
> https://cwiki.apache.org/confluence/display/Hive/Vectorized+Query+Execution.
> To be complete, we should add support for complex types as well.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
