jcamachor commented on a change in pull request #1892:
URL: https://github.com/apache/hive/pull/1892#discussion_r564004991
##########
File path:
itests/hive-blobstore/src/test/results/clientpositive/insert_into_dynamic_partitions.q.out
##########
@@ -100,7 +100,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column
stats: COMPLETE
GatherStats: false
Select Operator
- expressions: array(const struct(1,'101'),const
struct(2,'202'),const struct(3,'303'),const struct(4,'404'),const
struct(5,'505')) (type: array<struct<col1:int,col2:string>>)
+ expressions:
array(named_struct('col1',1,'col2','101'),named_struct('col1',2,'col2','202'),named_struct('col1',3,'col2','303'),named_struct('col1',4,'col2','404'),named_struct('col1',5,'col2','505'))
(type: array<struct<col1:int,col2:string>>)
Review comment:
This should not change, i.e., it seems we are failing to recognize
constant structs now. Iirc this is important because there are specific
optimization on execution for some of these specific data structures (also for
translation to other engines). The consolidation could possibly be done in
constant folding (`ConstantPropagateProcFactory`) if the changes in the parser
need to be done this way.
##########
File path:
itests/hive-blobstore/src/test/results/clientpositive/insert_into_dynamic_partitions.q.out
##########
@@ -100,7 +100,7 @@ STAGE PLANS:
Statistics: Num rows: 1 Data size: 10 Basic stats: COMPLETE Column
stats: COMPLETE
GatherStats: false
Select Operator
- expressions: array(const struct(1,'101'),const
struct(2,'202'),const struct(3,'303'),const struct(4,'404'),const
struct(5,'505')) (type: array<struct<col1:int,col2:string>>)
+ expressions:
array(named_struct('col1',1,'col2','101'),named_struct('col1',2,'col2','202'),named_struct('col1',3,'col2','303'),named_struct('col1',4,'col2','404'),named_struct('col1',5,'col2','505'))
(type: array<struct<col1:int,col2:string>>)
Review comment:
This should not change, i.e., it seems we are failing to recognize
constant structs now. Iirc this is important because there are specific
optimization on execution for some of these specific data structures (also for
translation to other engines). The consolidation could possibly be done when we
are creating the Calcite plan?
##########
File path: ql/src/test/results/clientpositive/llap/values.q.out
##########
@@ -10,7 +10,7 @@ POSTHOOK: Input: _dummy_database@_dummy_table
#### A masked pattern was here ####
Explain
CBO PLAN:
-HiveTableFunctionScan(invocation=[inline(ARRAY(ROW(1, 2, 3), ROW(4, 5, 6)))],
rowType=[RecordType(INTEGER col1, INTEGER col2, INTEGER col3)])
Review comment:
Related to the previous comment, then this would not change (it's
important it is recognized as ROW).
##########
File path: parser/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g
##########
@@ -151,11 +182,46 @@ expressionsNotInParenthesis[boolean isStruct, boolean
forceStruct]
-> {$more.tree}
;
-expressionPart[CommonTree t, boolean isStruct]
+expressionPart[CommonTree firstExprTree, boolean isStruct]
:
(COMMA expression)+
- -> {isStruct}? ^(TOK_FUNCTION Identifier["struct"] {$t} expression+)
- -> {$t} expression+
+ -> {isStruct}? ^(TOK_FUNCTION Identifier["struct"] {$firstExprTree}
expression+)
+ -> {$firstExprTree} expression+
+ ;
+
+// Parses comma separated list of expressions with optionally specified
aliases and store the aliases for further usage.
+// <expression> [<alias>] [, <expression> [<alias>]]
+firstExpressionsWithAlias
+@init { initAliases(); }
+ :
+ first=expression colAlias=identifier? (COMMA expressionWithAlias)*
+ -> {colAlias != null}? ^(TOK_FUNCTION Identifier["named_struct"] {
adaptor.create(Identifier, addAlias($colAlias.tree.getText())) } {$first.tree}
expressionWithAlias*)
Review comment:
Could we maybe identify this construct with a specific token? Would that
help to identify later in the plan generation this case?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
