[
https://issues.apache.org/jira/browse/HIVE-10811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14559104#comment-14559104
]
Hive QA commented on HIVE-10811:
--------------------------------
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12735296/HIVE-10811.01.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4040/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/4040/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-4040/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.7.0_45-cloudera/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-4040/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 1f75e34 HIVE-9605:Remove parquet nested objects from wrapper
writable objects (Sergio Pena, reviewed by Ferdinand Xu)
+ git clean -f -d
Removing hcatalog/streaming/src/java/org/apache/hive/hcatalog/streaming/mutate/
Removing hcatalog/streaming/src/test/org/apache/hive/hcatalog/streaming/mutate/
+ git checkout master
Already on 'master'
+ git reset --hard origin/master
HEAD is now at 1f75e34 HIVE-9605:Remove parquet nested objects from wrapper
writable objects (Sergio Pena, reviewed by Ferdinand Xu)
+ git merge --ff-only origin/master
Already up-to-date.
+ git gc
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12735296 - PreCommit-HIVE-TRUNK-Build
> RelFieldTrimmer throws NoSuchElementException in some cases
> -----------------------------------------------------------
>
> Key: HIVE-10811
> URL: https://issues.apache.org/jira/browse/HIVE-10811
> Project: Hive
> Issue Type: Bug
> Components: CBO
> Reporter: Jesus Camacho Rodriguez
> Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-10811.01.patch, HIVE-10811.patch
>
>
> RelFieldTrimmer runs into NoSuchElementException in some cases.
> Stack trace:
> {noformat}
> Exception in thread "main" java.lang.AssertionError: Internal error: While
> invoking method 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Sort,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:743)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:543)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:269)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trim(RelFieldTrimmer.java:175)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:947)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:820)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:768)
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:109)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:730)
> at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:145)
> at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:105)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:607)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:244)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10048)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:207)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:227)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:424)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:308)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1122)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1170)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1059)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1049)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:213)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:536)
> ... 32 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Sort,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:743)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:543)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:269)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:210)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:499)
> ... 37 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:536)
> ... 40 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:743)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:543)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:269)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:210)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:499)
> ... 45 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:536)
> ... 48 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Aggregate,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:743)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:543)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:269)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:210)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:345)
> ... 53 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:536)
> ... 56 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:743)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:543)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:269)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:210)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:772)
> ... 61 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:536)
> ... 64 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:743)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:543)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:269)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:210)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:345)
> ... 69 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:536)
> ... 72 more
> Caused by: java.util.NoSuchElementException
> at java.util.AbstractList$Itr.next(AbstractList.java:364)
> at java.util.AbstractList.hashCode(AbstractList.java:540)
> at org.apache.calcite.util.Util.hash(Util.java:230)
> at org.apache.calcite.util.Pair.hashCode(Pair.java:87)
> at
> com.google.common.base.Equivalences$Equals.doHash(Equivalences.java:70)
> at com.google.common.base.Equivalence.hash(Equivalence.java:105)
> at com.google.common.cache.LocalCache.hash(LocalCache.java:1888)
> at com.google.common.cache.LocalCache.getIfPresent(LocalCache.java:3953)
> at
> com.google.common.cache.LocalCache$LocalManualCache.getIfPresent(LocalCache.java:4758)
> at
> org.apache.calcite.rel.type.RelDataTypeFactoryImpl.canonize(RelDataTypeFactoryImpl.java:352)
> at
> org.apache.calcite.rel.type.RelDataTypeFactoryImpl.createStructType(RelDataTypeFactoryImpl.java:148)
> at
> org.apache.calcite.rel.type.RelDataTypeFactoryImpl.createStructType(RelDataTypeFactoryImpl.java:172)
> at org.apache.calcite.plan.RelOptUtil.permute(RelOptUtil.java:2589)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:383)
> ... 77 more
> {noformat}
> query:
> {noformat}
> SELECT w_warehouse_name,
> w_warehouse_sq_ft,
> w_city,
> w_county,
> w_state,
> w_country,
> ship_carriers,
> year,
> Sum(jan_sales) AS jan_sales,
> Sum(feb_sales) AS feb_sales,
> Sum(jan_sales / w_warehouse_sq_ft) AS jan_sales_per_sq_foot,
> Sum(feb_sales / w_warehouse_sq_ft) AS feb_sales_per_sq_foot,
> Sum(jan_net) AS jan_net,
> Sum(feb_net) AS feb_net
> FROM (SELECT w_warehouse_name,
> w_warehouse_sq_ft,
> w_city,
> w_county,
> w_state,
> w_country,
> Concat('DHL', ',', 'BARIAN') AS ship_carriers,
> d_year AS year,
> Sum(CASE
> WHEN d_moy = 1 THEN ws_ext_sales_price * ws_quantity
> ELSE Cast(0 AS DECIMAL(7, 2))
> end) AS jan_sales,
> Sum(CASE
> WHEN d_moy = 2 THEN ws_ext_sales_price * ws_quantity
> ELSE Cast(0 AS DECIMAL(7, 2))
> end) AS feb_sales,
> Sum(CASE
> WHEN d_moy = 1 THEN ws_net_paid * ws_quantity
> ELSE Cast(0 AS DECIMAL(7, 2))
> end) AS jan_net,
> Sum(CASE
> WHEN d_moy = 2 THEN ws_net_paid * ws_quantity
> ELSE Cast(0 AS DECIMAL(7, 2))
> end) AS feb_net
> FROM web_sales
> JOIN warehouse
> ON web_sales.ws_warehouse_sk = warehouse.w_warehouse_sk
> JOIN date_dim
> ON web_sales.ws_sold_date_sk = date_dim.d_date_sk
> JOIN time_dim
> ON web_sales.ws_sold_time_sk = time_dim.t_time_sk
> JOIN ship_mode
> ON web_sales.ws_ship_mode_sk = ship_mode.sm_ship_mode_sk
> WHERE d_year = 2001
> AND t_time BETWEEN 30838 AND 30838 + 28800
> AND sm_carrier IN ( 'DHL', 'BARIAN' )
> GROUP BY w_warehouse_name,
> w_warehouse_sq_ft,
> w_city,
> w_county,
> w_state,
> w_country,
> d_year
> UNION DISTINCT
> SELECT w_warehouse_name,
> w_warehouse_sq_ft,
> w_city,
> w_county,
> w_state,
> w_country,
> Concat('DHL', ',', 'BARIAN') AS ship_carriers,
> d_year AS year,
> Sum(CASE
> WHEN d_moy = 1 THEN cs_sales_price * cs_quantity
> ELSE Cast(0 AS DECIMAL(7, 2))
> end) AS jan_sales,
> Sum(CASE
> WHEN d_moy = 2 THEN cs_sales_price * cs_quantity
> ELSE Cast(0 AS DECIMAL(7, 2))
> end) AS feb_sales,
> Sum(CASE
> WHEN d_moy = 1 THEN cs_net_paid_inc_tax * cs_quantity
> ELSE Cast(0 AS DECIMAL(7, 2))
> end) AS jan_net,
> Sum(CASE
> WHEN d_moy = 2 THEN cs_net_paid_inc_tax * cs_quantity
> ELSE Cast(0 AS DECIMAL(7, 2))
> end) AS feb_net
> FROM catalog_sales
> JOIN warehouse
> ON catalog_sales.cs_warehouse_sk = warehouse.w_warehouse_sk
> JOIN date_dim
> ON catalog_sales.cs_sold_date_sk = date_dim.d_date_sk
> JOIN time_dim
> ON catalog_sales.cs_sold_time_sk = time_dim.t_time_sk
> JOIN ship_mode
> ON catalog_sales.cs_ship_mode_sk = ship_mode.sm_ship_mode_sk
> WHERE d_year = 2001
> AND t_time BETWEEN 30838 AND 30838 + 28800
> AND sm_carrier IN ( 'DHL', 'BARIAN' )
> GROUP BY w_warehouse_name,
> w_warehouse_sq_ft,
> w_city,
> w_county,
> w_state,
> w_country,
> d_year) x
> GROUP BY w_warehouse_name,
> w_warehouse_sq_ft,
> w_city,
> w_county,
> w_state,
> w_country,
> ship_carriers,
> year
> ORDER BY w_warehouse_name
> LIMIT 100;
> {noformat}
> Seems like use of Sum(jan_sales / w_warehouse_sq_ft) AS
> jan_sales_per_sq_foot,
> Sum(feb_sales / w_warehouse_sq_ft) AS feb_sales_per_sq_foot in the select
> clause causing the error.
> Query works fine if I turn off cbo or remove these two columns from select
> clause.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
