[jira] [Commented] (HIVE-18191) Vectorization: When text input format is vectorized, TableScanOperator needs to not try to gather statistics
[ https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275457#comment-16275457 ] Matt McCline commented on HIVE-18191: - Test failures ARE RELATED. Perhaps this isn't the correct or full solution. > Vectorization: When text input format is vectorized, TableScanOperator needs > to not try to gather statistics > > > Key: HIVE-18191 > URL: https://issues.apache.org/jira/browse/HIVE-18191 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18191.01.patch > > > That is, to not an try to use row-mode gatherStats method... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18144) Runtime type inference error when join three table for different column type
[
https://issues.apache.org/jira/browse/HIVE-18144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275445#comment-16275445
]
Wang Haihua commented on HIVE-18144:
[~pxiong] we formerly found this bug in 2.1/2.2 and in condition of MapJoin.
For master branch, i saw query of ptest file just succeed before fixed, which
mean i have not found a way to Reproduce this bug. But from code analysis
below, i saw this bug still exists in master branch.
> Runtime type inference error when join three table for different column type
> -
>
> Key: HIVE-18144
> URL: https://issues.apache.org/jira/browse/HIVE-18144
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Affects Versions: 2.1.1, 2.2.0
>Reporter: Wang Haihua
>Assignee: Wang Haihua
> Attachments: HIVE-18144.1.patch
>
>
> Union operation with three or more table, which has different column types,
> may cause type inference error when Task execution.
> E.g, e.g. t1(with column int) union all t2(with column int) union all t3(with
> column bigint), finally should be {{bigint}},
> RowSchema of union t1 with t2, we call {{leftOp}}, should be int, then leftOp
> union t3 should finally be bigint.
> This mean RowSchema of leftOp would be {{bigint}} instead of {{int}}
> However we see in {{SemanticAnalyzer.java}}, leftOp RowSchema is finally
> {{int}} which was wrong:
> {code}
> (_col0: int|{t01-subquery1}diff_long_type,_col1:
> int|{t01-subquery1}id2,_col2: bigint|{t01-subquery1}id3)}}
> {code}
> Impacted code in SemanticAnalyzer.java:
> {code}
> if(!(leftOp instanceof UnionOperator)) {
> Operator oldChild = leftOp;
> leftOp = (Operator) leftOp.getParentOperators().get(0);
> leftOp.removeChildAndAdoptItsChildren(oldChild);
> }
> // make left a child of right
> List> child =
> new ArrayList>();
> child.add(leftOp);
> rightOp.setChildOperators(child);
> List> parent = leftOp
> .getParentOperators();
> parent.add(rightOp);
> UnionDesc uDesc = ((UnionOperator) leftOp).getConf();
> // Here we should set RowSchema of leftOp to unionoutRR's, or else the
> RowSchema of leftOp is wrong.
> // leftOp.setSchema(new RowSchema(unionoutRR.getColumnInfos()));
> uDesc.setNumInputs(uDesc.getNumInputs() + 1);
> return putOpInsertMap(leftOp, unionoutRR);
> {code}
> Operation for reproduce：
> {code}
> create table test_union_different_type(id bigint, id2 bigint, id3 bigint,
> name string);
> set hive.auto.convert.join=true;
> insert overwrite table test_union_different_type select 1, 2, 3,
> "test_union_different_type";
> select
> t01.diff_long_type as diff_long_type,
> t01.id2 as id2,
> t00.id as id,
> t01.id3 as id3
> from test_union_different_type t00
> left join
> (
> select 1 as diff_long_type, 30 as id2, id3 from test_union_different_type
> union ALL
> select 2 as diff_long_type, 20 as id2, id3 from test_union_different_type
> union ALL
> select id as diff_long_type, id2, 30 as id3 from test_union_different_type
> ) t01
> on t00.id = t01.diff_long_type
> ;
> {code}
> Stack trace:
> {code}
> Diagnostic Messages for this Task:
> Error: java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row {"id":1,"id2":null,"id3":null,"name":null}
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:169)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row {"id":1,"id2":null,"id3":null,"name":null}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:499)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:160)
> ... 8 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected
> exception from MapJoinOperator : org.apache.hadoop.io.LongWritable cannot be
> cast to org.apache.hadoop.io.IntWritable
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:465)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:879)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(Ta
[jira] [Commented] (HIVE-18196) Druid Mini Cluster to run Qtests integrations tests.
[
https://issues.apache.org/jira/browse/HIVE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275420#comment-16275420
]
Hive QA commented on HIVE-18196:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900329/HIVE-18196.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8083/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8083/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8083/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-12-02 05:13:31.809
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8083/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-12-02 05:13:31.811
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
5e939d4..5f12cb8 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 5e939d4 HIVE-17853: RetryingMetaStoreClient loses UGI
impersonation-context when reconnecting after timeout (Chris Drome, reviewed by
Mithun Radhakrishnan)
+ git clean -f -d
Removing ql/src/test/queries/clientpositive/vector_gather_stats.q
Removing ql/src/test/results/clientpositive/vector_gather_stats.q.out
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 5f12cb8 HIVE-17856 : MM tables - IOW is not ACID compliant
(Steve Yeom, reviewed by Sergey Shelukhin and Eugene Koifman)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-12-02 05:13:35.732
+ rm -rf ../yetus
+ mkdir ../yetus
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8083/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: patch -p1
patching file common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
patching file data/scripts/q_test_cleanup_druid.sql
patching file data/scripts/q_test_druid_init.sql
patching file druid-handler/pom.xml
patching file
druid-handler/src/java/org/apache/hadoop/hive/druid/DruidStorageHandler.java
patching file
druid-handler/src/java/org/apache/hadoop/hive/druid/DruidStorageHandlerUtils.java
patching file
druid-handler/src/test/org/apache/hadoop/hive/druid/TestDruidStorageHandler.java
patching file
druid-handler/src/test/org/apache/hadoop/hive/ql/io/TestDruidRecordWriter.java
patching file itests/pom.xml
patching file itests/qtest-druid/pom.xml
patching file
itests/qtest-druid/src/main/java/org/apache/hive/druid/DruidNode.java
patching file
itests/qtest-druid/src/main/java/org/apache/hive/druid/ForkingDruidNode.java
patching file
itests/qtest-druid/src/main/java/org/apache/hive/druid/MiniDruidCluster.java
patching file itests/qtest/pom.xml
patching file
itests/qtest/src/test/java/org/apache/hadoop/hive/cli/TestMiniDruidCliDriver.java
patching file itests/src/test/resources/testconfiguration.properties
patching file itests/util/pom.xml
patching file
itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CliConfigs.java
patching file itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java
patching file pom.xml
patching file ql/src/test/queries/clientpositive/druidmini_test1.q
patching file ql/src/test/queries/clientpositive/druidmini_test_insert.q
patching file ql/src/test/results/clientpositive/druid/druidmini_test1.q.out
patching file
ql/src/test/results/clientpositive/druid/druidmini_test_insert.q.out
+ [[ maven == \m\a\v\e\n ]
[jira] [Updated] (HIVE-18196) Druid Mini Cluster to run Qtests integrations tests.
[ https://issues.apache.org/jira/browse/HIVE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-18196: -- Attachment: HIVE-18196.2.patch update the jersey dependency to 1.19.3 (was 1.14) > Druid Mini Cluster to run Qtests integrations tests. > > > Key: HIVE-18196 > URL: https://issues.apache.org/jira/browse/HIVE-18196 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: Ashutosh Chauhan > Attachments: HIVE-18196.2.patch, HIVE-18196.patch > > > The overall Goal of this is to add a new Module that can fork a druid cluster > to run integration testing as part of the Mini Clusters Qtest suite. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18191) Vectorization: When text input format is vectorized, TableScanOperator needs to not try to gather statistics
[
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275386#comment-16275386
]
Hive QA commented on HIVE-18191:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900316/HIVE-18191.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 11498 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_1]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_tablesample_rows]
(batchId=51)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_1]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_2]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_grouping_window]
(batchId=169)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=227)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8082/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8082/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8082/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12900316 - PreCommit-HIVE-Build
> Vectorization: When text input format is vectorized, TableScanOperator needs
> to not try to gather statistics
>
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch
>
>
> That is, to not an try to use row-mode gatherStats method...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-17856:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Committed to master after a minor rebase.
Thanks for all the work on the patch!
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Fix For: 3.0.0
>
> Attachments: HIVE-17856.1.patch, HIVE-17856.10.patch,
> HIVE-17856.11.patch, HIVE-17856.12.patch, HIVE-17856.13.patch,
> HIVE-17856.14.patch, HIVE-17856.15.patch, HIVE-17856.16.patch,
> HIVE-17856.17.patch, HIVE-17856.2.patch, HIVE-17856.3.patch,
> HIVE-17856.4.patch, HIVE-17856.5.patch, HIVE-17856.6.patch,
> HIVE-17856.7.patch, HIVE-17856.8.patch, HIVE-17856.9.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18191) Vectorization: When text input format is vectorized, TableScanOperator needs to not try to gather statistics
[
https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275366#comment-16275366
]
Hive QA commented on HIVE-18191:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
32s{color} | {color:red} ql: The patch generated 1 new + 5 unchanged - 0 fixed
= 6 total (was 5) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 5e939d4 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8082/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8082/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: When text input format is vectorized, TableScanOperator needs
> to not try to gather statistics
>
>
> Key: HIVE-18191
> URL: https://issues.apache.org/jira/browse/HIVE-18191
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18191.01.patch
>
>
> That is, to not an try to use row-mode gatherStats method...
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[
https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275352#comment-16275352
]
Hive QA commented on HIVE-18054:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900300/HIVE-18054.9.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11498 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join]
(batchId=44)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39]
(batchId=249)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=227)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8081/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8081/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8081/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12900300 - PreCommit-HIVE-Build
> Make Lineage work with concurrent queries on a Session
> ---
>
> Key: HIVE-18054
> URL: https://issues.apache.org/jira/browse/HIVE-18054
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch,
> HIVE-18054.3.patch, HIVE-18054.4.patch, HIVE-18054.5.patch,
> HIVE-18054.6.patch, HIVE-18054.7.patch, HIVE-18054.8.patch, HIVE-18054.9.patch
>
>
> A Hive Session can contain multiple concurrent sql Operations.
> Lineage is currently tracked in SessionState and is cleared when a query
> completes. This results in Lineage for other running queries being lost.
> To fix this, move LineageState from SessionState to QueryState.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18191) Vectorization: When text input format is vectorized, TableScanOperator needs to not try to gather statistics
[ https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275319#comment-16275319 ] Matt McCline commented on HIVE-18191: - Needs quick code review (tests pending). > Vectorization: When text input format is vectorized, TableScanOperator needs > to not try to gather statistics > > > Key: HIVE-18191 > URL: https://issues.apache.org/jira/browse/HIVE-18191 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18191.01.patch > > > That is, to not an try to use row-mode gatherStats method... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18191) Vectorization: When text input format is vectorized, TableScanOperator needs to not try to gather statistics
[ https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18191: Status: Patch Available (was: Open) > Vectorization: When text input format is vectorized, TableScanOperator needs > to not try to gather statistics > > > Key: HIVE-18191 > URL: https://issues.apache.org/jira/browse/HIVE-18191 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18191.01.patch > > > That is, to not an try to use row-mode gatherStats method... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18191) Vectorization: When text input format is vectorized, TableScanOperator needs to not try to gather statistics
[ https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18191: Attachment: HIVE-18191.01.patch > Vectorization: When text input format is vectorized, TableScanOperator needs > to not try to gather statistics > > > Key: HIVE-18191 > URL: https://issues.apache.org/jira/browse/HIVE-18191 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18191.01.patch > > > That is, to not an try to use row-mode gatherStats method... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18191) Vectorization: When text input format is vectorized, TableScanOperator needs to not try to gather statistics
[ https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18191: Attachment: (was: HIVE-18191.01.patch) > Vectorization: When text input format is vectorized, TableScanOperator needs > to not try to gather statistics > > > Key: HIVE-18191 > URL: https://issues.apache.org/jira/browse/HIVE-18191 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18191.01.patch > > > That is, to not an try to use row-mode gatherStats method... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18191) Vectorization: When text input format is vectorized, TableScanOperator needs to not try to gather statistics
[ https://issues.apache.org/jira/browse/HIVE-18191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18191: Attachment: HIVE-18191.01.patch > Vectorization: When text input format is vectorized, TableScanOperator needs > to not try to gather statistics > > > Key: HIVE-18191 > URL: https://issues.apache.org/jira/browse/HIVE-18191 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-18191.01.patch > > > That is, to not an try to use row-mode gatherStats method... -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[
https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275305#comment-16275305
]
Hive QA commented on HIVE-18054:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
28s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
56s{color} | {color:red} ql: The patch generated 13 new + 2535 unchanged - 21
fixed = 2548 total (was 2556) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 5e939d4 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8081/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8081/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Make Lineage work with concurrent queries on a Session
> ---
>
> Key: HIVE-18054
> URL: https://issues.apache.org/jira/browse/HIVE-18054
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch,
> HIVE-18054.3.patch, HIVE-18054.4.patch, HIVE-18054.5.patch,
> HIVE-18054.6.patch, HIVE-18054.7.patch, HIVE-18054.8.patch, HIVE-18054.9.patch
>
>
> A Hive Session can contain multiple concurrent sql Operations.
> Lineage is currently tracked in SessionState and is cleared when a query
> completes. This results in Lineage for other running queries being lost.
> To fix this, move LineageState from SessionState to QueryState.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18196) Druid Mini Cluster to run Qtests integrations tests.
[
https://issues.apache.org/jira/browse/HIVE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275237#comment-16275237
]
Hive QA commented on HIVE-18196:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900287/HIVE-18196.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 458 failed/errored test(s), 9979 tests
executed
*Failed tests:*
{noformat}
TestBeeLineWithArgs - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestBeelineConnectionUsingHiveSite - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestBeelinePasswordOption - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestClientSideAuthorizationProvider - did not produce a TEST-*.xml file (likely
timed out) (batchId=229)
TestCustomAuthentication - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestHiveAuthorizerShowFilters - did not produce a TEST-*.xml file (likely timed
out) (batchId=229)
TestHs2HooksWithMiniKdc - did not produce a TEST-*.xml file (likely timed out)
(batchId=248)
TestJdbcWithDBTokenStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=248)
TestMetaStoreEventListener - did not produce a TEST-*.xml file (likely timed
out) (batchId=219)
TestRemoteHiveMetaStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=217)
TestSSLWithMiniKdc - did not produce a TEST-*.xml file (likely timed out)
(batchId=248)
TestSetUGIOnBothClientServer - did not produce a TEST-*.xml file (likely timed
out) (batchId=213)
TestSetUGIOnOnlyClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=212)
TestSetUGIOnOnlyServer - did not produce a TEST-*.xml file (likely timed out)
(batchId=222)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_custom_key2]
(batchId=239)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_custom_key]
(batchId=239)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_index]
(batchId=239)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_joins]
(batchId=239)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=239)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=239)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=239)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[buckets]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[create_database]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[create_like]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_blobstore_to_blobstore]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_blobstore_to_hdfs]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_hdfs_to_blobstore]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[explain]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[having]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_blobstore]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_local]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_blobstore_to_warehouse]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_addpartition_local_to_blobstore]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_blobstore]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_blobstore_nonpart]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_local]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_blobstore_to_warehouse_nonpart]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[import_local_to_blobstore]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_blobstore_to_blobstore]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_empty_into_blobstore]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table]
(batchId=250)
org.apache.hadoop.hive.cli.TestBlobstoreCl
[jira] [Updated] (HIVE-18163) Stats: create materialized view should also collect stats
[
https://issues.apache.org/jira/browse/HIVE-18163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18163:
---
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master, thanks [~kgyrtkirk]!
> Stats: create materialized view should also collect stats
> -
>
> Key: HIVE-18163
> URL: https://issues.apache.org/jira/browse/HIVE-18163
> Project: Hive
> Issue Type: Sub-task
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Fix For: 3.0.0
>
> Attachments: HIVE-18163.01.patch, HIVE-18163.02.patch,
> HIVE-18163.03.patch
>
>
> not having basic stats on the materialized view may cause it to be "ruled
> out" as a viable alternative by the cbo.
> repro set {{set hive.stats.deserialization.factor=10.0}} in
> {{ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q}}
> blocks: HIVE-18149
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18163) Stats: create materialized view should also collect stats
[
https://issues.apache.org/jira/browse/HIVE-18163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275187#comment-16275187
]
Jesus Camacho Rodriguez commented on HIVE-18163:
+1
> Stats: create materialized view should also collect stats
> -
>
> Key: HIVE-18163
> URL: https://issues.apache.org/jira/browse/HIVE-18163
> Project: Hive
> Issue Type: Sub-task
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
> Attachments: HIVE-18163.01.patch, HIVE-18163.02.patch,
> HIVE-18163.03.patch
>
>
> not having basic stats on the materialized view may cause it to be "ruled
> out" as a viable alternative by the cbo.
> repro set {{set hive.stats.deserialization.factor=10.0}} in
> {{ql/src/test/queries/clientpositive/materialized_view_create_rewrite.q}}
> blocks: HIVE-18149
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18196) Druid Mini Cluster to run Qtests integrations tests.
[
https://issues.apache.org/jira/browse/HIVE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275181#comment-16275181
]
Hive QA commented on HIVE-18196:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} druid-handler: The patch generated 39 new + 129
unchanged - 64 fixed = 168 total (was 193) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
44s{color} | {color:red} root: The patch generated 95 new + 1277 unchanged - 64
fixed = 1372 total (was 1341) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
8s{color} | {color:red} itests/qtest-druid: The patch generated 53 new + 0
unchanged - 0 fixed = 53 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/util: The patch generated 3 new + 217 unchanged
- 0 fixed = 220 total (was 217) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 5e939d4 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8080/yetus/diff-checkstyle-druid-handler.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8080/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8080/yetus/diff-checkstyle-itests_qtest-druid.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8080/yetus/diff-checkstyle-itests_util.txt
|
| modules | C: common ql druid-handler . itests itests/qtest itests/qtest-druid
itests/util U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8080/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Druid Mini Cluster to run Qtests integrations tests.
>
>
> Key: HIVE-18196
> URL: https://issues.apache.org/jira/browse/HIVE-18196
> Project: Hive
> Issue Type: Bug
[jira] [Commented] (HIVE-17988) Replace patch utility usage with git apply in ptest
[
https://issues.apache.org/jira/browse/HIVE-17988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275171#comment-16275171
]
Sergio Peña commented on HIVE-17988:
+1
Please just make sure that you can use the patch utility to apply binary files
as well. If you already did, then go ahead and commit it.
> Replace patch utility usage with git apply in ptest
> ---
>
> Key: HIVE-17988
> URL: https://issues.apache.org/jira/browse/HIVE-17988
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Zoltan Haindrich
>Assignee: Daniel Voros
> Attachments: HIVE-17988.1.patch, HIVE-17988.2.patch
>
>
> It would be great to replace the standard diff util because {{git}} can do a
> 3-way merge - which in most cases successfull.
> This could reduce the ptest results which are erroring out because of build
> failure.
> {code}
> error: patch failed:
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:7003
> Falling back to three-way merge...
> Applied patch to
> 'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java' cleanly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[ https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18054: -- Attachment: HIVE-18054.9.patch fix some checkstyle issues > Make Lineage work with concurrent queries on a Session > --- > > Key: HIVE-18054 > URL: https://issues.apache.org/jira/browse/HIVE-18054 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch, > HIVE-18054.3.patch, HIVE-18054.4.patch, HIVE-18054.5.patch, > HIVE-18054.6.patch, HIVE-18054.7.patch, HIVE-18054.8.patch, HIVE-18054.9.patch > > > A Hive Session can contain multiple concurrent sql Operations. > Lineage is currently tracked in SessionState and is cleared when a query > completes. This results in Lineage for other running queries being lost. > To fix this, move LineageState from SessionState to QueryState. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[
https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275102#comment-16275102
]
Hive QA commented on HIVE-18054:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900282/HIVE-18054.8.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11498 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample_islocalmode_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join]
(batchId=44)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=227)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8079/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8079/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8079/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12900282 - PreCommit-HIVE-Build
> Make Lineage work with concurrent queries on a Session
> ---
>
> Key: HIVE-18054
> URL: https://issues.apache.org/jira/browse/HIVE-18054
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch,
> HIVE-18054.3.patch, HIVE-18054.4.patch, HIVE-18054.5.patch,
> HIVE-18054.6.patch, HIVE-18054.7.patch, HIVE-18054.8.patch
>
>
> A Hive Session can contain multiple concurrent sql Operations.
> Lineage is currently tracked in SessionState and is cleared when a query
> completes. This results in Lineage for other running queries being lost.
> To fix this, move LineageState from SessionState to QueryState.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18202) Automatically migrate hbase.table.name to hbase.mapreduce.hfileoutputformat.table.name for hbase-based table
[
https://issues.apache.org/jira/browse/HIVE-18202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu reassigned HIVE-18202:
---
> Automatically migrate hbase.table.name to
> hbase.mapreduce.hfileoutputformat.table.name for hbase-based table
>
>
> Key: HIVE-18202
> URL: https://issues.apache.org/jira/browse/HIVE-18202
> Project: Hive
> Issue Type: Sub-task
> Components: HBase Handler
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>
> The property name for Hbase table mapping is changed from hbase.table.name to
> hbase.mapreduce.hfileoutputformat.table.name in HBase 2.
> We can include such upgrade for existing hbase-based tables in DB upgrade
> script to automatically change such values.
> For the new tables, the query will be like:
> create table hbase_table(key int, val string) stored by
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties
> ('hbase.columns.mapping' = ':key,cf:val') tblproperties
> ('hbase.mapreduce.hfileoutputformat.table.name' =
> 'positive_hbase_handler_bulk')
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18201) Disable XPROD_EDGE for sq_count_check() created for scalar subqueries
[ https://issues.apache.org/jira/browse/HIVE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275063#comment-16275063 ] Gopal V commented on HIVE-18201: It would also be helpful to remove the cross-product warnings for this case in general, because it is a very low-impact one. > Disable XPROD_EDGE for sq_count_check() created for scalar subqueries > -- > > Key: HIVE-18201 > URL: https://issues.apache.org/jira/browse/HIVE-18201 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Nita Dembla > Attachments: query6.explain2.out > > > sq_count_check() will either return an error at runtime or a single row. In > case of query6, the subquery has avg() function that should return a single > row. Attaching the explain. > This does not need an x-prod, because it is not useful to shuffle the big > table side for a cross-product against 1 row. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18201) Disable XPROD_EDGE for sq_count_check() created for scalar subqueries
[ https://issues.apache.org/jira/browse/HIVE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-18201: --- Description: sq_count_check() will either return an error at runtime or a single row. In case of query6, the subquery has avg() function that should return a single row. Attaching the explain. This does not need an x-prod, because it is not useful to shuffle the big table side for a cross-product against 1 row. was:IBM is running into this problem with TPCDS query6. sq_count_check() will either return an error at runtime or a single row. In case of query6, the subquery has avg() function that should return a single row. Attaching the explain. > Disable XPROD_EDGE for sq_count_check() created for scalar subqueries > -- > > Key: HIVE-18201 > URL: https://issues.apache.org/jira/browse/HIVE-18201 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Nita Dembla > Attachments: query6.explain2.out > > > sq_count_check() will either return an error at runtime or a single row. In > case of query6, the subquery has avg() function that should return a single > row. Attaching the explain. > This does not need an x-prod, because it is not useful to shuffle the big > table side for a cross-product against 1 row. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Moved] (HIVE-18201) Disable XPROD_EDGE for sq_count_check() created for scalar subqueries
[ https://issues.apache.org/jira/browse/HIVE-18201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V moved TEZ-3871 to HIVE-18201: - Affects Version/s: (was: 0.9.1) 3.0.0 Key: HIVE-18201 (was: TEZ-3871) Project: Hive (was: Apache Tez) > Disable XPROD_EDGE for sq_count_check() created for scalar subqueries > -- > > Key: HIVE-18201 > URL: https://issues.apache.org/jira/browse/HIVE-18201 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Nita Dembla > Attachments: query6.explain2.out > > > IBM is running into this problem with TPCDS query6. sq_count_check() will > either return an error at runtime or a single row. In case of query6, the > subquery has avg() function that should return a single row. Attaching the > explain. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17853) RetryingMetaStoreClient loses UGI impersonation-context when reconnecting after timeout
[
https://issues.apache.org/jira/browse/HIVE-17853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17853:
Resolution: Fixed
Fix Version/s: 2.2.1
2.4.0
3.0.0
Status: Resolved (was: Patch Available)
> RetryingMetaStoreClient loses UGI impersonation-context when reconnecting
> after timeout
> ---
>
> Key: HIVE-17853
> URL: https://issues.apache.org/jira/browse/HIVE-17853
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0, 2.4.0, 2.2.1
>Reporter: Mithun Radhakrishnan
>Assignee: Chris Drome
>Priority: Critical
> Fix For: 3.0.0, 2.4.0, 2.2.1
>
> Attachments: HIVE-17853.01-branch-2.patch, HIVE-17853.01.patch
>
>
> The {{RetryingMetaStoreClient}} is used to automatically reconnect to the
> Hive metastore, after client timeout, transparently to the user.
> In case of user impersonation (e.g. Oozie super-user {{oozie}} impersonating
> a Hadoop user {{mithun}}, to run a workflow), in case of timeout, we find
> that the reconnect causes the {{UGI.doAs()}} context to be lost. Any further
> metastore operations will be attempted as the login-user ({{oozie}}), as
> opposed to the effective user ({{mithun}}).
> We should have a fix for this shortly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17853) RetryingMetaStoreClient loses UGI impersonation-context when reconnecting after timeout
[
https://issues.apache.org/jira/browse/HIVE-17853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275046#comment-16275046
]
Mithun Radhakrishnan commented on HIVE-17853:
-
Checked into {{master}}, {{branch-2}}, and {{branch-2.2}}. Thank you for the
fix, [~cdrome]! And thank you, [~vihangk1], for taking the time to review.
> RetryingMetaStoreClient loses UGI impersonation-context when reconnecting
> after timeout
> ---
>
> Key: HIVE-17853
> URL: https://issues.apache.org/jira/browse/HIVE-17853
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0, 2.4.0, 2.2.1
>Reporter: Mithun Radhakrishnan
>Assignee: Chris Drome
>Priority: Critical
> Fix For: 3.0.0, 2.4.0, 2.2.1
>
> Attachments: HIVE-17853.01-branch-2.patch, HIVE-17853.01.patch
>
>
> The {{RetryingMetaStoreClient}} is used to automatically reconnect to the
> Hive metastore, after client timeout, transparently to the user.
> In case of user impersonation (e.g. Oozie super-user {{oozie}} impersonating
> a Hadoop user {{mithun}}, to run a workflow), in case of timeout, we find
> that the reconnect causes the {{UGI.doAs()}} context to be lost. Any further
> metastore operations will be attempted as the login-user ({{oozie}}), as
> opposed to the effective user ({{mithun}}).
> We should have a fix for this shortly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (HIVE-18198) TablePropertyEnrichmentOptimizer.java is missing the Apache license header
[
https://issues.apache.org/jira/browse/HIVE-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gunther Hagleitner resolved HIVE-18198.
---
Resolution: Fixed
Fix Version/s: 3.0.0
Committed to master. Thank you [~deepesh].
> TablePropertyEnrichmentOptimizer.java is missing the Apache license header
> --
>
> Key: HIVE-18198
> URL: https://issues.apache.org/jira/browse/HIVE-18198
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Fix For: 3.0.0
>
> Attachments: HIVE-18198.patch
>
>
> This causes warnings in the yetus check:
> {quote}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /data/hiveptest/working/yetus/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TablePropertyEnrichmentOptimizer.java
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18196) Druid Mini Cluster to run Qtests integrations tests.
[ https://issues.apache.org/jira/browse/HIVE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-18196: -- Attachment: HIVE-18196.patch > Druid Mini Cluster to run Qtests integrations tests. > > > Key: HIVE-18196 > URL: https://issues.apache.org/jira/browse/HIVE-18196 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: Ashutosh Chauhan > Attachments: HIVE-18196.patch > > > The overall Goal of this is to add a new Module that can fork a druid cluster > to run integration testing as part of the Mini Clusters Qtest suite. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18196) Druid Mini Cluster to run Qtests integrations tests.
[ https://issues.apache.org/jira/browse/HIVE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-18196: -- Status: Patch Available (was: Open) > Druid Mini Cluster to run Qtests integrations tests. > > > Key: HIVE-18196 > URL: https://issues.apache.org/jira/browse/HIVE-18196 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: Ashutosh Chauhan > > The overall Goal of this is to add a new Module that can fork a druid cluster > to run integration testing as part of the Mini Clusters Qtest suite. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[
https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275038#comment-16275038
]
Hive QA commented on HIVE-18054:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
57s{color} | {color:red} ql: The patch generated 25 new + 2535 unchanged - 21
fixed = 2560 total (was 2556) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 1b4baf4 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8079/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8079/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8079/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8079/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Make Lineage work with concurrent queries on a Session
> ---
>
> Key: HIVE-18054
> URL: https://issues.apache.org/jira/browse/HIVE-18054
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch,
> HIVE-18054.3.patch, HIVE-18054.4.patch, HIVE-18054.5.patch,
> HIVE-18054.6.patch, HIVE-18054.7.patch, HIVE-18054.8.patch
>
>
> A Hive Session can contain multiple concurrent sql Operations.
> Lineage is currently tracked in SessionState and is cleared when a query
> completes. This results in Lineage for other running queries being lost.
> To fix this, move LineageState from SessionState to QueryState.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18043) Vectorization: Support List type in MapWork
[
https://issues.apache.org/jira/browse/HIVE-18043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275035#comment-16275035
]
Zoltan Haindrich commented on HIVE-18043:
-
[~colinma] I've bisected that test failure to this commit; and now I've checked
it again - but unfortunately I've came to the same conclusionso if you
would like to check it yourself, please run the following script:
{code}
#!/bin/bash
set -e
set -x
C=b8aa16ff6c2ec0185cd953a7854d6abda2306df7
M_OPTS+="
-Dorg.slf4j.simpleLogger.log.org.apache.maven.plugin.surefire.SurefirePlugin=INFO"
M_OPTS+=" -q -Dmaven.surefire.plugin.version=2.20.1"
M_OPTS+=" -Pitests -DskipSparkTests"
M_OPTS+=" -pl itests/qtest -am"
M_OPTS+=" -Dtest=TestCliDriver#testCliDriver[vector_complex_join]"
M_OPTS+=" install"
export M_OPTS
git log -n1 $C
echo "* check good"
git clean -dfx
git checkout ${C}^
time mvn $M_OPTS
echo "* check bad"
git clean -dfx
git checkout ${C}
time mvn $M_OPTS
echo "*** I was wrong..."
{code}
It might be possible that your patch have caused some minor change in
combination with another patch; which was committed arround the same time.
> Vectorization: Support List type in MapWork
> ---
>
> Key: HIVE-18043
> URL: https://issues.apache.org/jira/browse/HIVE-18043
> Project: Hive
> Issue Type: Improvement
>Reporter: Colin Ma
>Assignee: Colin Ma
> Fix For: 3.0.0
>
> Attachments: HIVE-18043.001.patch, HIVE-18043.002.patch,
> HIVE-18043.003.patch, HIVE-18043.004.patch, HIVE-18043.005.patch
>
>
> Support Complex Types in vectorization is finished in HIVE-16589, but List
> type is still not support in MapWork. It should be supported to improve the
> performance when vectorization is enable.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gunther Hagleitner updated HIVE-18195:
--
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275031#comment-16275031
]
Gunther Hagleitner commented on HIVE-18195:
---
Committed to master. Thanks [~deepesh] and [~sershe].
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Fix For: 3.0.0
>
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Work started] (HIVE-18200) Bucket Map Join : Use correct algorithm to pick the big table
[ https://issues.apache.org/jira/browse/HIVE-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-18200 started by Deepak Jaiswal. - > Bucket Map Join : Use correct algorithm to pick the big table > - > > Key: HIVE-18200 > URL: https://issues.apache.org/jira/browse/HIVE-18200 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal > > Currently the algorithm to pick the big table is flawed due to complexity > associated with n-way joins. > It could result in OOM, consider the following scenario, > CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) > CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE; > CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED > BY (key) INTO 2 BUCKETS STORED AS TEXTFILE; > Lets say tab has size of 2GB and tab_part has size of 500MB and > noconditionaltasksize is 200MB, then bucket map join should not happen as > atleast one hash table will be more than 250 MB, which may cause OOM. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18198) TablePropertyEnrichmentOptimizer.java is missing the Apache license header
[
https://issues.apache.org/jira/browse/HIVE-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275023#comment-16275023
]
Gunther Hagleitner commented on HIVE-18198:
---
+1
> TablePropertyEnrichmentOptimizer.java is missing the Apache license header
> --
>
> Key: HIVE-18198
> URL: https://issues.apache.org/jira/browse/HIVE-18198
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-18198.patch
>
>
> This causes warnings in the yetus check:
> {quote}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /data/hiveptest/working/yetus/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TablePropertyEnrichmentOptimizer.java
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18200) Bucket Map Join : Use correct algorithm to pick the big table
[ https://issues.apache.org/jira/browse/HIVE-18200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal reassigned HIVE-18200: - > Bucket Map Join : Use correct algorithm to pick the big table > - > > Key: HIVE-18200 > URL: https://issues.apache.org/jira/browse/HIVE-18200 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal > > Currently the algorithm to pick the big table is flawed due to complexity > associated with n-way joins. > It could result in OOM, consider the following scenario, > CREATE TABLE tab_part (key int, value string) PARTITIONED BY(ds STRING) > CLUSTERED BY (key) INTO 4 BUCKETS STORED AS TEXTFILE; > CREATE TABLE tab(key int, value string) PARTITIONED BY(ds STRING) CLUSTERED > BY (key) INTO 2 BUCKETS STORED AS TEXTFILE; > Lets say tab has size of 2GB and tab_part has size of 500MB and > noconditionaltasksize is 200MB, then bucket map join should not happen as > atleast one hash table will be more than 250 MB, which may cause OOM. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17853) RetryingMetaStoreClient loses UGI impersonation-context when reconnecting after timeout
[
https://issues.apache.org/jira/browse/HIVE-17853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275017#comment-16275017
]
Mithun Radhakrishnan commented on HIVE-17853:
-
I'll check this version of the fix in. I have raised HIVE-18199 to track the
addition of the unit-test.
> RetryingMetaStoreClient loses UGI impersonation-context when reconnecting
> after timeout
> ---
>
> Key: HIVE-17853
> URL: https://issues.apache.org/jira/browse/HIVE-17853
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0, 2.4.0, 2.2.1
>Reporter: Mithun Radhakrishnan
>Assignee: Chris Drome
>Priority: Critical
> Attachments: HIVE-17853.01-branch-2.patch, HIVE-17853.01.patch
>
>
> The {{RetryingMetaStoreClient}} is used to automatically reconnect to the
> Hive metastore, after client timeout, transparently to the user.
> In case of user impersonation (e.g. Oozie super-user {{oozie}} impersonating
> a Hadoop user {{mithun}}, to run a workflow), in case of timeout, we find
> that the reconnect causes the {{UGI.doAs()}} context to be lost. Any further
> metastore operations will be attempted as the login-user ({{oozie}}), as
> opposed to the effective user ({{mithun}}).
> We should have a fix for this shortly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18199) Add unit-test for lost UGI doAs() context in RetryingMetaStoreClient
[
https://issues.apache.org/jira/browse/HIVE-18199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan reassigned HIVE-18199:
---
> Add unit-test for lost UGI doAs() context in RetryingMetaStoreClient
>
>
> Key: HIVE-18199
> URL: https://issues.apache.org/jira/browse/HIVE-18199
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Mithun Radhakrishnan
>Assignee: Chris Drome
>
> This has to do with HIVE-17853. The {{RetryingMetaStoreClient}} would lose
> the {{UGI.doAs()}} context, in case of a socket timeout. The connection after
> the reconnect might cause operation failures, because of using the wrong user.
> We'll need to add a unit-test to simulate this case, if possib.e
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275013#comment-16275013
]
Hive QA commented on HIVE-18195:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900258/HIVE-18195.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 11496 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_simple_select]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join]
(batchId=44)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=227)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8078/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8078/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8078/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12900258 - PreCommit-HIVE-Build
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18127) Do not strip '--' comments from shell commands issued from CliDriver
[ https://issues.apache.org/jira/browse/HIVE-18127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275008#comment-16275008 ] Andrew Sherman commented on HIVE-18127: --- Thanks [~stakiar] > Do not strip '--' comments from shell commands issued from CliDriver > > > Key: HIVE-18127 > URL: https://issues.apache.org/jira/browse/HIVE-18127 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Fix For: 3.0.0 > > Attachments: HIVE-18127.1.patch, HIVE-18127.2.patch > > > CLiDriver has the ability to run shell commands by prefixing them with '!". > This behavior is not widely used (there are only 3 examples in .q files). > Since HIVE-16935 started stripping comments starting with '\-\-', a shell > command containing '--' will not work correctly. > Fix this by using the unstripped command for shell commands. > Note that it would be a security hole for HS2 to allow execution of arbitrary > shell commands from a client command. > Add tests to nail down correct behavior with '--' comments: > * CliDriver should not strip strings starting with '--' in a shell command > (FIXED in this change). > * HiveCli should strip '--' comments. > * A Jdbc program should allow commands starting with "!" but these will fail > in the sql parser. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17870) Update NoDeleteRollingFileAppender to use Log4j2 api
[ https://issues.apache.org/jira/browse/HIVE-17870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275005#comment-16275005 ] Andrew Sherman commented on HIVE-17870: --- Thanks [~aihuaxu] > Update NoDeleteRollingFileAppender to use Log4j2 api > > > Key: HIVE-17870 > URL: https://issues.apache.org/jira/browse/HIVE-17870 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Andrew Sherman > Fix For: 3.0.0 > > Attachments: HIVE-17870.1.patch, HIVE-17870.2.patch > > > NoDeleteRollingFileAppender is still using log4jv1 api. Since we already > moved to use log4j2 in hive, we better update to use log4jv2 as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16890) org.apache.hadoop.hive.serde2.io.HiveVarcharWritable - Adds Superfluous Wrapper
[
https://issues.apache.org/jira/browse/HIVE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam updated HIVE-16890:
-
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Fix has been committed to master. Thank you for your contribution [~belugabehr]
> org.apache.hadoop.hive.serde2.io.HiveVarcharWritable - Adds Superfluous
> Wrapper
> ---
>
> Key: HIVE-16890
> URL: https://issues.apache.org/jira/browse/HIVE-16890
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HIVE-16890.1.patch, HIVE-16890.1.patch,
> HIVE-16890.1.patch
>
>
> Class {{org.apache.hadoop.hive.serde2.io.HiveVarcharWritable}} creates a
> superfluous wrapper and then immediately unwraps it. Don't bother wrapping
> in this scenario.
> {code}
> public void set(HiveVarchar val, int len) {
> set(val.getValue(), len);
> }
> public void set(String val, int maxLength) {
> value.set(HiveBaseChar.enforceMaxLength(val, maxLength));
> }
> public HiveVarchar getHiveVarchar() {
> return new HiveVarchar(value.toString(), -1);
> }
> // Here calls getHiveVarchar() which creates a new HiveVarchar object with
> a string in it
> // The object is passed to set(HiveVarchar val, int len)
> // The string is pulled out
> public void enforceMaxLength(int maxLength) {
> // Might be possible to truncate the existing Text value, for now just do
> something simple.
> if (value.getLength()>maxLength && getCharacterLength()>maxLength)
> set(getHiveVarchar(), maxLength);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17870) Update NoDeleteRollingFileAppender to use Log4j2 api
[ https://issues.apache.org/jira/browse/HIVE-17870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-17870: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks [~asherman] for the work. > Update NoDeleteRollingFileAppender to use Log4j2 api > > > Key: HIVE-17870 > URL: https://issues.apache.org/jira/browse/HIVE-17870 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Andrew Sherman > Fix For: 3.0.0 > > Attachments: HIVE-17870.1.patch, HIVE-17870.2.patch > > > NoDeleteRollingFileAppender is still using log4jv1 api. Since we already > moved to use log4j2 in hive, we better update to use log4jv2 as well. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[ https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18054: -- Attachment: HIVE-18054.8.patch Small fixes to address review comments, + rebase. > Make Lineage work with concurrent queries on a Session > --- > > Key: HIVE-18054 > URL: https://issues.apache.org/jira/browse/HIVE-18054 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch, > HIVE-18054.3.patch, HIVE-18054.4.patch, HIVE-18054.5.patch, > HIVE-18054.6.patch, HIVE-18054.7.patch, HIVE-18054.8.patch > > > A Hive Session can contain multiple concurrent sql Operations. > Lineage is currently tracked in SessionState and is cleared when a query > completes. This results in Lineage for other running queries being lost. > To fix this, move LineageState from SessionState to QueryState. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274949#comment-16274949
]
Deepesh Khandelwal commented on HIVE-18195:
---
The Apache Yetus warning is due to missing Apache header in
ql/src/java/org/apache/hadoop/hive/ql/optimizer/TablePropertyEnrichmentOptimizer.java
that got committed as part of HIVE-14792, created a separate bug HIVE-18198 to
track that one. [~sershe] can you review that one as well?
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274944#comment-16274944
]
Sergey Shelukhin commented on HIVE-17856:
-
[~ekoifman] [~steveyeom2017] is this ready to be committed?
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.10.patch,
> HIVE-17856.11.patch, HIVE-17856.12.patch, HIVE-17856.13.patch,
> HIVE-17856.14.patch, HIVE-17856.15.patch, HIVE-17856.16.patch,
> HIVE-17856.17.patch, HIVE-17856.2.patch, HIVE-17856.3.patch,
> HIVE-17856.4.patch, HIVE-17856.5.patch, HIVE-17856.6.patch,
> HIVE-17856.7.patch, HIVE-17856.8.patch, HIVE-17856.9.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18198) TablePropertyEnrichmentOptimizer.java is missing the Apache license header
[
https://issues.apache.org/jira/browse/HIVE-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-18198:
--
Attachment: HIVE-18198.patch
> TablePropertyEnrichmentOptimizer.java is missing the Apache license header
> --
>
> Key: HIVE-18198
> URL: https://issues.apache.org/jira/browse/HIVE-18198
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
> Attachments: HIVE-18198.patch
>
>
> This causes warnings in the yetus check:
> {quote}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /data/hiveptest/working/yetus/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TablePropertyEnrichmentOptimizer.java
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18179) Implement validate resource plan.
[ https://issues.apache.org/jira/browse/HIVE-18179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274940#comment-16274940 ] Sergey Shelukhin commented on HIVE-18179: - One small nit. Action and expression stuff can be done in separate jira unless you already have it in progress. Let me know if it should it be on this jira or separate > Implement validate resource plan. > - > > Key: HIVE-18179 > URL: https://issues.apache.org/jira/browse/HIVE-18179 > Project: Hive > Issue Type: Sub-task >Reporter: Harish Jaiprakash >Assignee: Harish Jaiprakash > Attachments: HIVE-18179.01.patch, HIVE-18179.02.patch, > HIVE-18179.03.patch > > > For now mostly pool alloc fraction to add up to 1.0 and some null or invalid > value checks. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18198) TablePropertyEnrichmentOptimizer.java is missing the Apache license header
[
https://issues.apache.org/jira/browse/HIVE-18198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-18198:
-
> TablePropertyEnrichmentOptimizer.java is missing the Apache license header
> --
>
> Key: HIVE-18198
> URL: https://issues.apache.org/jira/browse/HIVE-18198
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>
> This causes warnings in the yetus check:
> {quote}
> Lines that start with ? in the ASF License report indicate files that do
> not have an Apache license header:
> !?
> /data/hiveptest/working/yetus/ql/src/java/org/apache/hadoop/hive/ql/optimizer/TablePropertyEnrichmentOptimizer.java
> {quote}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274912#comment-16274912
]
Hive QA commented on HIVE-18195:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
26s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 33d527f |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8078/yetus/patch-asflicense-problems.txt
|
| modules | C: metastore U: metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8078/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18196) Druid Mini Cluster to run Qtests integrations tests.
[ https://issues.apache.org/jira/browse/HIVE-18196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra reassigned HIVE-18196: - > Druid Mini Cluster to run Qtests integrations tests. > > > Key: HIVE-18196 > URL: https://issues.apache.org/jira/browse/HIVE-18196 > Project: Hive > Issue Type: Bug >Reporter: slim bouguerra >Assignee: Ashutosh Chauhan > > The overall Goal of this is to add a new Module that can fork a druid cluster > to run integration testing as part of the Mini Clusters Qtest suite. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18189) Order by position does not work when cbo is disabled
[
https://issues.apache.org/jira/browse/HIVE-18189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274900#comment-16274900
]
Sergey Shelukhin commented on HIVE-18189:
-
The test appears to have failed. In general, looks good. One nit:
{noformat}
node = (ASTNode) colNode.getChild(0);
+if (node != null
{noformat}
in a couple places; it seems like length needs to be checked before accessing
the element by index, and null check is unnecessary cause it will throw if it's
absent right now.
> Order by position does not work when cbo is disabled
>
>
> Key: HIVE-18189
> URL: https://issues.apache.org/jira/browse/HIVE-18189
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-18189.1.patch
>
>
> Investigating a failed query:
> {code}
> set hive.cbo.enable=false;
> set hive.orderby.position.alias=true;
> select distinct age from student order by 1 desc limit 20;
> {code}
> The query does not sort the output correctly when cbo is
> disabled/inactivated. I found two issues:
> 1. "order by position" query is broken by HIVE-16774
> 2. In particular, select distinct query never work for "order by position"
> query
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274868#comment-16274868
]
Sergey Shelukhin commented on HIVE-18195:
-
+1 pending tests
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin reassigned HIVE-18195:
---
Assignee: Deepesh Khandelwal (was: Sergey Shelukhin)
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18195:
Status: Patch Available (was: Open)
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Sergey Shelukhin
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin reassigned HIVE-18195:
---
Assignee: Sergey Shelukhin (was: Deepesh Khandelwal)
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Sergey Shelukhin
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-12679) Allow users to be able to specify an implementation of IMetaStoreClient via HiveConf
[ https://issues.apache.org/jira/browse/HIVE-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274864#comment-16274864 ] Rentao Wu commented on HIVE-12679: -- I would like to bring up this JIRA again for consideration as it allows for alternative metastore clients such to AWS Glue DataCatalog. Would the approach of changing SessionHiveMetastoreClient to have a delegate implementation of IMetastoreClient work? (Removing inheritance of extending HiveMetastoreClient impl) If that is fine approach I'll work on a new patch for this. http://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive-metastore-glue.html https://aws.amazon.com/glue/details/ > Allow users to be able to specify an implementation of IMetaStoreClient via > HiveConf > > > Key: HIVE-12679 > URL: https://issues.apache.org/jira/browse/HIVE-12679 > Project: Hive > Issue Type: Improvement > Components: Configuration, Metastore, Query Planning >Reporter: Austin Lee >Assignee: Rentao Wu >Priority: Minor > Labels: metastore > Attachments: HIVE-12679.1.patch, HIVE-12679.2.patch, HIVE-12679.patch > > > Hi, > I would like to propose a change that would make it possible for users to > choose an implementation of IMetaStoreClient via HiveConf, i.e. > hive-site.xml. Currently, in Hive the choice is hard coded to be > SessionHiveMetaStoreClient in org.apache.hadoop.hive.ql.metadata.Hive. There > is no other direct reference to SessionHiveMetaStoreClient other than the > hard coded class name in Hive.java and the QL component operates only on the > IMetaStoreClient interface so the change would be minimal and it would be > quite similar to how an implementation of RawStore is specified and loaded in > hive-metastore. One use case this change would serve would be one where a > user wishes to use an implementation of this interface without the dependency > on the Thrift server. > > Thank you, > Austin -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-12679) Allow users to be able to specify an implementation of IMetaStoreClient via HiveConf
[ https://issues.apache.org/jira/browse/HIVE-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rentao Wu reassigned HIVE-12679: Assignee: Rentao Wu (was: Austin Lee) > Allow users to be able to specify an implementation of IMetaStoreClient via > HiveConf > > > Key: HIVE-12679 > URL: https://issues.apache.org/jira/browse/HIVE-12679 > Project: Hive > Issue Type: Improvement > Components: Configuration, Metastore, Query Planning >Reporter: Austin Lee >Assignee: Rentao Wu >Priority: Minor > Labels: metastore > Attachments: HIVE-12679.1.patch, HIVE-12679.2.patch, HIVE-12679.patch > > > Hi, > I would like to propose a change that would make it possible for users to > choose an implementation of IMetaStoreClient via HiveConf, i.e. > hive-site.xml. Currently, in Hive the choice is hard coded to be > SessionHiveMetaStoreClient in org.apache.hadoop.hive.ql.metadata.Hive. There > is no other direct reference to SessionHiveMetaStoreClient other than the > hard coded class name in Hive.java and the QL component operates only on the > IMetaStoreClient interface so the change would be minimal and it would be > quite similar to how an implementation of RawStore is specified and loaded in > hive-metastore. One use case this change would serve would be one where a > user wishes to use an implementation of this interface without the dependency > on the Thrift server. > > Thank you, > Austin -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-12679) Allow users to be able to specify an implementation of IMetaStoreClient via HiveConf
[ https://issues.apache.org/jira/browse/HIVE-12679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rentao Wu updated HIVE-12679: - Affects Version/s: (was: 2.1.0) Status: Open (was: Patch Available) > Allow users to be able to specify an implementation of IMetaStoreClient via > HiveConf > > > Key: HIVE-12679 > URL: https://issues.apache.org/jira/browse/HIVE-12679 > Project: Hive > Issue Type: Improvement > Components: Configuration, Metastore, Query Planning >Reporter: Austin Lee >Assignee: Rentao Wu >Priority: Minor > Labels: metastore > Attachments: HIVE-12679.1.patch, HIVE-12679.2.patch, HIVE-12679.patch > > > Hi, > I would like to propose a change that would make it possible for users to > choose an implementation of IMetaStoreClient via HiveConf, i.e. > hive-site.xml. Currently, in Hive the choice is hard coded to be > SessionHiveMetaStoreClient in org.apache.hadoop.hive.ql.metadata.Hive. There > is no other direct reference to SessionHiveMetaStoreClient other than the > hard coded class name in Hive.java and the QL component operates only on the > IMetaStoreClient interface so the change would be minimal and it would be > quite similar to how an implementation of RawStore is specified and loaded in > hive-metastore. One use case this change would serve would be one where a > user wishes to use an implementation of this interface without the dependency > on the Thrift server. > > Thank you, > Austin -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17940) IllegalArgumentException when reading last row-group in an ORC stripe
[
https://issues.apache.org/jira/browse/HIVE-17940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17940:
Resolution: Fixed
Fix Version/s: 1.2.3
1.3.0
Status: Resolved (was: Patch Available)
I have checked this into {{branch-1}}, and {{branch-1.2}}. Thanks for the
contribution, [~selinazh], and [~cdrome]. And thank you for the review,
[~sershe]!
> IllegalArgumentException when reading last row-group in an ORC stripe
> -
>
> Key: HIVE-17940
> URL: https://issues.apache.org/jira/browse/HIVE-17940
> Project: Hive
> Issue Type: Bug
> Components: ORC
>Affects Versions: 1.3.0, 1.2.2
>Reporter: Mithun Radhakrishnan
>Assignee: Chris Drome
> Fix For: 1.3.0, 1.2.3
>
> Attachments: HIVE-17940.1-branch-1.2.patch,
> HIVE-17940.1-branch-1.patch
>
>
> (This is a backport of HIVE-10024 to {{branch-1.2}}, and {{branch-1}}.)
> When the last row-group in an ORC stripe contains fewer records than
> specified in {{$\{orc.row.index.stride\}}}, and if a column value is sparse
> (i.e. mostly nulls), then one sees the following failure when reading the ORC
> stripe:
> {noformat}
> java.lang.IllegalArgumentException: Seek in Stream for column 82 kind DATA
> to 130 is outside of the data
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1738)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.lang.IllegalArgumentException: Seek in Stream for
> column 82 kind DATA to 130 is outside of the data
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:71)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:322)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:148)
> ... 14 more
> {noformat}
> [~sershe] had a fix for this in HIVE-10024, in {{branch-2}}. After running
> into this in production with {{branch-1}}+, we find that the fix for
> HIVE-10024 sorts this out in {{branch-1}} as well.
> This is a fairly rare case, but it leads to bad reads on valid ORC files. I
> will back-port this shortly.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17949) itests compile is busted on branch-1.2
[
https://issues.apache.org/jira/browse/HIVE-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-17949:
Resolution: Fixed
Fix Version/s: 1.2.3
Status: Resolved (was: Patch Available)
> itests compile is busted on branch-1.2
> --
>
> Key: HIVE-17949
> URL: https://issues.apache.org/jira/browse/HIVE-17949
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 1.2.3
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Fix For: 1.2.3
>
> Attachments: HIVE-17949.01-branch-1.2.patch
>
>
> {{commit 18ddf46e0a8f092358725fc102235cbe6ba3e24d}} on {{branch-1.2}} was for
> {{Preparing for 1.2.3 development}}. This should have also included
> corresponding changes to all the pom-files under {{itests}}. As it stands
> now, the build fails with the following:
> {noformat}
> [ERROR] location: class org.apache.hadoop.hive.metastore.api.Role
> [ERROR]
> /Users/mithunr/workspace/dev/hive/apache/branch-1.2/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:[512,19]
> no suitable method found for
> updatePartitionStatsFast(org.apache.hadoop.hive.metastore.api.Partition,org.apache.hadoop.hive.metastore.Warehouse)
> [ERROR] method
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updatePartitionStatsFast(org.apache.hadoop.hive.metastore.api.Partition,org.apache.hadoop.hive.metastore.Warehouse,org.apache.hadoop.hive.metastore.api.EnvironmentContext)
> is not applicable
> [ERROR] (actual and formal argument lists differ in length)
> [ERROR] method
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updatePartitionStatsFast(org.apache.hadoop.hive.metastore.api.Partition,org.apache.hadoop.hive.metastore.Warehouse,boolean,org.apache.hadoop.hive.metastore.api.EnvironmentContext)
> is not applicable
> [ERROR] (actual and formal argument lists differ in length)
> [ERROR] method
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updatePartitionStatsFast(org.apache.hadoop.hive.metastore.api.Partition,org.apache.hadoop.hive.metastore.Warehouse,boolean,boolean,org.apache.hadoop.hive.metastore.api.EnvironmentContext)
> is not applicable
> [ERROR] (actual and formal argument lists differ in length)
> [ERROR] method
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updatePartitionStatsFast(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy.PartitionIterator,org.apache.hadoop.hive.metastore.Warehouse,boolean,boolean,org.apache.hadoop.hive.metastore.api.EnvironmentContext)
> is not applicable
> [ERROR] (actual and formal argument lists differ in length)
> [ERROR]
> /Users/mithunr/workspace/dev/hive/apache/branch-1.2/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreWithEnvironmentContext.java:[181,45]
> incompatible types: org.apache.hadoop.hive.metastore.api.EnvironmentContext
> cannot be converted to boolean
> [ERROR]
> /Users/mithunr/workspace/dev/hive/apache/branch-1.2/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreWithEnvironmentContext.java:[190,45]
> incompatible types: org.apache.hadoop.hive.metastore.api.EnvironmentContext
> cannot be converted to boolean
> [ERROR]
> /Users/mithunr/workspace/dev/hive/apache/branch-1.2/itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java:[53,26]
> cannot find symbol
> [ERROR] symbol: class MiniZooKeeperCluster
> [ERROR] location: class
> org.apache.hadoop.hive.thrift.TestZooKeeperTokenStore
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hive-it-unit
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17949) itests compile is busted on branch-1.2
[
https://issues.apache.org/jira/browse/HIVE-17949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274812#comment-16274812
]
Mithun Radhakrishnan commented on HIVE-17949:
-
bq. I'll check this compile fix in now.
I'm back to this, at last. I've checked this fix into {{branch-1.2}}. Thank you
for sparing the time to review, [~sershe]!
> itests compile is busted on branch-1.2
> --
>
> Key: HIVE-17949
> URL: https://issues.apache.org/jira/browse/HIVE-17949
> Project: Hive
> Issue Type: Bug
> Components: Test
>Affects Versions: 1.2.3
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Attachments: HIVE-17949.01-branch-1.2.patch
>
>
> {{commit 18ddf46e0a8f092358725fc102235cbe6ba3e24d}} on {{branch-1.2}} was for
> {{Preparing for 1.2.3 development}}. This should have also included
> corresponding changes to all the pom-files under {{itests}}. As it stands
> now, the build fails with the following:
> {noformat}
> [ERROR] location: class org.apache.hadoop.hive.metastore.api.Role
> [ERROR]
> /Users/mithunr/workspace/dev/hive/apache/branch-1.2/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:[512,19]
> no suitable method found for
> updatePartitionStatsFast(org.apache.hadoop.hive.metastore.api.Partition,org.apache.hadoop.hive.metastore.Warehouse)
> [ERROR] method
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updatePartitionStatsFast(org.apache.hadoop.hive.metastore.api.Partition,org.apache.hadoop.hive.metastore.Warehouse,org.apache.hadoop.hive.metastore.api.EnvironmentContext)
> is not applicable
> [ERROR] (actual and formal argument lists differ in length)
> [ERROR] method
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updatePartitionStatsFast(org.apache.hadoop.hive.metastore.api.Partition,org.apache.hadoop.hive.metastore.Warehouse,boolean,org.apache.hadoop.hive.metastore.api.EnvironmentContext)
> is not applicable
> [ERROR] (actual and formal argument lists differ in length)
> [ERROR] method
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updatePartitionStatsFast(org.apache.hadoop.hive.metastore.api.Partition,org.apache.hadoop.hive.metastore.Warehouse,boolean,boolean,org.apache.hadoop.hive.metastore.api.EnvironmentContext)
> is not applicable
> [ERROR] (actual and formal argument lists differ in length)
> [ERROR] method
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updatePartitionStatsFast(org.apache.hadoop.hive.metastore.partition.spec.PartitionSpecProxy.PartitionIterator,org.apache.hadoop.hive.metastore.Warehouse,boolean,boolean,org.apache.hadoop.hive.metastore.api.EnvironmentContext)
> is not applicable
> [ERROR] (actual and formal argument lists differ in length)
> [ERROR]
> /Users/mithunr/workspace/dev/hive/apache/branch-1.2/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreWithEnvironmentContext.java:[181,45]
> incompatible types: org.apache.hadoop.hive.metastore.api.EnvironmentContext
> cannot be converted to boolean
> [ERROR]
> /Users/mithunr/workspace/dev/hive/apache/branch-1.2/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStoreWithEnvironmentContext.java:[190,45]
> incompatible types: org.apache.hadoop.hive.metastore.api.EnvironmentContext
> cannot be converted to boolean
> [ERROR]
> /Users/mithunr/workspace/dev/hive/apache/branch-1.2/itests/hive-unit/src/test/java/org/apache/hadoop/hive/thrift/TestZooKeeperTokenStore.java:[53,26]
> cannot find symbol
> [ERROR] symbol: class MiniZooKeeperCluster
> [ERROR] location: class
> org.apache.hadoop.hive.thrift.TestZooKeeperTokenStore
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with the -e
> switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions, please
> read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn -rf :hive-it-unit
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal updated HIVE-18195:
--
Attachment: HIVE-18195.patch
Attaching patch with the fix. Please review.
cc. [~sershe]
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
> Attachments: HIVE-18195.patch
>
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18195) Hive schema broken on postgres
[
https://issues.apache.org/jira/browse/HIVE-18195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-18195:
-
> Hive schema broken on postgres
> --
>
> Key: HIVE-18195
> URL: https://issues.apache.org/jira/browse/HIVE-18195
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Deepesh Khandelwal
>Priority: Blocker
>
> Hive metastore schema on postgres is broken after the commit for HIVE-17954.
> Basicaly the following file
> metastore/scripts/upgrade/postgres/hive-schema-3.0.0.postgres.sql incorrectly
> defines WM_POOL with column ALLOC_FRACTION with DOUBLE data type, it should
> be double precision.
> {noformat}
> CREATE TABLE "WM_POOL" (
> "POOL_ID" bigint NOT NULL,
> "RP_ID" bigint NOT NULL,
> "PATH" character varying(1024) NOT NULL,
> "ALLOC_FRACTION" DOUBLE,
> "QUERY_PARALLELISM" integer,
> "SCHEDULING_POLICY" character varying(1024)
> );
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18192) Introduce WriteId per table rather than using global transaction id
[ https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274739#comment-16274739 ] Alan Gates commented on HIVE-18192: --- Before we go too far down this road, is there a design doc on how this will work? This will be a big change to the transaction management system. In particular it seems like multi-statement transactions will be affected. > Introduce WriteId per table rather than using global transaction id > --- > > Key: HIVE-18192 > URL: https://issues.apache.org/jira/browse/HIVE-18192 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: anishek > Fix For: 3.0.0 > > > To support ACID replication, we will be introducing a per table write Id > which will replace the transaction id in the primary key for each row in a > ACID table. > The current primary key is determined via > > which will move to > > a persistable map of global txn id -> to table -> write id for that table has > to be maintained to now allow Snapshot isolation. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18098) Add support for Export/Import for Acid tables
[ https://issues.apache.org/jira/browse/HIVE-18098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18098: -- Description: How should this work? For regular tables export just copies the files under table root to a specified directory. This doesn't make sense for Acid tables: * Some data may belong to aborted transactons * Transaction IDs are imbedded into data/files names. You'd have export delta/ and base/ each of which may have files with the same names, e.g. bucket_0. * On import these IDs won't make sense in a different cluster or even a different table (which may have delta_x_x for example for the same x (but different data of course). * Export creates a _metadata column types, storage format, etc. Perhaps it can include info about aborted IDs (if the whole file can't be skipped). * Even importing into the same table on the same cluster may be a problem. For example delta_5_5/ existed at the time of export and was included in the export. But 2 days later it may not exist because it was compacted and cleaned. * If importing back into the same table on the same cluster, the data could be imported into a different transaction (assuming per table writeIDs) w/o having to remap the IDs in the rows themselves. * support Import Overwrite? * Support Import as a new txn with remapping of ROW_IDs? The new writeID can be stored in a delta_x_x/_meta_data and ROW__IDs can be remapped at read time (like isOriginal) and made permanent by compaction. * It doesn't seem reasonable to import acid data into non-acid table Perhaps import can work similar to Load Data: look at the file imported, if it has Acid columns, leave a note in the delta_x_x/_meta_data to indicate that these columns should be skipped a new ROW_IDs assigned at read time. h3. Case I Table has delta_7_7 and delta_8_8. Sine both may have bucket_, we could export to export_dir and rename files as bucket_ and bucket__copy_1. Load Data supports input dir with copy_N files. h3. Case II what if we have delete_delta_9_9 in the source. Now you can't just ignore ROW_IDs after import. * -Only export the latest base_N? Or more generally up to the smallest deleted ROW_ID (which may be hard to find w/o scanning all deletes. The export then would have to be done under X lock to prevent new concurrent deletes)- * Stash all deletes in some additional file file which on import gets added into the target delta/ so that Acid reader can apply them to the data in this delta/ but so that they don't clash with 'normal' deletes that exist in the table. ** here we may also have multiple delete_delta/ with identical file names. Does delete delta reader handle copy_N files? was: How should this work? For regular tables export just copies the files under table root to a specified directory. This doesn't make sense for Acid tables: * Some data may belong to aborted transactons * Transaction IDs are imbedded into data/files names. You'd have export delta/ and base/ each of which may have files with the same names, e.g. bucket_0. * On import these IDs won't make sense in a different cluster or even a different table (which may have delta_x_x for example for the same x (but different data of course). * Export creates a _metadata column types, storage format, etc. Perhaps it can include info about aborted IDs (if the whole file can't be skipped). * Even importing into the same table on the same cluster may be a problem. For example delta_5_5/ existed at the time of export and was included in the export. But 2 days later it may not exist because it was compacted and cleaned. * If importing back into the same table on the same cluster, the data could be imported into a different transaction (assuming per table writeIDs) w/o having to remap the IDs in the rows themselves. * support Import Overwrite? * Support Import as a new txn with remapping of ROW_IDs? The new writeID can be stored in a delta_x_x/_meta_data and ROW__IDs can be remapped at read time (like isOriginal) and made permanent by compaction. * It doesn't seem reasonable to import acid data into non-acid table Perhaps import can work similar to Load Data: look at the file imported, if it has Acid columns, leave a note in the delta_x_x/_meta_data to indicate that these columns should be skipped a new ROW_IDs assigned at read time. h3. Case I Table has delta_7_7 and delta_8_8. Sine both may have bucket_, we could export to export_dir and rename files as bucket_ and bucket__copy_1. Load Data supports input dir with copy_N files. h3. Case II what if we have delete_delta_9_9 in the source. Now you can't just ignore ROW_IDs after import. * Only export the latest base_N? Or more generally up to the smallest deleted ROW_ID (which may be hard to find w/o scanning all deletes. The export then would have to be done under X lock to prevent n
[jira] [Updated] (HIVE-18098) Add support for Export/Import for Acid tables
[ https://issues.apache.org/jira/browse/HIVE-18098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18098: -- Description: How should this work? For regular tables export just copies the files under table root to a specified directory. This doesn't make sense for Acid tables: * Some data may belong to aborted transactons * Transaction IDs are imbedded into data/files names. You'd have export delta/ and base/ each of which may have files with the same names, e.g. bucket_0. * On import these IDs won't make sense in a different cluster or even a different table (which may have delta_x_x for example for the same x (but different data of course). * Export creates a _metadata column types, storage format, etc. Perhaps it can include info about aborted IDs (if the whole file can't be skipped). * Even importing into the same table on the same cluster may be a problem. For example delta_5_5/ existed at the time of export and was included in the export. But 2 days later it may not exist because it was compacted and cleaned. * If importing back into the same table on the same cluster, the data could be imported into a different transaction (assuming per table writeIDs) w/o having to remap the IDs in the rows themselves. * support Import Overwrite? * Support Import as a new txn with remapping of ROW_IDs? The new writeID can be stored in a delta_x_x/_meta_data and ROW__IDs can be remapped at read time (like isOriginal) and made permanent by compaction. * It doesn't seem reasonable to import acid data into non-acid table Perhaps import can work similar to Load Data: look at the file imported, if it has Acid columns, leave a note in the delta_x_x/_meta_data to indicate that these columns should be skipped a new ROW_IDs assigned at read time. h3. Case I Table has delta_7_7 and delta_8_8. Sine both may have bucket_, we could export to export_dir and rename files as bucket_ and bucket__copy_1. Load Data supports input dir with copy_N files. h3. Case II what if we have delete_delta_9_9 in the source. Now you can't just ignore ROW_IDs after import. * Only export the latest base_N? Or more generally up to the smallest deleted ROW_ID (which may be hard to find w/o scanning all deletes. The export then would have to be done under X lock to prevent new concurrent deletes) * Stash all deletes in some additional file file which on import gets added into the target delta/ so that Acid reader can apply them to the data in this delta/ but so that they don't clash with 'normal' deletes that exist in the table. ** here we may also have multiple delete_delta/ with identical file names. Does delete delta reader handle copy_N files? was: How should this work? For regular tables export just copies the files under table root to a specified directory. This doesn't make sense for Acid tables: * Some data may belong to aborted transactons * Transaction IDs are imbedded into data/files names. You'd have export delta/ and base/ each of which may have files with the same names, e.g. bucket_0. * On import these IDs won't make sense in a different cluster or even a different table (which may have delta_x_x for example for the same x (but different data of course). * Export creates a _metadata column types, storage format, etc. Perhaps it can include info about aborted IDs (if the whole file can't be skipped). * Even importing into the same table on the same cluster may be a problem. For example delta_5_5/ existed at the time of export and was included in the export. But 2 days later it may not exist because it was compacted and cleaned. * If importing back into the same table on the same cluster, the data could be imported into a different transaction (assuming per table writeIDs) w/o having to remap the IDs in the rows themselves. * support Import Overwrite? * Support Import as a new txn with remapping of ROW_IDs? The new writeID can be stored in a delta_x_x/_meta_data and ROW__IDs can be remapped at read time (like isOriginal) and made permanent by compaction. * It doesn't seem reasonable to import acid data into non-acid table Perhaps import can work similar to Load Data: look at the file imported, if it has Acid columns, leave a note in the delta_x_x/_meta_data to indicate that these columns should be skipped a new ROW_IDs assigned at read time. h3. Case I Table has delta_7_7 and delta_8_8. Sine both may have bucket_, we could export to export_dir and rename files as bucket_ and bucket__copy_1. Load Data supports input dir with copy_N files. h3. Case II what if we have delete_delta_9_9 in the source. > Add support for Export/Import for Acid tables > - > > Key: HIVE-18098 > URL: https://issues.apache.org/jira/browse/HIVE-18098 > Project: Hive > Issue Typ
[jira] [Commented] (HIVE-16890) org.apache.hadoop.hive.serde2.io.HiveVarcharWritable - Adds Superfluous Wrapper
[
https://issues.apache.org/jira/browse/HIVE-16890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274701#comment-16274701
]
Naveen Gangam commented on HIVE-16890:
--
The test failures seem unrelated to the patch. The prior build also has the
same failures and more. Will commit the fix shortly.
> org.apache.hadoop.hive.serde2.io.HiveVarcharWritable - Adds Superfluous
> Wrapper
> ---
>
> Key: HIVE-16890
> URL: https://issues.apache.org/jira/browse/HIVE-16890
> Project: Hive
> Issue Type: Improvement
> Components: Serializers/Deserializers
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Trivial
> Attachments: HIVE-16890.1.patch, HIVE-16890.1.patch,
> HIVE-16890.1.patch
>
>
> Class {{org.apache.hadoop.hive.serde2.io.HiveVarcharWritable}} creates a
> superfluous wrapper and then immediately unwraps it. Don't bother wrapping
> in this scenario.
> {code}
> public void set(HiveVarchar val, int len) {
> set(val.getValue(), len);
> }
> public void set(String val, int maxLength) {
> value.set(HiveBaseChar.enforceMaxLength(val, maxLength));
> }
> public HiveVarchar getHiveVarchar() {
> return new HiveVarchar(value.toString(), -1);
> }
> // Here calls getHiveVarchar() which creates a new HiveVarchar object with
> a string in it
> // The object is passed to set(HiveVarchar val, int len)
> // The string is pulled out
> public void enforceMaxLength(int maxLength) {
> // Might be possible to truncate the existing Text value, for now just do
> something simple.
> if (value.getLength()>maxLength && getCharacterLength()>maxLength)
> set(getHiveVarchar(), maxLength);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18098) Add support for Export/Import for Acid tables
[ https://issues.apache.org/jira/browse/HIVE-18098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18098: -- Description: How should this work? For regular tables export just copies the files under table root to a specified directory. This doesn't make sense for Acid tables: * Some data may belong to aborted transactons * Transaction IDs are imbedded into data/files names. You'd have export delta/ and base/ each of which may have files with the same names, e.g. bucket_0. * On import these IDs won't make sense in a different cluster or even a different table (which may have delta_x_x for example for the same x (but different data of course). * Export creates a _metadata column types, storage format, etc. Perhaps it can include info about aborted IDs (if the whole file can't be skipped). * Even importing into the same table on the same cluster may be a problem. For example delta_5_5/ existed at the time of export and was included in the export. But 2 days later it may not exist because it was compacted and cleaned. * If importing back into the same table on the same cluster, the data could be imported into a different transaction (assuming per table writeIDs) w/o having to remap the IDs in the rows themselves. * support Import Overwrite? * Support Import as a new txn with remapping of ROW_IDs? The new writeID can be stored in a delta_x_x/_meta_data and ROW__IDs can be remapped at read time (like isOriginal) and made permanent by compaction. * It doesn't seem reasonable to import acid data into non-acid table Perhaps import can work similar to Load Data: look at the file imported, if it has Acid columns, leave a note in the delta_x_x/_meta_data to indicate that these columns should be skipped a new ROW_IDs assigned at read time. h3. Case I Table has delta_7_7 and delta_8_8. Sine both may have bucket_, we could export to export_dir and rename files as bucket_ and bucket__copy_1. Load Data supports input dir with copy_N files. h3. Case II what if we have delete_delta_9_9 in the source. was: How should this work? For regular tables export just copies the files under table root to a specified directory. This doesn't make sense for Acid tables: * Some data may belong to aborted transactons * Transaction IDs are imbedded into data/files names. You'd have export delta/ and base/ each of which may have files with the same names, e.g. bucket_0. * On import these IDs won't make sense in a different cluster or even a different table (which may have delta_x_x for example for the same x (but different data of course). * Export creates a _metadata column types, storage format, etc. Perhaps it can include info about aborted IDs (if the whole file can't be skipped). * Even importing into the same table on the same cluster may be a problem. For example delta_5_5/ existed at the time of export and was included in the export. But 2 days later it may not exist because it was compacted and cleaned. * If importing back into the same table on the same cluster, the data could be imported into a different transaction (assuming per table writeIDs) w/o having to remap the IDs in the rows themselves. * support Import Overwrite? * Support Import as a new txn with remapping of ROW_IDs? The new writeID can be stored in a delta_x_x/_meta_data and ROW__IDs can be remapped at read time (like isOriginal) and made permanent by compaction. * It doesn't seem reasonable to import acid data into non-acid table Perhaps import can work similar to Load Data: look at the file imported, if it has Acid columns, leave a note in the delta_x_x/_meta_data to indicate that these columns should be skipped a new ROW_IDs assigned at read time. > Add support for Export/Import for Acid tables > - > > Key: HIVE-18098 > URL: https://issues.apache.org/jira/browse/HIVE-18098 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Eugene Koifman >Assignee: Eugene Koifman > > How should this work? > For regular tables export just copies the files under table root to a > specified directory. > This doesn't make sense for Acid tables: > * Some data may belong to aborted transactons > * Transaction IDs are imbedded into data/files names. You'd have export > delta/ and base/ each of which may have files with the same names, e.g. > bucket_0. > * On import these IDs won't make sense in a different cluster or even a > different table (which may have delta_x_x for example for the same x (but > different data of course). > * Export creates a _metadata column types, storage format, etc. Perhaps it > can include info about aborted IDs (if the whole file can't be skipped). > * Even importing into the same table on the same cluster may be a probl
[jira] [Updated] (HIVE-18127) Do not strip '--' comments from shell commands issued from CliDriver
[ https://issues.apache.org/jira/browse/HIVE-18127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-18127: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. > Do not strip '--' comments from shell commands issued from CliDriver > > > Key: HIVE-18127 > URL: https://issues.apache.org/jira/browse/HIVE-18127 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Fix For: 3.0.0 > > Attachments: HIVE-18127.1.patch, HIVE-18127.2.patch > > > CLiDriver has the ability to run shell commands by prefixing them with '!". > This behavior is not widely used (there are only 3 examples in .q files). > Since HIVE-16935 started stripping comments starting with '\-\-', a shell > command containing '--' will not work correctly. > Fix this by using the unstripped command for shell commands. > Note that it would be a security hole for HS2 to allow execution of arbitrary > shell commands from a client command. > Add tests to nail down correct behavior with '--' comments: > * CliDriver should not strip strings starting with '--' in a shell command > (FIXED in this change). > * HiveCli should strip '--' comments. > * A Jdbc program should allow commands starting with "!" but these will fail > in the sql parser. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18193) Migrate existing ACID tables to use write id per table rather than global transaction id
[ https://issues.apache.org/jira/browse/HIVE-18193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274638#comment-16274638 ] Andrew Sherman commented on HIVE-18193: --- Hi [~anishek] can you explain a little more about this. How is the write id generated? Will this require on-disk data to be rewritten? Sorry if I missed previous discussions but is there a design for snapshot isolation. Thanks > Migrate existing ACID tables to use write id per table rather than global > transaction id > > > Key: HIVE-18193 > URL: https://issues.apache.org/jira/browse/HIVE-18193 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: anishek > Fix For: 3.0.0 > > > dependent upon HIVE-18192 > For existing ACID Tables we need to update the table level write id > metatables/sequences so any new operations on these tables works seamlessly > without any conflicting data in existing base/delta files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18177) Error creating temporary staging folder on HDFS when creating Hive table
[
https://issues.apache.org/jira/browse/HIVE-18177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274606#comment-16274606
]
Aegeaner commented on HIVE-18177:
-
Configuration problem. The aux lib dir under $HIVE_HOME lacks packages.
> Error creating temporary staging folder on HDFS when creating Hive table
>
>
> Key: HIVE-18177
> URL: https://issues.apache.org/jira/browse/HIVE-18177
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Affects Versions: 2.3.1
>Reporter: Aegeaner
>
> When creating a table in hive using statement:
> {code:java}
> create database if not exists ${DB};
> use ${DB};
> drop table if exists date_dim;
> create table date_dim
> stored as ${FILE}
> as select * from ${SOURCE}.date_dim;
> {code}
> The statement execution failed as:
> {code:java}
> FAILED: SemanticException 0:0 Error creating temporary folder on:
> hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db. Error
> encountered near token 'TOK_TMP_FILE'
> FAILED: SemanticException 0:0 Error creating temporary folder on:
> hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db. Error
> encountered near token 'TOK_TMP_FILE'
> {code}
> We got this exception stack：
> {code:java}
> 2017-11-29T17:32:47,646 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: Completed phase 1 of Semantic Analysis
> 2017-11-29T17:32:47,646 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: Get metadata for source tables
> 2017-11-29T17:32:47,646 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> metastore.HiveMetaStore: 0: get_table : db=tpcds_text_2 tbl=date_dim
> 2017-11-29T17:32:47,647 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_table :
> db=tpcds_text_2 tbl=date_dim
> 2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: Get metadata for subqueries
> 2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: Get metadata for destination tables
> 2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> metastore.HiveMetaStore: 0: get_database: tpcds_bin_partitioned_orc_2
> 2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_database:
> tpcds_bin_partitioned_orc_2
> 2017-11-29T17:32:48,308 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> common.FileUtils: Creating directory if it doesn't exist:
> hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db/.hive-staging_hive_2017-11-29_17-32-47_541_232506518783479-1
> 2017-11-29T17:32:48,330 ERROR [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: org.apache.hadoop.hive.ql.parse.SemanticException: 0:0
> Error creating temporary folder on:
> hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db. Error
> encountered near token 'TOK_TMP_FILE'
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2211)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1934)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:11080)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11133)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336)
> at
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:474)
> at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:490)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
> at sun.reflect.NativeMethodAccessor
[jira] [Resolved] (HIVE-18177) Error creating temporary staging folder on HDFS when creating Hive table
[
https://issues.apache.org/jira/browse/HIVE-18177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aegeaner resolved HIVE-18177.
-
Resolution: Won't Fix
Release Note: Configuration problem. The aux lib dir under $HIVE_HOME lacks
packages.
> Error creating temporary staging folder on HDFS when creating Hive table
>
>
> Key: HIVE-18177
> URL: https://issues.apache.org/jira/browse/HIVE-18177
> Project: Hive
> Issue Type: Bug
> Components: Parser
>Affects Versions: 2.3.1
>Reporter: Aegeaner
>
> When creating a table in hive using statement:
> {code:java}
> create database if not exists ${DB};
> use ${DB};
> drop table if exists date_dim;
> create table date_dim
> stored as ${FILE}
> as select * from ${SOURCE}.date_dim;
> {code}
> The statement execution failed as:
> {code:java}
> FAILED: SemanticException 0:0 Error creating temporary folder on:
> hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db. Error
> encountered near token 'TOK_TMP_FILE'
> FAILED: SemanticException 0:0 Error creating temporary folder on:
> hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db. Error
> encountered near token 'TOK_TMP_FILE'
> {code}
> We got this exception stack：
> {code:java}
> 2017-11-29T17:32:47,646 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: Completed phase 1 of Semantic Analysis
> 2017-11-29T17:32:47,646 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: Get metadata for source tables
> 2017-11-29T17:32:47,646 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> metastore.HiveMetaStore: 0: get_table : db=tpcds_text_2 tbl=date_dim
> 2017-11-29T17:32:47,647 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_table :
> db=tpcds_text_2 tbl=date_dim
> 2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: Get metadata for subqueries
> 2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: Get metadata for destination tables
> 2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> metastore.HiveMetaStore: 0: get_database: tpcds_bin_partitioned_orc_2
> 2017-11-29T17:32:47,748 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> HiveMetaStore.audit: ugi=hadoop ip=unknown-ip-addr cmd=get_database:
> tpcds_bin_partitioned_orc_2
> 2017-11-29T17:32:48,308 INFO [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> common.FileUtils: Creating directory if it doesn't exist:
> hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db/.hive-staging_hive_2017-11-29_17-32-47_541_232506518783479-1
> 2017-11-29T17:32:48,330 ERROR [4d9462cf-43b0-4fea-b0c2-c1a9969d9763 main]
> parse.CalcitePlanner: org.apache.hadoop.hive.ql.parse.SemanticException: 0:0
> Error creating temporary folder on:
> hdfs://ns-offline/user/hive2/warehouse/tpcds_bin_partitioned_orc_2.db. Error
> encountered near token 'TOK_TMP_FILE'
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:2211)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1934)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:11080)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11133)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:286)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:512)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1317)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1457)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336)
> at
> org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:474)
> at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:490)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
> at sun.reflect.NativeMethodAccessorImpl.invo
[jira] [Commented] (HIVE-18088) Add WM event traces at query level for debugging
[
https://issues.apache.org/jira/browse/HIVE-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274280#comment-16274280
]
Hive QA commented on HIVE-18088:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900118/HIVE-18088.3.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 11493 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join]
(batchId=44)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=227)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers1
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitions
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsMultiInsert
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
(batchId=234)
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerTotalTasks
(batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8077/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8077/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8077/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12900118 - PreCommit-HIVE-Build
> Add WM event traces at query level for debugging
>
>
> Key: HIVE-18088
> URL: https://issues.apache.org/jira/browse/HIVE-18088
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18088.1.patch, HIVE-18088.2.patch,
> HIVE-18088.3.patch, HIVE-18088.WIP.patch
>
>
> For debugging and testing purpose, expose workload manager events via /jmx
> endpoint and print summary at the scope of query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18088) Add WM event traces at query level for debugging
[
https://issues.apache.org/jira/browse/HIVE-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274230#comment-16274230
]
Hive QA commented on HIVE-18088:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
9s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 75 new + 570 unchanged - 34
fixed = 645 total (was 604) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / f68ebdc |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8077/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8077/yetus/patch-asflicense-problems.txt
|
| modules | C: common ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8077/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add WM event traces at query level for debugging
>
>
> Key: HIVE-18088
> URL: https://issues.apache.org/jira/browse/HIVE-18088
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18088.1.patch, HIVE-18088.2.patch,
> HIVE-18088.3.patch, HIVE-18088.WIP.patch
>
>
> For debugging and testing purpose, expose workload manager events via /jmx
> endpoint and print summary at the scope of query.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18189) Order by position does not work when cbo is disabled
[
https://issues.apache.org/jira/browse/HIVE-18189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274205#comment-16274205
]
Hive QA commented on HIVE-18189:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900113/HIVE-18189.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 13 failed/errored test(s), 11493 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_by_pos]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join]
(batchId=44)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[orderby_position_unsupported]
(batchId=93)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_position]
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query47]
(batchId=249)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query57]
(batchId=249)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query47]
(batchId=247)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query57]
(batchId=247)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=227)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8076/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8076/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8076/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 13 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12900113 - PreCommit-HIVE-Build
> Order by position does not work when cbo is disabled
>
>
> Key: HIVE-18189
> URL: https://issues.apache.org/jira/browse/HIVE-18189
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-18189.1.patch
>
>
> Investigating a failed query:
> {code}
> set hive.cbo.enable=false;
> set hive.orderby.position.alias=true;
> select distinct age from student order by 1 desc limit 20;
> {code}
> The query does not sort the output correctly when cbo is
> disabled/inactivated. I found two issues:
> 1. "order by position" query is broken by HIVE-16774
> 2. In particular, select distinct query never work for "order by position"
> query
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17361) Support LOAD DATA for transactional tables
[ https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274191#comment-16274191 ] Lefty Leverenz commented on HIVE-17361: --- Doc note: Support of LOAD DATA for transactional tables should be documented in the Transactions doc. * [Hive Transactions -- Limitations | https://cwiki.apache.org/confluence/display/Hive/Hive+Transactions#HiveTransactions-Limitations] Added a TODOC3.0 label. (The changed description of *hive.txn.operational.properties* does not need documentation because the parameter is for internal use.) > Support LOAD DATA for transactional tables > -- > > Key: HIVE-17361 > URL: https://issues.apache.org/jira/browse/HIVE-17361 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Wei Zheng >Assignee: Eugene Koifman >Priority: Critical > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-17361.07.patch, HIVE-17361.08.patch, > HIVE-17361.09.patch, HIVE-17361.1.patch, HIVE-17361.10.patch, > HIVE-17361.11.patch, HIVE-17361.12.patch, HIVE-17361.14.patch, > HIVE-17361.16.patch, HIVE-17361.17.patch, HIVE-17361.19.patch, > HIVE-17361.2.patch, HIVE-17361.20.patch, HIVE-17361.21.patch, > HIVE-17361.23.patch, HIVE-17361.24.patch, HIVE-17361.25.patch, > HIVE-17361.3.patch, HIVE-17361.4.patch > > > LOAD DATA was not supported since ACID was introduced. Need to fill this gap > between ACID table and regular hive table. > Current Documentation is under [DML > Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] > and [Loading files into > tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: > \\ > * Load Data performs very limited validations of the data, in particular it > uses the input file name which may not be in 0_0 which can break some > read logic. (Certainly will for Acid). > * It does not check the schema of the file. This may be a non issue for Acid > which requires ORC which is self describing so Schema Evolution may handle > this seamlessly. (Assuming Schema is not too different). > * It does check that _InputFormat_S are compatible. > * Bucketed (and thus sorted) tables don't support Load Data (but only if > hive.strict.checks.bucketing=true (default)). Will keep this restriction for > Acid. > * Load Data supports OVERWRITE clause > * What happens to file permissions/ownership: rename vs copy differences > \\ > The implementation will follow the same idea as in HIVE-14988 and use a > base_N/ dir for OVERWRITE clause. > \\ > How is minor compaction going to handle delta/base with original files? > Since delta_8_8/_meta_data is created before files are moved, delta_8_8 > becomes visible before it's populated. Is that an issue? > It's not since txn 8 is not committed. > h3. Implementation Notes/Limitations (patch 25) > * bucketed/sorted tables are not supported > * input files names must be of the form 0_0/0_0_copy_1 - enforced. > (HIVE-18125) > * Load Data creates a delta_x_x/ that contains new files > * Load Data w/Overwrite creates a base_x/ that contains new files > * A '_metadata_acid' file is placed in the target directory to indicate it > requires special handling on read > * The input files must be 'plain' ORC files, i.e. not contain acid metadata > columns as would be the case if these files were copied from another Acid > table. In the latter case, the ROW_IDs embedded in the data may not make > sense in the target table (if it's in a different cluster, for example). > Such files may also have a mix of committed and aborted data. > ** this could be relaxed later by adding info to the _metadata_acid file to > ignore existing ROW_IDs on read. > * ROW_IDs are attached dynamically at read time and made permanent by > compaction. This is done the same way has handling of files that were > written to a table before it was converted to Acid. > * Vectorization is supported -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17361) Support LOAD DATA for transactional tables
[ https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-17361: -- Labels: TODOC3.0 (was: ) > Support LOAD DATA for transactional tables > -- > > Key: HIVE-17361 > URL: https://issues.apache.org/jira/browse/HIVE-17361 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Wei Zheng >Assignee: Eugene Koifman >Priority: Critical > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-17361.07.patch, HIVE-17361.08.patch, > HIVE-17361.09.patch, HIVE-17361.1.patch, HIVE-17361.10.patch, > HIVE-17361.11.patch, HIVE-17361.12.patch, HIVE-17361.14.patch, > HIVE-17361.16.patch, HIVE-17361.17.patch, HIVE-17361.19.patch, > HIVE-17361.2.patch, HIVE-17361.20.patch, HIVE-17361.21.patch, > HIVE-17361.23.patch, HIVE-17361.24.patch, HIVE-17361.25.patch, > HIVE-17361.3.patch, HIVE-17361.4.patch > > > LOAD DATA was not supported since ACID was introduced. Need to fill this gap > between ACID table and regular hive table. > Current Documentation is under [DML > Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] > and [Loading files into > tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: > \\ > * Load Data performs very limited validations of the data, in particular it > uses the input file name which may not be in 0_0 which can break some > read logic. (Certainly will for Acid). > * It does not check the schema of the file. This may be a non issue for Acid > which requires ORC which is self describing so Schema Evolution may handle > this seamlessly. (Assuming Schema is not too different). > * It does check that _InputFormat_S are compatible. > * Bucketed (and thus sorted) tables don't support Load Data (but only if > hive.strict.checks.bucketing=true (default)). Will keep this restriction for > Acid. > * Load Data supports OVERWRITE clause > * What happens to file permissions/ownership: rename vs copy differences > \\ > The implementation will follow the same idea as in HIVE-14988 and use a > base_N/ dir for OVERWRITE clause. > \\ > How is minor compaction going to handle delta/base with original files? > Since delta_8_8/_meta_data is created before files are moved, delta_8_8 > becomes visible before it's populated. Is that an issue? > It's not since txn 8 is not committed. > h3. Implementation Notes/Limitations (patch 25) > * bucketed/sorted tables are not supported > * input files names must be of the form 0_0/0_0_copy_1 - enforced. > (HIVE-18125) > * Load Data creates a delta_x_x/ that contains new files > * Load Data w/Overwrite creates a base_x/ that contains new files > * A '_metadata_acid' file is placed in the target directory to indicate it > requires special handling on read > * The input files must be 'plain' ORC files, i.e. not contain acid metadata > columns as would be the case if these files were copied from another Acid > table. In the latter case, the ROW_IDs embedded in the data may not make > sense in the target table (if it's in a different cluster, for example). > Such files may also have a mix of committed and aborted data. > ** this could be relaxed later by adding info to the _metadata_acid file to > ignore existing ROW_IDs on read. > * ROW_IDs are attached dynamically at read time and made permanent by > compaction. This is done the same way has handling of files that were > written to a table before it was converted to Acid. > * Vectorization is supported -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17988) Replace patch utility usage with git apply in ptest
[
https://issues.apache.org/jira/browse/HIVE-17988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274161#comment-16274161
]
Daniel Voros commented on HIVE-17988:
-
[~spena] thank you for taking a look! That file is part of the Yetus
integration introduced in HIVE-15051 and if I'm not mistaken it doesn't use the
patch utility to apply the patches, so I think there's no need to modify around
that.
> Replace patch utility usage with git apply in ptest
> ---
>
> Key: HIVE-17988
> URL: https://issues.apache.org/jira/browse/HIVE-17988
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Zoltan Haindrich
>Assignee: Daniel Voros
> Attachments: HIVE-17988.1.patch, HIVE-17988.2.patch
>
>
> It would be great to replace the standard diff util because {{git}} can do a
> 3-way merge - which in most cases successfull.
> This could reduce the ptest results which are erroring out because of build
> failure.
> {code}
> error: patch failed:
> ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:7003
> Falling back to three-way merge...
> Applied patch to
> 'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java' cleanly.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18189) Order by position does not work when cbo is disabled
[
https://issues.apache.org/jira/browse/HIVE-18189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274158#comment-16274158
]
Hive QA commented on HIVE-18189:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 4 new + 934 unchanged - 0
fixed = 938 total (was 934) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / f68ebdc |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8076/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8076/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8076/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Order by position does not work when cbo is disabled
>
>
> Key: HIVE-18189
> URL: https://issues.apache.org/jira/browse/HIVE-18189
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: HIVE-18189.1.patch
>
>
> Investigating a failed query:
> {code}
> set hive.cbo.enable=false;
> set hive.orderby.position.alias=true;
> select distinct age from student order by 1 desc limit 20;
> {code}
> The query does not sort the output correctly when cbo is
> disabled/inactivated. I found two issues:
> 1. "order by position" query is broken by HIVE-16774
> 2. In particular, select distinct query never work for "order by position"
> query
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18036) Stats: Remove usage of clone() methods
[
https://issues.apache.org/jira/browse/HIVE-18036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274134#comment-16274134
]
Hive QA commented on HIVE-18036:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12900105/HIVE-18036.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 11493 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join]
(batchId=44)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=227)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8075/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8075/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8075/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12900105 - PreCommit-HIVE-Build
> Stats: Remove usage of clone() methods
> --
>
> Key: HIVE-18036
> URL: https://issues.apache.org/jira/browse/HIVE-18036
> Project: Hive
> Issue Type: Sub-task
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Bertalan Kondrat
> Attachments: HIVE-18036.2.patch, HIVE-18036.3.patch, HIVE-18036.patch
>
>
> {{Statistics}} and {{ColStats}} implements cloneable; however they never
> throw clonenotsupported; and this causes try / catch blocks which are just
> noise
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18036) Stats: Remove usage of clone() methods
[
https://issues.apache.org/jira/browse/HIVE-18036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16274095#comment-16274095
]
Hive QA commented on HIVE-18036:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
34s{color} | {color:red} ql: The patch generated 10 new + 243 unchanged - 7
fixed = 253 total (was 250) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / f68ebdc |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8075/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8075/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8075/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Stats: Remove usage of clone() methods
> --
>
> Key: HIVE-18036
> URL: https://issues.apache.org/jira/browse/HIVE-18036
> Project: Hive
> Issue Type: Sub-task
> Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Bertalan Kondrat
> Attachments: HIVE-18036.2.patch, HIVE-18036.3.patch, HIVE-18036.patch
>
>
> {{Statistics}} and {{ColStats}} implements cloneable; however they never
> throw clonenotsupported; and this causes try / catch blocks which are just
> noise
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
