[jira] [Commented] (HIVE-18743) CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
[
https://issues.apache.org/jira/browse/HIVE-18743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384540#comment-16384540
]
Hive QA commented on HIVE-18743:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912729/HIVE-18743.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 13430 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Updated] (HIVE-14737) Problem accessing /logs in a Kerberized Hive Server 2 Web UI
[ https://issues.apache.org/jira/browse/HIVE-14737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-14737: -- Attachment: HIVE-14737.patch > Problem accessing /logs in a Kerberized Hive Server 2 Web UI > > > Key: HIVE-14737 > URL: https://issues.apache.org/jira/browse/HIVE-14737 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Matyas Orhidi >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-14737.patch > > > The /logs menu fails with error [1] when the cluster is Kerberized. Other > menu items are working properly. > [1] HTTP ERROR: 401 > Problem accessing /logs/. Reason: > Unauthenticated users are not authorized to access this page. > Powered by Jetty:// -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-14737) Problem accessing /logs in a Kerberized Hive Server 2 Web UI
[ https://issues.apache.org/jira/browse/HIVE-14737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-14737: -- Attachment: HIVE-14737.patch > Problem accessing /logs in a Kerberized Hive Server 2 Web UI > > > Key: HIVE-14737 > URL: https://issues.apache.org/jira/browse/HIVE-14737 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Matyas Orhidi >Assignee: Rajkumar Singh >Priority: Major > > The /logs menu fails with error [1] when the cluster is Kerberized. Other > menu items are working properly. > [1] HTTP ERROR: 401 > Problem accessing /logs/. Reason: > Unauthenticated users are not authorized to access this page. > Powered by Jetty:// -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-14737) Problem accessing /logs in a Kerberized Hive Server 2 Web UI
[ https://issues.apache.org/jira/browse/HIVE-14737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-14737: -- Attachment: (was: HIVE-14737.patch) > Problem accessing /logs in a Kerberized Hive Server 2 Web UI > > > Key: HIVE-14737 > URL: https://issues.apache.org/jira/browse/HIVE-14737 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Matyas Orhidi >Assignee: Rajkumar Singh >Priority: Major > > The /logs menu fails with error [1] when the cluster is Kerberized. Other > menu items are working properly. > [1] HTTP ERROR: 401 > Problem accessing /logs/. Reason: > Unauthenticated users are not authorized to access this page. > Powered by Jetty:// -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-14737) Problem accessing /logs in a Kerberized Hive Server 2 Web UI
[ https://issues.apache.org/jira/browse/HIVE-14737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh reassigned HIVE-14737: - Assignee: Rajkumar Singh (was: Johndee Burks) > Problem accessing /logs in a Kerberized Hive Server 2 Web UI > > > Key: HIVE-14737 > URL: https://issues.apache.org/jira/browse/HIVE-14737 > Project: Hive > Issue Type: Bug >Affects Versions: 1.1.0 >Reporter: Matyas Orhidi >Assignee: Rajkumar Singh >Priority: Major > > The /logs menu fails with error [1] when the cluster is Kerberized. Other > menu items are working properly. > [1] HTTP ERROR: 401 > Problem accessing /logs/. Reason: > Unauthenticated users are not authorized to access this page. > Powered by Jetty:// -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18743) CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
[
https://issues.apache.org/jira/browse/HIVE-18743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384521#comment-16384521
]
Hive QA commented on HIVE-18743:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} standalone-metastore: The patch generated 0 new +
484 unchanged - 3 fixed = 484 total (was 487) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9452/dev-support/hive-personality.sh
|
| git revision | master / 1a3090f |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9452/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9452/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround
> is buggy.
> ---
>
> Key: HIVE-18743
> URL: https://issues.apache.org/jira/browse/HIVE-18743
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 1.2.0, 1.1.0, 2.0.2, 3.0.0
>Reporter: Alexander Behm
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-18743.04.patch
>
>
> When hive.stats.autogather=true then the Metastore lists all files under the
> table directory to populate basic stats like file counts and sizes. This file
> listing operation can be very expensive particularly on filesystems like S3.
> One way to address this issue is to reconfigure hive.stats.autogather=false.
> *Here's the bug*
> It is my understanding that the DO_NOT_UPDATE_STATS table property is
> intended to selectively prevent this stats collection. Unfortunately, this
> table property is checked *after* the expensive file listing operation, so
> the DO_NOT_UPDATE_STATS does not seem to work as intended. See:
> https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java#L633
> Relevant code snippet:
> {code}
> public static boolean updateTableStatsFast(Database db, Table tbl,
> Warehouse wh,
> boolean madeDir, boolean
> forceRecompute, EnvironmentContext environmentContext) throws MetaException {
> if (tbl.getPartitionKeysSize()
[jira] [Commented] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384502#comment-16384502
]
Hive QA commented on HIVE-18848:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912809/HIVE-18848.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 85 failed/errored test(s), 13451 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Commented] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384489#comment-16384489
]
Hive QA commented on HIVE-18848:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 40 new + 37 unchanged - 1
fixed = 77 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
21s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9451/dev-support/hive-personality.sh
|
| git revision | master / 1a3090f |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9451/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9451/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9451/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve readability of filter conditions in explain plan when CBO is run
>
>
> Key: HIVE-18848
> URL: https://issues.apache.org/jira/browse/HIVE-18848
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18848.01.patch, HIVE-18848.02.patch
>
>
> CBO might return comparison operands in any non-deterministic order. Try to
> show {{ }} when possible, i.e., {{c < 10}} rather
> than {{10 > c}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384487#comment-16384487
]
Hive QA commented on HIVE-18848:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912809/HIVE-18848.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 88 failed/errored test(s), 13451 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Commented] (HIVE-18812) Hive Runtime Error while processing row
[ https://issues.apache.org/jira/browse/HIVE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384478#comment-16384478 ] Xin shiyou commented on HIVE-18812: --- Thanks for your attention. Two figures was added in my issue. > Hive Runtime Error while processing row > --- > > Key: HIVE-18812 > URL: https://issues.apache.org/jira/browse/HIVE-18812 > Project: Hive > Issue Type: Bug > Components: CLI >Affects Versions: 1.1.0 > Environment: 1、JAVA: 1.8.0_151 > 2、Hive :1.1.0-cdh5.12.1 >Reporter: Xin shiyou >Priority: Critical > Attachments: 屏幕快照 2018-03-03 下午12.43.13.png, 屏幕快照 2018-03-03 > 下午12.43.23.png > > > execute sql: > select count(1) from userevent; > error messages : > Error: java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime > Error while processing row > at > org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52) > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170) > ... 8 more > Caused by: java.lang.ArrayIndexOutOfBoundsException: 72 > at > org.apache.hadoop.hive.ql.exec.vector.expressions.ConstantVectorExpression.evaluateLong(ConstantVectorExpression.java:99) > at > org.apache.hadoop.hive.ql.exec.vector.expressions.ConstantVectorExpression.evaluate(ConstantVectorExpression.java:147) > at > org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.aggregateInput(VectorUDAFCount.java:170) > at > org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeGlobalAggregate.processBatch(VectorGroupByOperator.java:193) > at > org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:866) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815) > at > org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:138) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815) > at > org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:98) > at > org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157) > at > org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45) > ... 9 more > > but when I update settings : set hive.vectorized.execution.enabled=false; > then the result is OK. > I don't know why or lost some jars in my environment . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18812) Hive Runtime Error while processing row
[ https://issues.apache.org/jira/browse/HIVE-18812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xin shiyou updated HIVE-18812: -- Attachment: 屏幕快照 2018-03-03 下午12.43.23.png 屏幕快照 2018-03-03 下午12.43.13.png > Hive Runtime Error while processing row > --- > > Key: HIVE-18812 > URL: https://issues.apache.org/jira/browse/HIVE-18812 > Project: Hive > Issue Type: Bug > Components: CLI >Affects Versions: 1.1.0 > Environment: 1、JAVA: 1.8.0_151 > 2、Hive :1.1.0-cdh5.12.1 >Reporter: Xin shiyou >Priority: Critical > Attachments: 屏幕快照 2018-03-03 下午12.43.13.png, 屏幕快照 2018-03-03 > 下午12.43.23.png > > > execute sql: > select count(1) from userevent; > error messages : > Error: java.lang.RuntimeException: > org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while > processing row > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:459) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) > Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime > Error while processing row > at > org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:52) > at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170) > ... 8 more > Caused by: java.lang.ArrayIndexOutOfBoundsException: 72 > at > org.apache.hadoop.hive.ql.exec.vector.expressions.ConstantVectorExpression.evaluateLong(ConstantVectorExpression.java:99) > at > org.apache.hadoop.hive.ql.exec.vector.expressions.ConstantVectorExpression.evaluate(ConstantVectorExpression.java:147) > at > org.apache.hadoop.hive.ql.exec.vector.expressions.aggregates.VectorUDAFCount.aggregateInput(VectorUDAFCount.java:170) > at > org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator$ProcessingModeGlobalAggregate.processBatch(VectorGroupByOperator.java:193) > at > org.apache.hadoop.hive.ql.exec.vector.VectorGroupByOperator.processOp(VectorGroupByOperator.java:866) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815) > at > org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:138) > at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815) > at > org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:98) > at > org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157) > at > org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45) > ... 9 more > > but when I update settings : set hive.vectorized.execution.enabled=false; > then the result is OK. > I don't know why or lost some jars in my environment . -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18811) Fix desc table, column comments are not displayed
[
https://issues.apache.org/jira/browse/HIVE-18811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384476#comment-16384476
]
tartarus commented on HIVE-18811:
-
[~vgarg] Thanks for your reminder, I will confirm it again.
> Fix desc table, column comments are not displayed
> -
>
> Key: HIVE-18811
> URL: https://issues.apache.org/jira/browse/HIVE-18811
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 1.2.1, 2.3.2
> Environment: CentOS 6.5
> Hive-1.2.1
> Hive-3.0.0
>Reporter: tartarus
>Assignee: tartarus
>Priority: Major
> Labels: patch
> Fix For: 3.0.0
>
> Attachments: HIVE_18811.patch
>
>
> when column comment contain \t
> eg: CREATE TABLE `zhangmang_test`(`name` string COMMENT
> 'name{color:#ff}\t{color}zm');
> then execute : {color:#ff}desc zhangmang_test {color}
> {color:#ff}{color:#33}return :{color} name string
> name{color}
> because \t is the separator, so we should translate it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384462#comment-16384462
]
Hive QA commented on HIVE-18848:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 40 new + 37 unchanged - 1
fixed = 77 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
19s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9450/dev-support/hive-personality.sh
|
| git revision | master / 1a3090f |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9450/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9450/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9450/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve readability of filter conditions in explain plan when CBO is run
>
>
> Key: HIVE-18848
> URL: https://issues.apache.org/jira/browse/HIVE-18848
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18848.01.patch, HIVE-18848.02.patch
>
>
> CBO might return comparison operands in any non-deterministic order. Try to
> show {{ }} when possible, i.e., {{c < 10}} rather
> than {{10 > c}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384455#comment-16384455
]
Hive QA commented on HIVE-18848:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912809/HIVE-18848.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 85 failed/errored test(s), 13054 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Updated] (HIVE-18738) LLAP IO ACID - includes handling is broken
[ https://issues.apache.org/jira/browse/HIVE-18738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18738: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the review! > LLAP IO ACID - includes handling is broken > -- > > Key: HIVE-18738 > URL: https://issues.apache.org/jira/browse/HIVE-18738 > Project: Hive > Issue Type: Bug >Reporter: Deepesh Khandelwal >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18738.01.patch, HIVE-18738.02.patch, > HIVE-18738.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18571) stats issues for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384446#comment-16384446 ] Sergey Shelukhin commented on HIVE-18571: - Another iteration. Addressed CR feedback, rebased, fixed CTAS WriteEntry to actually have table properties (not sure if that affected anything else but it definitely affected the WriteEntry write ID patch, and also hasTransactional... flag in SemanticAnalyzer), and also fixed some stats aggregation logic; mm_all stats tables now have all the stats but without the "accurate" flag. Some other out files will definitely change, I will see if anything else needs fixing or if we can just update those. Filed the followup jira for ACID stats; will also file more jiras based on other TODOs in the end. > stats issues for MM tables > -- > > Key: HIVE-18571 > URL: https://issues.apache.org/jira/browse/HIVE-18571 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18571.01.patch, HIVE-18571.02.patch, > HIVE-18571.03.patch, HIVE-18571.04.patch, HIVE-18571.patch > > > There are multiple stats aggregation issues with MM tables. > Some simple stats are double counted and some stats (simple stats) are > invalid for ACID table dirs altogether. > I have a patch almost ready, need to fix some more stuff and clean up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18571) stats issues for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18571: Attachment: HIVE-18571.04.patch > stats issues for MM tables > -- > > Key: HIVE-18571 > URL: https://issues.apache.org/jira/browse/HIVE-18571 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18571.01.patch, HIVE-18571.02.patch, > HIVE-18571.03.patch, HIVE-18571.04.patch, HIVE-18571.patch > > > There are multiple stats aggregation issues with MM tables. > Some simple stats are double counted and some stats (simple stats) are > invalid for ACID table dirs altogether. > I have a patch almost ready, need to fix some more stuff and clean up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384442#comment-16384442
]
Hive QA commented on HIVE-18848:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 40 new + 37 unchanged - 1
fixed = 77 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 12s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9449/dev-support/hive-personality.sh
|
| git revision | master / a4198f5 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9449/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9449/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9449/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Improve readability of filter conditions in explain plan when CBO is run
>
>
> Key: HIVE-18848
> URL: https://issues.apache.org/jira/browse/HIVE-18848
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18848.01.patch, HIVE-18848.02.patch
>
>
> CBO might return comparison operands in any non-deterministic order. Try to
> show {{ }} when possible, i.e., {{c < 10}} rather
> than {{10 > c}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18726) Implement DEFAULT constraint
[
https://issues.apache.org/jira/browse/HIVE-18726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384430#comment-16384430
]
Hive QA commented on HIVE-18726:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912691/HIVE-18726.4.patch
{color:green}SUCCESS:{color} +1 due to 15 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 77 failed/errored test(s), 13464 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Commented] (HIVE-18743) CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround is buggy.
[
https://issues.apache.org/jira/browse/HIVE-18743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384416#comment-16384416
]
Alexander Kolbasov commented on HIVE-18743:
---
[~kgyrtkirk] If you are fixing HIVE-17478, is there any value in fixing this
or we should just wait for HIVE-17478? Do you plan to do the same for hive-2
as well? If not, should this be hive-2 only fix? What do you think?
> CREATE TABLE on S3 data can be extremely slow. DO_NOT_UPDATE_STATS workaround
> is buggy.
> ---
>
> Key: HIVE-18743
> URL: https://issues.apache.org/jira/browse/HIVE-18743
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 1.2.0, 1.1.0, 2.0.2, 3.0.0
>Reporter: Alexander Behm
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-18743.04.patch
>
>
> When hive.stats.autogather=true then the Metastore lists all files under the
> table directory to populate basic stats like file counts and sizes. This file
> listing operation can be very expensive particularly on filesystems like S3.
> One way to address this issue is to reconfigure hive.stats.autogather=false.
> *Here's the bug*
> It is my understanding that the DO_NOT_UPDATE_STATS table property is
> intended to selectively prevent this stats collection. Unfortunately, this
> table property is checked *after* the expensive file listing operation, so
> the DO_NOT_UPDATE_STATS does not seem to work as intended. See:
> https://github.com/apache/hive/blob/master/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/utils/MetaStoreUtils.java#L633
> Relevant code snippet:
> {code}
> public static boolean updateTableStatsFast(Database db, Table tbl,
> Warehouse wh,
> boolean madeDir, boolean
> forceRecompute, EnvironmentContext environmentContext) throws MetaException {
> if (tbl.getPartitionKeysSize() == 0) {
> // Update stats only when unpartitioned
> FileStatus[] fileStatuses = wh.getFileStatusesForUnpartitionedTable(db,
> tbl);
> return updateTableStatsFast(tbl, fileStatuses, madeDir, forceRecompute,
> environmentContext); <--- DO_NOT_UPDATE_STATS is checked in here after
> wh.getFileStatusesForUnpartitionedTable() has already been called
> } else {
> return false;
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18854) Add metastore event handler for DEFAULT constraint
[
https://issues.apache.org/jira/browse/HIVE-18854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384411#comment-16384411
]
Vineet Garg commented on HIVE-18854:
[~akolb] HIVE-18726 is adding DEFAULT constraint. Similar to
{{AddNotNullConstraintEvent, AddUniqueConstraintEvent etc}} code needs to be
added to handle metastore event handling for this constraint.
> Add metastore event handler for DEFAULT constraint
> --
>
> Key: HIVE-18854
> URL: https://issues.apache.org/jira/browse/HIVE-18854
> Project: Hive
> Issue Type: Task
> Components: Metastore
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18726) Implement DEFAULT constraint
[ https://issues.apache.org/jira/browse/HIVE-18726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384409#comment-16384409 ] Ashutosh Chauhan commented on HIVE-18726: - +1 pending tests. > Implement DEFAULT constraint > > > Key: HIVE-18726 > URL: https://issues.apache.org/jira/browse/HIVE-18726 > Project: Hive > Issue Type: New Feature > Components: Query Planning, Query Processor >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18726.1.patch, HIVE-18726.2.patch, > HIVE-18726.3.patch, HIVE-18726.4.patch, HIVE-18726.5.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18854) Add metastore event handler for DEFAULT constraint
[ https://issues.apache.org/jira/browse/HIVE-18854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384405#comment-16384405 ] Alexander Kolbasov edited comment on HIVE-18854 at 3/3/18 1:28 AM: --- Please clarify what is the issue that this ticket suggests to resolve. was (Author: akolb): Please clarify what is the issue that this ticket suggest to resolve. > Add metastore event handler for DEFAULT constraint > -- > > Key: HIVE-18854 > URL: https://issues.apache.org/jira/browse/HIVE-18854 > Project: Hive > Issue Type: Task > Components: Metastore >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18854) Add metastore event handler for DEFAULT constraint
[ https://issues.apache.org/jira/browse/HIVE-18854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384405#comment-16384405 ] Alexander Kolbasov commented on HIVE-18854: --- Please clarify what is the issue that this ticket suggest to resolve. > Add metastore event handler for DEFAULT constraint > -- > > Key: HIVE-18854 > URL: https://issues.apache.org/jira/browse/HIVE-18854 > Project: Hive > Issue Type: Task > Components: Metastore >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[
https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384404#comment-16384404
]
Alexander Kolbasov commented on HIVE-18712:
---
As we start thinking about updating Hive Metastore APIs, we need to immediately
think about the object ownership issue.
Here is, for example, how Database object is defined:
{code:java}
enum PrincipalType {
USER = 1,
ROLE = 2,
GROUP = 3,
}{code}
{code:java}
// namespace for tables
struct Database {
1: string name,
...
5: optional PrincipalPrivilegeSet privileges,
6: optional string ownerName,
7: optional PrincipalType ownerType
} {code}
Is this really the model we would like to carry forward? Does USER/ROLE/GROUP
type adequately describe the ownership model we want to have? Would this still
apply in mixed cloud-like environments?
More over, do we really need the notion of the owner in the first place? We do
need an initial set of permissions for the object, but do we really need the
concept of an owner?
If we are defining the model from scratch, what do you think is a useful model
here?
> Design HMS Api v2
> -
>
> Key: HIVE-18712
> URL: https://issues.apache.org/jira/browse/HIVE-18712
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
>
> This is an umbrella Jira covering the design of Hive Metastore API v2.
> It is supposed to be a placeholder for discussion and design documents.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules
[ https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384400#comment-16384400 ] Alexander Kolbasov commented on HIVE-17751: --- My initial approach was for have hive-metastore-common and hive-metastore-client live as submodules of top-level Hive. [~pvary] suggested to move these modules in standalone-metastore and now I am running into weird maven dependency issues. [~alangates] [~vihangk1] - where do you thing is the right location for 1) Common code shared between hive-standalone-metastore and Hive needed for the client 2) Actual implementation of hive metastore client > Separate HMS Client and HMS server into separate sub-modules > > > Key: HIVE-17751 > URL: https://issues.apache.org/jira/browse/HIVE-17751 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Vihang Karajgaonkar >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-17751.06-standalone-metastore.patch > > > external applications which are interfacing with HMS should ideally only > include HMSClient library instead of one big library containing server as > well. We should ideally have a thin client library so that cross version > support for external applications is easier. We should sub-divide the > standalone module into possibly 3 modules (one for common classes, one for > client classes and one for server) or 2 sub-modules (one for client and one > for server) so that we can generate separate jars for HMS client and server. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-18712: - Assignee: Eugene Koifman > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Eugene Koifman >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman reassigned HIVE-18712: - Assignee: Alexander Kolbasov (was: Eugene Koifman) > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18726) Implement DEFAULT constraint
[ https://issues.apache.org/jira/browse/HIVE-18726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18726: --- Attachment: HIVE-18726.5.patch > Implement DEFAULT constraint > > > Key: HIVE-18726 > URL: https://issues.apache.org/jira/browse/HIVE-18726 > Project: Hive > Issue Type: New Feature > Components: Query Planning, Query Processor >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18726.1.patch, HIVE-18726.2.patch, > HIVE-18726.3.patch, HIVE-18726.4.patch, HIVE-18726.5.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18726) Implement DEFAULT constraint
[ https://issues.apache.org/jira/browse/HIVE-18726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18726: --- Status: Open (was: Patch Available) > Implement DEFAULT constraint > > > Key: HIVE-18726 > URL: https://issues.apache.org/jira/browse/HIVE-18726 > Project: Hive > Issue Type: New Feature > Components: Query Planning, Query Processor >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18726.1.patch, HIVE-18726.2.patch, > HIVE-18726.3.patch, HIVE-18726.4.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18726) Implement DEFAULT constraint
[ https://issues.apache.org/jira/browse/HIVE-18726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18726: --- Status: Patch Available (was: Open) > Implement DEFAULT constraint > > > Key: HIVE-18726 > URL: https://issues.apache.org/jira/browse/HIVE-18726 > Project: Hive > Issue Type: New Feature > Components: Query Planning, Query Processor >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18726.1.patch, HIVE-18726.2.patch, > HIVE-18726.3.patch, HIVE-18726.4.patch, HIVE-18726.5.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18855) Fix unit test TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
[ https://issues.apache.org/jira/browse/HIVE-18855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere reassigned HIVE-18855: - > Fix unit test TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1] > --- > > Key: HIVE-18855 > URL: https://issues.apache.org/jira/browse/HIVE-18855 > Project: Hive > Issue Type: Sub-task > Components: Tests >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > > Looks like this test has been broken for a while. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18726) Implement DEFAULT constraint
[
https://issues.apache.org/jira/browse/HIVE-18726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384382#comment-16384382
]
Hive QA commented on HIVE-18726:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 18s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-9448/patches/PreCommit-HIVE-Build-9448.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9448/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement DEFAULT constraint
>
>
> Key: HIVE-18726
> URL: https://issues.apache.org/jira/browse/HIVE-18726
> Project: Hive
> Issue Type: New Feature
> Components: Query Planning, Query Processor
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18726.1.patch, HIVE-18726.2.patch,
> HIVE-18726.3.patch, HIVE-18726.4.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18837) add a flag and disable some object pools in LLAP until further testing
[
https://issues.apache.org/jira/browse/HIVE-18837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384377#comment-16384377
]
Hive QA commented on HIVE-18837:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912686/HIVE-18837.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 13437 tests
executed
*Failed tests:*
{noformat}
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestLocationQueries - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=94)
[jira] [Updated] (HIVE-18051) qfiles: dataset support
[
https://issues.apache.org/jira/browse/HIVE-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor updated HIVE-18051:
Attachment: HIVE-18051.11.patch
> qfiles: dataset support
> ---
>
> Key: HIVE-18051
> URL: https://issues.apache.org/jira/browse/HIVE-18051
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-18051.01.patch, HIVE-18051.02.patch,
> HIVE-18051.03.patch, HIVE-18051.04.patch, HIVE-18051.05.patch,
> HIVE-18051.06.patch, HIVE-18051.07.patch, HIVE-18051.08.patch,
> HIVE-18051.09.patch, HIVE-18051.10.patch, HIVE-18051.11.patch
>
>
> it would be great to have some kind of test dataset support; currently there
> is the {{q_test_init.sql}} which is quite large; and I'm often override it
> with an invalid string; because I write independent qtests most of the time -
> and the load of {{src}} and other tables are just a waste of time for me ;
> not to mention that the loading of those tables may also trigger breakpoints
> - which is a bit annoying.
> Most of the tests are "only" using the {{src}} table and possibly 2 others;
> however the main init script contains a bunch of tables - meanwhile there are
> quite few other tests which could possibly also benefit from a more general
> feature; for example the creation of {{bucket_small}} is present in 20 q
> files.
> the proposal would be to enable the qfiles to be annotated with metadata like
> datasets:
> {code}
> --! qt:dataset:src,bucket_small
> {code}
> proposal for storing a dataset:
> * the loader script would be at: {{data/datasets/__NAME__/load.hive.sql}}
> * the table data could be stored under that location
> a draft about this; and other qfiles related ideas:
> https://docs.google.com/document/d/1KtcIx8ggL9LxDintFuJo8NQuvNWkmtvv_ekbWrTLNGc/edit?usp=sharing
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HIVE-18789) Disallow embedded element in UDFXPathUtil
[ https://issues.apache.org/jira/browse/HIVE-18789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reopened HIVE-18789: --- > Disallow embedded element in UDFXPathUtil > - > > Key: HIVE-18789 > URL: https://issues.apache.org/jira/browse/HIVE-18789 > Project: Hive > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18789.1.patch, HIVE-18789.2-branch-2.3.patch, > HIVE-18789.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18789) Disallow embedded element in UDFXPathUtil
[ https://issues.apache.org/jira/browse/HIVE-18789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384325#comment-16384325 ] Daniel Dai commented on HIVE-18789: --- Reopen to kick off branch 2.3 ptest. > Disallow embedded element in UDFXPathUtil > - > > Key: HIVE-18789 > URL: https://issues.apache.org/jira/browse/HIVE-18789 > Project: Hive > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18789.1.patch, HIVE-18789.2-branch-2.3.patch, > HIVE-18789.2.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18837) add a flag and disable some object pools in LLAP until further testing
[
https://issues.apache.org/jira/browse/HIVE-18837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384324#comment-16384324
]
Hive QA commented on HIVE-18837:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} llap-server: The patch generated 4 new + 177 unchanged
- 5 fixed = 181 total (was 182) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 1 new + 118 unchanged - 0
fixed = 119 total (was 118) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
16s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 29s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9447/dev-support/hive-personality.sh
|
| git revision | master / a4198f5 |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9447/yetus/patch-mvninstall-llap-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9447/yetus/diff-checkstyle-llap-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9447/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9447/yetus/patch-asflicense-problems.txt
|
| modules | C: common llap-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9447/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> add a flag and disable some object pools in LLAP until further testing
> --
>
> Key: HIVE-18837
> URL: https://issues.apache.org/jira/browse/HIVE-18837
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18837.patch
>
>
> There appears to be some subtle concurrency issue in FixedSizedObjectPool
> that happens with multiple consumers where some object may be retrieved
> twice.
> Unfortunately running a load test for hour(s) does not trigger it for me and
> overall it happens extremely rarely on non-specific tests; adding debug info
> at this level is a little bit difficult to determine how it could have
> happened and interlocked operations in
[jira] [Updated] (HIVE-18815) Remove unused feature in HPL/SQL
[ https://issues.apache.org/jira/browse/HIVE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-18815: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.3.3 3.0.0 Status: Resolved (was: Patch Available) Further pushed to 2.3 branch. > Remove unused feature in HPL/SQL > > > Key: HIVE-18815 > URL: https://issues.apache.org/jira/browse/HIVE-18815 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Fix For: 3.0.0, 2.3.3 > > Attachments: HIVE-18815.1-branch-2.3.patch, HIVE-18815.1.patch > > > Remove FTP feature in HPL/SQL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18788) Clean up inputs in JDBC PreparedStatement
[ https://issues.apache.org/jira/browse/HIVE-18788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-18788: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.3.3 3.0.0 Status: Resolved (was: Patch Available) Further commit to 2.3 branch. > Clean up inputs in JDBC PreparedStatement > - > > Key: HIVE-18788 > URL: https://issues.apache.org/jira/browse/HIVE-18788 > Project: Hive > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Fix For: 3.0.0, 2.3.3 > > Attachments: HIVE-18788.1.patch, HIVE-18788.2.patch, > HIVE-18788.3-branch-2.3.patch, HIVE-18788.3.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18854) Add metastore event handler for DEFAULT constraint
[ https://issues.apache.org/jira/browse/HIVE-18854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384314#comment-16384314 ] Vineet Garg commented on HIVE-18854: [~daijy] comment on HIVE-18726 "Default constraint should be handled similar to notnull/unique. We need to handle both bootstrap (similar to HIVE-17366) and increment (similar HIVE-15705) cases, and also add proper test to TestReplicationScenarios. I think it is fine to add it on a separate ticket." > Add metastore event handler for DEFAULT constraint > -- > > Key: HIVE-18854 > URL: https://issues.apache.org/jira/browse/HIVE-18854 > Project: Hive > Issue Type: Task > Components: Metastore >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18854) Add metastore event handler for DEFAULT constraint
[ https://issues.apache.org/jira/browse/HIVE-18854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18854: --- Fix Version/s: 3.0.0 > Add metastore event handler for DEFAULT constraint > -- > > Key: HIVE-18854 > URL: https://issues.apache.org/jira/browse/HIVE-18854 > Project: Hive > Issue Type: Task > Components: Metastore >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18854) Add metastore event handler for DEFAULT constraint
[ https://issues.apache.org/jira/browse/HIVE-18854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-18854: -- > Add metastore event handler for DEFAULT constraint > -- > > Key: HIVE-18854 > URL: https://issues.apache.org/jira/browse/HIVE-18854 > Project: Hive > Issue Type: Task > Components: Metastore >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18852) Misleading error message in alter table validation
[
https://issues.apache.org/jira/browse/HIVE-18852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384306#comment-16384306
]
Vineet Garg commented on HIVE-18852:
You are right about the offending line. But I don't think changing {{dbname}}
to {{newDbName}} would be correct thing to do since this error is under {{catch
(NoSuchObjectException e)}} and could result due to non-existence of any other
object. We should probably change the error message to {{Unable to change
partition or table. Object does not exist: + e.getMessage()}}
> Misleading error message in alter table validation
> --
>
> Key: HIVE-18852
> URL: https://issues.apache.org/jira/browse/HIVE-18852
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 2.4.0
>Reporter: Dan Burkert
>Priority: Major
>
> The metastore's validation error message when attempting to rename a table to
> a non-existent database is wrong. For instance, attempting to alter table
> 'db.table' to 'non_existent_database.table' results in the Thrift error:
> {{TException - service has thrown: InvalidOperationException(message=Unable
> to change partition or table. Database db does not exist Check metastore logs
> for detailed stack.non_existent_database)}}
> I believe the offending line of code is
> [here|https://github.com/apache/hive/blob/branch-2/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java?utf8=%E2%9C%93#L331-L333],
> notice that {{dbname}} is used in the message, not {{newDbName}}. I don't
> know if switching that would cause the case of a non-existing {{dbname}} case
> to regress, though.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18788) Clean up inputs in JDBC PreparedStatement
[
https://issues.apache.org/jira/browse/HIVE-18788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384297#comment-16384297
]
Hive QA commented on HIVE-18788:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912687/HIVE-18788.3-branch-2.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10571 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion02
(batchId=263)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testNonAcidToAcidConversion02
(batchId=275)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion02
(batchId=272)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9446/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9446/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9446/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12912687 - PreCommit-HIVE-Build
> Clean up inputs in JDBC PreparedStatement
> -
>
> Key: HIVE-18788
> URL: https://issues.apache.org/jira/browse/HIVE-18788
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18788.1.patch, HIVE-18788.2.patch,
> HIVE-18788.3-branch-2.3.patch, HIVE-18788.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18853) Alter table change column shouldn't let define same constraint on a column with existing constraint
[
https://issues.apache.org/jira/browse/HIVE-18853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18853:
---
Description:
{code:sql}
create table ttest(i int constraint nn1 not null enable);
alter table ttest change i i int constraint nn2 not null enable);
{code}
The above statements will end up creating multiple not null constraints on
column i/j.
{code:sql}
desc formatted ttest;
# Not Null Constraints
Table: constraints.ttest
Constraint Name:nn1
Column Name:i
Constraint Name:nn2
Column Name:i
{code}
was:
{code:sql}
create table ttest(i int constraint nn1 not null enable);
alter table ttest change i j int constraint nn2 not null enable);
{code}
The above statements will end up creating multiple not null constraints on
column i/j.
{code:sql}
desc formatted ttest;
# Not Null Constraints
Table: constraints.ttest
Constraint Name:nn1
Column Name:null
Constraint Name:nn2
Column Name:j
{code}
> Alter table change column shouldn't let define same constraint on a column
> with existing constraint
> ---
>
> Key: HIVE-18853
> URL: https://issues.apache.org/jira/browse/HIVE-18853
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> {code:sql}
> create table ttest(i int constraint nn1 not null enable);
> alter table ttest change i i int constraint nn2 not null enable);
> {code}
> The above statements will end up creating multiple not null constraints on
> column i/j.
> {code:sql}
> desc formatted ttest;
> # Not Null Constraints
> Table:constraints.ttest
> Constraint Name: nn1
> Column Name: i
> Constraint Name: nn2
> Column Name: i
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18853) Alter table change column shouldn't let define same constraint on a column with existing constraint
[
https://issues.apache.org/jira/browse/HIVE-18853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-18853:
--
> Alter table change column shouldn't let define same constraint on a column
> with existing constraint
> ---
>
> Key: HIVE-18853
> URL: https://issues.apache.org/jira/browse/HIVE-18853
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> {code:sql}
> create table ttest(i int constraint nn1 not null enable);
> alter table ttest change i j int constraint nn2 not null enable);
> {code}
> The above statements will end up creating multiple not null constraints on
> column i/j.
> {code:sql}
> desc formatted ttest;
> # Not Null Constraints
> Table:constraints.ttest
> Constraint Name: nn1
> Column Name: null
> Constraint Name: nn2
> Column Name: j
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18851) make Hive basic stats valid for ACID; clean up and refactor the code
[ https://issues.apache.org/jira/browse/HIVE-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18851: -- Component/s: Transactions > make Hive basic stats valid for ACID; clean up and refactor the code > > > Key: HIVE-18851 > URL: https://issues.apache.org/jira/browse/HIVE-18851 > Project: Hive > Issue Type: Bug > Components: Statistics, Transactions >Reporter: Sergey Shelukhin >Priority: Major > Labels: ACID > > HIVE-18571 started as a couple small fixes for MM tables, but ended up as a > somewhat major cleanup of stats for ACID tables; however it doesn't do that > rigorously and not for all cases. > This is a follow-up JIRA to implement stats for ACID properly (potentially > also with ACID semantics similar to those of queries, but that could be > another follow-up - for now, at least they should be based on the correct set > of files). > Overall I've discovered that Hive stats code is spread all over in random > places in code base and is brittle and inconsistent, esp. for any complex > scenario like ACID tables. > So, instead of making ad-hoc fixes everywhere, I think at the minimum it > should be moved to a single spot (so that e.g. BasicStatsTask, > BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same > code with the same logic) and made valid for ACID. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18851) make Hive basic stats valid for ACID; clean up and refactor the code
[ https://issues.apache.org/jira/browse/HIVE-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18851: -- Component/s: Statistics > make Hive basic stats valid for ACID; clean up and refactor the code > > > Key: HIVE-18851 > URL: https://issues.apache.org/jira/browse/HIVE-18851 > Project: Hive > Issue Type: Bug > Components: Statistics, Transactions >Reporter: Sergey Shelukhin >Priority: Major > Labels: ACID > > HIVE-18571 started as a couple small fixes for MM tables, but ended up as a > somewhat major cleanup of stats for ACID tables; however it doesn't do that > rigorously and not for all cases. > This is a follow-up JIRA to implement stats for ACID properly (potentially > also with ACID semantics similar to those of queries, but that could be > another follow-up - for now, at least they should be based on the correct set > of files). > Overall I've discovered that Hive stats code is spread all over in random > places in code base and is brittle and inconsistent, esp. for any complex > scenario like ACID tables. > So, instead of making ad-hoc fixes everywhere, I think at the minimum it > should be moved to a single spot (so that e.g. BasicStatsTask, > BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same > code with the same logic) and made valid for ACID. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18718) Integer like types throws error when there is a mismatch
[ https://issues.apache.org/jira/browse/HIVE-18718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janaki Lahorani updated HIVE-18718: --- Attachment: HIVE-18718.2.patch > Integer like types throws error when there is a mismatch > > > Key: HIVE-18718 > URL: https://issues.apache.org/jira/browse/HIVE-18718 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18718.1.patch, HIVE-18718.2.patch > > > If a value is saved with long type and read as int type it results in > FAILED: Execution Error, return code 2 from > org.apache.hadoop.hive.ql.exec.mr.MapRedTask -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18746) add_months should validate the date first
[
https://issues.apache.org/jira/browse/HIVE-18746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384261#comment-16384261
]
Kryvenko Igor commented on HIVE-18746:
--
[~vgarg] Please, could you review?
> add_months should validate the date first
> -
>
> Key: HIVE-18746
> URL: https://issues.apache.org/jira/browse/HIVE-18746
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Subhasis Gorai
>Assignee: Kryvenko Igor
>Priority: Minor
> Attachments: HIVE-18746.1.patch, HIVE-18746.3.patch,
> HIVE-18746.4.patch, HIVE-18746.5.patch, HIVE-18746.6.patch,
> HIVE-18746.7.patch, HIVE-18746.patch
>
>
> hive (sbg_hvc_ods)> select add_months('2017-02-28', 1);
> OK
> _c0
> 2017-03-31
> Time taken: 0.107 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)> select add_months('2017-02-29', 1);
> OK
> _c0
> 2017-04-01
> Time taken: 0.084 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)>
>
> '2017-02-29' is an invalid date.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18791) Fix TestJdbcWithMiniHS2#testHttpHeaderSize
[ https://issues.apache.org/jira/browse/HIVE-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18791: -- Attachment: HIVE-18791.1.patch > Fix TestJdbcWithMiniHS2#testHttpHeaderSize > -- > > Key: HIVE-18791 > URL: https://issues.apache.org/jira/browse/HIVE-18791 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18791.1.patch > > > TestJdbcWithMiniHS2#testHttpHeaderSize tests whether config of http header > sizes works by using a long username. The local scratch directory for the > session uses the username as part of its path. When this name is more than > 255 chars (on most modern file systems) then the directory creation will > fail. HIVE-18625 made this failure throw an exception, which has caused a > regression in testHttpHeaderSize. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18791) Fix TestJdbcWithMiniHS2#testHttpHeaderSize
[ https://issues.apache.org/jira/browse/HIVE-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18791: -- Status: Patch Available (was: Open) > Fix TestJdbcWithMiniHS2#testHttpHeaderSize > -- > > Key: HIVE-18791 > URL: https://issues.apache.org/jira/browse/HIVE-18791 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18791.1.patch > > > TestJdbcWithMiniHS2#testHttpHeaderSize tests whether config of http header > sizes works by using a long username. The local scratch directory for the > session uses the username as part of its path. When this name is more than > 255 chars (on most modern file systems) then the directory creation will > fail. HIVE-18625 made this failure throw an exception, which has caused a > regression in testHttpHeaderSize. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[ https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384254#comment-16384254 ] Deepak Jaiswal commented on HIVE-17055: --- I have seen it happen in many tests not just this one. The flaky stats seems like an independent issue. > Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb] > - > > Key: HIVE-17055 > URL: https://issues.apache.org/jira/browse/HIVE-17055 > Project: Hive > Issue Type: Sub-task >Reporter: Janaki Lahorani >Assignee: Deepak Jaiswal >Priority: Major > > Client Execution succeeded but contained differences (error code = 1) after > executing llap_smb.q > 324,325c324,325 > < 2000 9 52 > < 2001 0 139630 > --- > > 2001 4 139630 > > 2001 6 52 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18815) Remove unused feature in HPL/SQL
[
https://issues.apache.org/jira/browse/HIVE-18815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384245#comment-16384245
]
Hive QA commented on HIVE-18815:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912689/HIVE-18815.1-branch-2.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10569 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[comments] (batchId=35)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion02
(batchId=263)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testNonAcidToAcidConversion02
(batchId=275)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion02
(batchId=272)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9445/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9445/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9445/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12912689 - PreCommit-HIVE-Build
> Remove unused feature in HPL/SQL
>
>
> Key: HIVE-18815
> URL: https://issues.apache.org/jira/browse/HIVE-18815
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18815.1-branch-2.3.patch, HIVE-18815.1.patch
>
>
> Remove FTP feature in HPL/SQL.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18851) make Hive basic stats valid for ACID; clean up and refactor the code
[ https://issues.apache.org/jira/browse/HIVE-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18851: Description: HIVE-18571 started as a couple small fixes for MM tables, but ended up as a somewhat major cleanup of stats for ACID tables; however it doesn't do that rigorously and not for all cases. This is a follow-up JIRA to implement stats for ACID properly (potentially also with ACID semantics similar to those of queries, but that could be another follow-up - for now, at least they should be based on the correct set of files). Overall I've discovered that Hive stats code is spread all over in random places in code base and is brittle and inconsistent, esp. for any complex scenario like ACID tables. So, instead of making ad-hoc fixes everywhere, I think at the minimum it should be moved to a single spot (so that e.g. BasicStatsTask, BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same code with the same logic) and made valid for ACID. was: HIVE-18571 started as a couple small fixes for MM tables, but ends up making stats for ACID tables work better in general, but not rigorously and not for all cases. This is a follow-up JIRA to implement stats for ACID properly (potentially also with ACID semantics similar to those of queries, but that could be another follow-up - for now, at least they should be based on the correct set of files). Overall I've discovered that Hive stats code is spread all over in random places in code base and is brittle and inconsistent, esp. for any complex scenario like ACID tables. So, instead of making ad-hoc fixes everywhere, I think at the minimum it should be moved to a single spot (so that e.g. BasicStatsTask, BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same code with the same logic) and made valid for ACID. > make Hive basic stats valid for ACID; clean up and refactor the code > > > Key: HIVE-18851 > URL: https://issues.apache.org/jira/browse/HIVE-18851 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > Labels: ACID > > HIVE-18571 started as a couple small fixes for MM tables, but ended up as a > somewhat major cleanup of stats for ACID tables; however it doesn't do that > rigorously and not for all cases. > This is a follow-up JIRA to implement stats for ACID properly (potentially > also with ACID semantics similar to those of queries, but that could be > another follow-up - for now, at least they should be based on the correct set > of files). > Overall I've discovered that Hive stats code is spread all over in random > places in code base and is brittle and inconsistent, esp. for any complex > scenario like ACID tables. > So, instead of making ad-hoc fixes everywhere, I think at the minimum it > should be moved to a single spot (so that e.g. BasicStatsTask, > BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same > code with the same logic) and made valid for ACID. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18851) make Hive basic stats valid for ACID; clean up and refactor the code
[ https://issues.apache.org/jira/browse/HIVE-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18851: Description: HIVE-18571 started as a couple small fixes for MM tables, but ends up making stats for ACID tables work better in general, but not rigorously and not for all cases. This is a follow-up JIRA to implement stats for ACID properly (potentially also with ACID semantics similar to those of queries, but that could be another follow-up - for now, at least they should be based on the correct set of files). Overall I've discovered that Hive stats code is spread all over in random places in code base and is brittle and inconsistent, esp. for any complex scenario like ACID tables. So, instead of making ad-hoc fixes everywhere, I think at the minimum it should be moved to a single spot (so that e.g. BasicStatsTask, BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same code with the same logic) and made valid for ACID. was: HIVE-18571 that started as a couple small fixes for MM tables, but ends up making stats for ACID tables work better in general, but not rigorously and not for all cases. This is a follow-up JIRA to implement stats for ACID properly (potentially also with ACID semantics similar to those of queries, but that could be another follow-up - for now, at least they should be based on the correct set of files). Overall I've discovered that Hive stats code is spread all over in random places in code base and is brittle and inconsistent, esp. for any complex scenario like ACID tables. So, instead of making ad-hoc fixes everywhere, I think at the minimum it should be moved to a single spot (so that e.g. BasicStatsTask, BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same code with the same logic) and made valid for ACID. > make Hive basic stats valid for ACID; clean up and refactor the code > > > Key: HIVE-18851 > URL: https://issues.apache.org/jira/browse/HIVE-18851 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > Labels: ACID > > HIVE-18571 started as a couple small fixes for MM tables, but ends up making > stats for ACID tables work better in general, but not rigorously and not for > all cases. > This is a follow-up JIRA to implement stats for ACID properly (potentially > also with ACID semantics similar to those of queries, but that could be > another follow-up - for now, at least they should be based on the correct set > of files). > Overall I've discovered that Hive stats code is spread all over in random > places in code base and is brittle and inconsistent, esp. for any complex > scenario like ACID tables. > So, instead of making ad-hoc fixes everywhere, I think at the minimum it > should be moved to a single spot (so that e.g. BasicStatsTask, > BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same > code with the same logic) and made valid for ACID. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18851) make Hive basic stats valid for ACID; clean up and refactor the code
[ https://issues.apache.org/jira/browse/HIVE-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18851: Labels: ACID (was: ) > make Hive basic stats valid for ACID; clean up and refactor the code > > > Key: HIVE-18851 > URL: https://issues.apache.org/jira/browse/HIVE-18851 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > Labels: ACID > > HIVE-18571 that started as a couple small fixes for MM tables, but ends up > making stats for ACID tables work better in general, but not rigorously and > not for all cases. > This is a follow-up JIRA to implement stats for ACID properly (potentially > also with ACID semantics similar to those of queries, but that could be > another follow-up - for now, at least they should be based on the correct set > of files). > Overall I've discovered that Hive stats code is spread all over in random > places in code base and is brittle and inconsistent, esp. for any complex > scenario like ACID tables. > So, instead of making ad-hoc fixes everywhere, I think at the minimum it > should be moved to a single spot (so that e.g. BasicStatsTask, > BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same > code with the same logic) and made valid for ACID. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18851) make Hive basic stats valid for ACID; clean up and refactor the code
[ https://issues.apache.org/jira/browse/HIVE-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18851: Description: HIVE-18571 that started as a couple small fixes for MM tables, but ends up making stats for ACID tables work better in general, but not rigorously and not for all cases. This is a follow-up JIRA to implement stats for ACID properly (potentially also with ACID semantics similar to those of queries, but that could be another follow-up - for now, at least they should be based on the correct set of files). Overall I've discovered that Hive stats code is spread all over in random places in code base and is brittle and inconsistent, esp. for any complex scenario like ACID tables. So, instead of making ad-hoc fixes everywhere, I think at the minimum it should be moved to a single spot (so that e.g. BasicStatsTask, BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same code with the same logic) and made valid for ACID. was: Based on HIVE-18571 that started as a couple small fixes for MM tables, but ends up making stats for ACID tables work better in general, but not rigorously and not for all cases. Overall I've discovered that Hive stats code is spread all over in random places in code base and is brittle and inconsistent, esp. for any complex scenario like ACID tables. I think at the minimum it should be moved to a single spot (so that e.g. BasicStatsTask, BasicStatsTaskNoJob, metastore stats generation, etc all use the same code with the same logic) and made valid for ACID. > make Hive basic stats valid for ACID; clean up and refactor the code > > > Key: HIVE-18851 > URL: https://issues.apache.org/jira/browse/HIVE-18851 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > > HIVE-18571 that started as a couple small fixes for MM tables, but ends up > making stats for ACID tables work better in general, but not rigorously and > not for all cases. > This is a follow-up JIRA to implement stats for ACID properly (potentially > also with ACID semantics similar to those of queries, but that could be > another follow-up - for now, at least they should be based on the correct set > of files). > Overall I've discovered that Hive stats code is spread all over in random > places in code base and is brittle and inconsistent, esp. for any complex > scenario like ACID tables. > So, instead of making ad-hoc fixes everywhere, I think at the minimum it > should be moved to a single spot (so that e.g. BasicStatsTask, > BasicStatsTaskNoJob, metastore "quick" stats generation, etc all use the same > code with the same logic) and made valid for ACID. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18851) make Hive basic stats valid for ACID; clean up and refactor the code
[ https://issues.apache.org/jira/browse/HIVE-18851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18851: Summary: make Hive basic stats valid for ACID; clean up and refactor the code (was: make Hive basic stats valid for ACID; cleanup and refactor the code) > make Hive basic stats valid for ACID; clean up and refactor the code > > > Key: HIVE-18851 > URL: https://issues.apache.org/jira/browse/HIVE-18851 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Priority: Major > > Based on HIVE-18571 that started as a couple small fixes for MM tables, but > ends up making stats for ACID tables work better in general, but not > rigorously and not for all cases. > Overall I've discovered that Hive stats code is spread all over in random > places in code base and is brittle and inconsistent, esp. for any complex > scenario like ACID tables. > I think at the minimum it should be moved to a single spot (so that e.g. > BasicStatsTask, BasicStatsTaskNoJob, metastore stats generation, etc all use > the same code with the same logic) and made valid for ACID. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18791) Fix TestJdbcWithMiniHS2#testHttpHeaderSize
[ https://issues.apache.org/jira/browse/HIVE-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384218#comment-16384218 ] Andrew Sherman commented on HIVE-18791: --- The test TestJdbcWithMiniHS2.testHttpHeaderSize() is testing that HS2 can be reconfigured to use different maximum sizes of http headers when Thrift uses http as its transport. In this mode the http header contains a user name and password encoded using http basic encoding. The original test varies the length of the user name to test (1) that that a too large header fails with 413 Payload Too Large, and (2) that increasing the maximum sizes of http headers allows the large header to work. In HS2 when the connection is made a Session is created (note that in this configuration the user name and password are not verified). The local scratch directory for the session uses the username as part of its path. When this name is more than 255 chars (on most modern file systems) then the directory creation will fail. HIVE-18625 made this failure newly throw an exception, which caused a regression in testHttpHeaderSize() when the session was created with a long user name. Fix this by having the test vary the password rather then the user name. The test is functionally identical but does not hit the directory creation exception. > Fix TestJdbcWithMiniHS2#testHttpHeaderSize > -- > > Key: HIVE-18791 > URL: https://issues.apache.org/jira/browse/HIVE-18791 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > > TestJdbcWithMiniHS2#testHttpHeaderSize tests whether config of http header > sizes works by using a long username. The local scratch directory for the > session uses the username as part of its path. When this name is more than > 255 chars (on most modern file systems) then the directory creation will > fail. HIVE-18625 made this failure throw an exception, which has caused a > regression in testHttpHeaderSize. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18825) Define ValidTxnList before starting query optimization
[
https://issues.apache.org/jira/browse/HIVE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384191#comment-16384191
]
Jesus Camacho Rodriguez commented on HIVE-18825:
HIVE-18825.03.patch cleans valid txn list property for {{compileAndRespond}}
statements (fixes {{TestDbTxnManager2}} failures).
> Define ValidTxnList before starting query optimization
> --
>
> Key: HIVE-18825
> URL: https://issues.apache.org/jira/browse/HIVE-18825
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-18825.01.patch, HIVE-18825.02.patch,
> HIVE-18825.03.patch, HIVE-18825.patch
>
>
> Consider a set of tables used by a materialized view where inserts happened
> after the materialization was created. To compute incremental view
> maintenance, we need to be able to filter only new rows from those base
> tables. That can be done by inserting a filter operator with condition e.g.
> {{ROW\_\_ID.transactionId < highwatermark and ROW\_\_ID.transactionId NOT
> IN()}} on top of the MVs query definition and triggering the
> rewriting (which should in turn produce a partial rewriting). However, to do
> that, we need to have a value for {{ValidTxnList}} during query compilation
> so we know the snapshot that we are querying.
> This patch aims to generate {{ValidTxnList}} before query optimization. There
> should not be any visible changes for end user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18825) Define ValidTxnList before starting query optimization
[
https://issues.apache.org/jira/browse/HIVE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18825:
---
Attachment: HIVE-18825.03.patch
> Define ValidTxnList before starting query optimization
> --
>
> Key: HIVE-18825
> URL: https://issues.apache.org/jira/browse/HIVE-18825
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-18825.01.patch, HIVE-18825.02.patch,
> HIVE-18825.03.patch, HIVE-18825.patch
>
>
> Consider a set of tables used by a materialized view where inserts happened
> after the materialization was created. To compute incremental view
> maintenance, we need to be able to filter only new rows from those base
> tables. That can be done by inserting a filter operator with condition e.g.
> {{ROW\_\_ID.transactionId < highwatermark and ROW\_\_ID.transactionId NOT
> IN()}} on top of the MVs query definition and triggering the
> rewriting (which should in turn produce a partial rewriting). However, to do
> that, we need to have a value for {{ValidTxnList}} during query compilation
> so we know the snapshot that we are querying.
> This patch aims to generate {{ValidTxnList}} before query optimization. There
> should not be any visible changes for end user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18746) add_months should validate the date first
[
https://issues.apache.org/jira/browse/HIVE-18746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384181#comment-16384181
]
Hive QA commented on HIVE-18746:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912682/HIVE-18746.7.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 102 failed/errored test(s), 13459 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Commented] (HIVE-18723) CompactorOutputCommitter.commitJob() - check rename() ret val
[
https://issues.apache.org/jira/browse/HIVE-18723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384153#comment-16384153
]
Kryvenko Igor commented on HIVE-18723:
--
I'll take a look on it.
> CompactorOutputCommitter.commitJob() - check rename() ret val
> -
>
> Key: HIVE-18723
> URL: https://issues.apache.org/jira/browse/HIVE-18723
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Kryvenko Igor
>Priority: Major
> Attachments: HIVE-18723.1.patch, HIVE-18723.2.patch, HIVE-18723.patch
>
>
> right now ret val is ignored {{fs.rename(fileStatus.getPath(), newPath); }}
> Should this use {{FileUtils.ename(FileSystem fs, Path sourcePath, Path
> destPath, Configuration conf) }}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18746) add_months should validate the date first
[
https://issues.apache.org/jira/browse/HIVE-18746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384142#comment-16384142
]
Hive QA commented on HIVE-18746:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
6s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
55s{color} | {color:red} root: The patch generated 9 new + 60 unchanged - 1
fixed = 69 total (was 61) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
34s{color} | {color:red} ql: The patch generated 9 new + 53 unchanged - 1 fixed
= 62 total (was 54) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9444/dev-support/hive-personality.sh
|
| git revision | master / a4198f5 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9444/yetus/diff-checkstyle-root.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9444/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9444/yetus/patch-asflicense-problems.txt
|
| modules | C: common . ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9444/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> add_months should validate the date first
> -
>
> Key: HIVE-18746
> URL: https://issues.apache.org/jira/browse/HIVE-18746
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Subhasis Gorai
>Assignee: Kryvenko Igor
>Priority: Minor
> Attachments: HIVE-18746.1.patch, HIVE-18746.3.patch,
> HIVE-18746.4.patch, HIVE-18746.5.patch, HIVE-18746.6.patch,
> HIVE-18746.7.patch, HIVE-18746.patch
>
>
> hive (sbg_hvc_ods)> select add_months('2017-02-28', 1);
> OK
> _c0
> 2017-03-31
> Time taken: 0.107 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)> select add_months('2017-02-29', 1);
> OK
> _c0
> 2017-04-01
> Time taken: 0.084 seconds, Fetched: 1 row(s)
> hive (sbg_hvc_ods)>
>
> '2017-02-29' is an invalid date.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18836) backport HIVE-17600 to hive 1
[ https://issues.apache.org/jira/browse/HIVE-18836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18836: Resolution: Fixed Fix Version/s: 1.3.0 Status: Resolved (was: Patch Available) I ran some tests and could repro the failures but then realized I forgot to apply the patch. Looks like branch-1 tests are fubar. Committed to branch-1. Thanks for the review! > backport HIVE-17600 to hive 1 > - > > Key: HIVE-18836 > URL: https://issues.apache.org/jira/browse/HIVE-18836 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 1.3.0 > > Attachments: HIVE-18836.01-branch-1.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18738) LLAP IO ACID - includes handling is broken
[ https://issues.apache.org/jira/browse/HIVE-18738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384120#comment-16384120 ] Sergey Shelukhin edited comment on HIVE-18738 at 3/2/18 8:31 PM: - Hm... all of the test failures are known except for udaf_context_ngrams that has a diff that looks like nothing related to this patch and also didn't fail on previous iter (or locally). Will commit later today if nobody objects. was (Author: sershe): Hm... all of the test failures are known except for udaf_context_ngrams that has a diff that looks like nothing related to this patch and also didn't fail on previous iter. Will commit later today if nobody objects. > LLAP IO ACID - includes handling is broken > -- > > Key: HIVE-18738 > URL: https://issues.apache.org/jira/browse/HIVE-18738 > Project: Hive > Issue Type: Bug >Reporter: Deepesh Khandelwal >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18738.01.patch, HIVE-18738.02.patch, > HIVE-18738.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18738) LLAP IO ACID - includes handling is broken
[ https://issues.apache.org/jira/browse/HIVE-18738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384120#comment-16384120 ] Sergey Shelukhin commented on HIVE-18738: - Hm... all of the test failures are known except for udaf_context_ngrams that has a diff that looks like nothing related to this patch and also didn't fail on previous iter. Will commit later today if nobody objects. > LLAP IO ACID - includes handling is broken > -- > > Key: HIVE-18738 > URL: https://issues.apache.org/jira/browse/HIVE-18738 > Project: Hive > Issue Type: Bug >Reporter: Deepesh Khandelwal >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18738.01.patch, HIVE-18738.02.patch, > HIVE-18738.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18723) CompactorOutputCommitter.commitJob() - check rename() ret val
[
https://issues.apache.org/jira/browse/HIVE-18723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384118#comment-16384118
]
Eugene Koifman commented on HIVE-18723:
---
unfortunately Jenkins output is gone. Are you sure
org.apache.hadoop.hive.ql.txn.compactor.TestWorker.minorWithOpenInMiddle
(batchId=268)
org.apache.hadoop.hive.ql.txn.compactor.TestWorker2.minorWithOpenInMiddle
(batchId=268)
are not related?
> CompactorOutputCommitter.commitJob() - check rename() ret val
> -
>
> Key: HIVE-18723
> URL: https://issues.apache.org/jira/browse/HIVE-18723
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 1.0.0
>Reporter: Eugene Koifman
>Assignee: Kryvenko Igor
>Priority: Major
> Attachments: HIVE-18723.1.patch, HIVE-18723.2.patch, HIVE-18723.patch
>
>
> right now ret val is ignored {{fs.rename(fileStatus.getPath(), newPath); }}
> Should this use {{FileUtils.ename(FileSystem fs, Path sourcePath, Path
> destPath, Configuration conf) }}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384113#comment-16384113 ] Vihang Karajgaonkar commented on HIVE-18712: I think its a great idea to have v2 APIs. I think we should to make sure that APIs are extensible for the future. A lot of the our current APIs are not very extensible which caused everyone to add newer APIs which could have easily been implemented by extending the older ones. +1 to have request/response objects for the API instead of non-extensible input and output arguments. I think one problem with the current APIs, is that there is no way to filter objects on the server side. For instance, it should be possible to get objects based on some filter criteria. Additionally, I think it would be good to have some way to fetch objects in batches rather than getting all the objects in one go. This adds a lot of memory pressure on HMS when clients requests huge number of objects. I like the partition expression proxy API which enables clients to push down filters when querying partitions. We should have similar generic way to return only stuff which is needed by the clients. Another interesting idea to consider is if we can have some kind of version handshake between client and server. This helps with cross-version compatibility. I am not sure how other systems handle these. But it would be good to take a look at that. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18712) Design HMS Api v2
[ https://issues.apache.org/jira/browse/HIVE-18712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384097#comment-16384097 ] Alexander Kolbasov commented on HIVE-18712: --- Looks like there are some tools (e.g. [gRPC-REST|[https://grpc.io/blog/coreos])] that allow building both gRPC and REST interface at the same time which is really nice. > Design HMS Api v2 > - > > Key: HIVE-18712 > URL: https://issues.apache.org/jira/browse/HIVE-18712 > Project: Hive > Issue Type: Improvement > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Alexander Kolbasov >Priority: Major > > This is an umbrella Jira covering the design of Hive Metastore API v2. > It is supposed to be a placeholder for discussion and design documents. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18738) LLAP IO ACID - includes handling is broken
[
https://issues.apache.org/jira/browse/HIVE-18738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384087#comment-16384087
]
Hive QA commented on HIVE-18738:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912678/HIVE-18738.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 13055 tests
executed
*Failed tests:*
{noformat}
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Commented] (HIVE-18685) Add catalogs to metastore
[ https://issues.apache.org/jira/browse/HIVE-18685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384079#comment-16384079 ] Vihang Karajgaonkar commented on HIVE-18685: Thanks [~alangates] for taking this up. I think this would be a good addition. I went through the doc attached and all the comments above and I have some questions: bq. It will not be possible to address a table as catalog.database.table. Which catalog you are connected to will be determine when the session is started and is immutable initially. If you cannot address table using catalog.database.table how would a user be able to issue cross catalog queries. eg. join between tables across catalogs? Regarding catalog location, Does it mean that all databases/tables created under a catalog default to a location of catalog? As a corollary to this, if a user gives a location which doesn't make sense with respect to the catalog's location (catalogs being the logical separation of systems) should we have hooks which can allow us to enforce the location of the catalogs? Should the catalog have a owner as well? Who is allowed to create/drop the catalog? > Add catalogs to metastore > - > > Key: HIVE-18685 > URL: https://issues.apache.org/jira/browse/HIVE-18685 > Project: Hive > Issue Type: New Feature > Components: Metastore, Parser, Security, SQL >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HMS Catalog Design Doc.pdf > > > SQL supports two levels of namespaces, called in the spec catalogs and > schemas (with schema being equivalent to Hive's database). I propose to add > the upper level of catalog. The attached design doc covers the use cases, > requirements, and brief discussion of how it will be implemented in a > backwards compatible way. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18833) Auto Merge fails when "insert into directory as orcfile"
[
https://issues.apache.org/jira/browse/HIVE-18833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384070#comment-16384070
]
Daniel Dai commented on HIVE-18833:
---
Need to update another golden file.
> Auto Merge fails when "insert into directory as orcfile"
>
>
> Key: HIVE-18833
> URL: https://issues.apache.org/jira/browse/HIVE-18833
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18833.1.patch, HIVE-18833.2.patch
>
>
> Here is the reproduction:
> {code}
> set mapreduce.job.reduces=2;
> set hive.merge.tezfiles=true;
> INSERT OVERWRITE DIRECTORY 'output' stored as orcfile select age, avg(gpa)
> from student group by age;
> {code}
> Error message: File Merge Stage after Maps completion is considering input as
> "input format: org.apache.hadoop.mapred.TextInputFormat" instead of
> "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18833) Auto Merge fails when "insert into directory as orcfile"
[
https://issues.apache.org/jira/browse/HIVE-18833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-18833:
--
Attachment: HIVE-18833.2.patch
> Auto Merge fails when "insert into directory as orcfile"
>
>
> Key: HIVE-18833
> URL: https://issues.apache.org/jira/browse/HIVE-18833
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-18833.1.patch, HIVE-18833.2.patch
>
>
> Here is the reproduction:
> {code}
> set mapreduce.job.reduces=2;
> set hive.merge.tezfiles=true;
> INSERT OVERWRITE DIRECTORY 'output' stored as orcfile select age, avg(gpa)
> from student group by age;
> {code}
> Error message: File Merge Stage after Maps completion is considering input as
> "input format: org.apache.hadoop.mapred.TextInputFormat" instead of
> "org.apache.hadoop.hive.ql.io.orc.OrcInputFormat"
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18824) ValidWriteIdList config should be defined on tables which has to collect stats after insert
[ https://issues.apache.org/jira/browse/HIVE-18824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18824: Resolution: Fixed Status: Resolved (was: Patch Available) Committed to master. Thanks for the reviews! > ValidWriteIdList config should be defined on tables which has to collect > stats after insert > --- > > Key: HIVE-18824 > URL: https://issues.apache.org/jira/browse/HIVE-18824 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sergey Shelukhin >Priority: Major > Labels: ACID, isolation > Fix For: 3.0.0 > > Attachments: HIVE-18824.01.patch, HIVE-18824.patch > > > In HIVE-18192 , per table write ID was introduced where snapshot isolation is > built using ValidWriteIdList on tables which are read with in a txn. > ReadEntity list is referred to decide which table is read within a txn. > For insert operation, table will be found only in WriteEntity, but the table > is read to collect stats. > So, it is needed to build the ValidWriteIdList for tables/partition part of > WriteEntity as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18202) Automatically migrate hbase.table.name to hbase.mapreduce.hfileoutputformat.table.name for hbase-based table
[
https://issues.apache.org/jira/browse/HIVE-18202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu reassigned HIVE-18202:
---
Assignee: (was: Aihua Xu)
> Automatically migrate hbase.table.name to
> hbase.mapreduce.hfileoutputformat.table.name for hbase-based table
>
>
> Key: HIVE-18202
> URL: https://issues.apache.org/jira/browse/HIVE-18202
> Project: Hive
> Issue Type: Sub-task
> Components: HBase Handler
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18202.1.patch, HIVE-18202.patch.addendum
>
>
> The property name for Hbase table mapping is changed from hbase.table.name to
> hbase.mapreduce.hfileoutputformat.table.name in HBase 2.
> We can include such upgrade for existing hbase-based tables in DB upgrade
> script to automatically change such values.
> For the new tables, the query will be like:
> create table hbase_table(key int, val string) stored by
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties
> ('hbase.columns.mapping' = ':key,cf:val') tblproperties
> ('hbase.mapreduce.hfileoutputformat.table.name' =
> 'positive_hbase_handler_bulk')
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18366) Update HBaseSerDe to use hbase.mapreduce.hfileoutputformat.table.name instead of hbase.table.name as the table name property
[
https://issues.apache.org/jira/browse/HIVE-18366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu reassigned HIVE-18366:
---
Assignee: (was: Aihua Xu)
> Update HBaseSerDe to use hbase.mapreduce.hfileoutputformat.table.name instead
> of hbase.table.name as the table name property
>
>
> Key: HIVE-18366
> URL: https://issues.apache.org/jira/browse/HIVE-18366
> Project: Hive
> Issue Type: Sub-task
> Components: HBase Handler
>Affects Versions: 3.0.0
>Reporter: Aihua Xu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18366.1.patch, HIVE-18366.2.patch
>
>
> HBase 2.0 changes the table name property to
> hbase.mapreduce.hfileoutputformat.table.name. HiveHFileOutputFormat is using
> the new property name while HiveHBaseTableOutputFormat is not. If we create
> the table as follows, HiveHBaseTableOutputFormat is used which still uses the
> old property hbase.table.name.
> {noformat}
> create table hbase_table2(key int, val string) stored by
> 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties
> ('hbase.columns.mapping' = ':key,cf:val') tblproperties
> ('hbase.mapreduce.hfileoutputformat.table.name' =
> 'positive_hbase_handler_bulk')
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18811) Fix desc table, column comments are not displayed
[
https://issues.apache.org/jira/browse/HIVE-18811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384031#comment-16384031
]
Vineet Garg commented on HIVE-18811:
[~tartarus] Unfortunately the link to test report is invalid now but
{{escape_comments}} failure looked to be due to your patch.
bq. How can I provide test cases for this patch?
You can add the reproducer you provided to escape_comments.q and generate an
updated escape_comments.q.out. You can find more details at [Wiki
page|https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ#HiveDeveloperFAQ-Testing]
> Fix desc table, column comments are not displayed
> -
>
> Key: HIVE-18811
> URL: https://issues.apache.org/jira/browse/HIVE-18811
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 1.2.1, 2.3.2
> Environment: CentOS 6.5
> Hive-1.2.1
> Hive-3.0.0
>Reporter: tartarus
>Assignee: tartarus
>Priority: Major
> Labels: patch
> Fix For: 3.0.0
>
> Attachments: HIVE_18811.patch
>
>
> when column comment contain \t
> eg: CREATE TABLE `zhangmang_test`(`name` string COMMENT
> 'name{color:#ff}\t{color}zm');
> then execute : {color:#ff}desc zhangmang_test {color}
> {color:#ff}{color:#33}return :{color} name string
> name{color}
> because \t is the separator, so we should translate it
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18824) ValidWriteIdList config should be defined on tables which has to collect stats after insert
[ https://issues.apache.org/jira/browse/HIVE-18824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18824: Summary: ValidWriteIdList config should be defined on tables which has to collect stats after insert (was: ValidWriteIdList config should be defined on tables which has to collect stats after insert.) > ValidWriteIdList config should be defined on tables which has to collect > stats after insert > --- > > Key: HIVE-18824 > URL: https://issues.apache.org/jira/browse/HIVE-18824 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sergey Shelukhin >Priority: Major > Labels: ACID, isolation > Fix For: 3.0.0 > > Attachments: HIVE-18824.01.patch, HIVE-18824.patch > > > In HIVE-18192 , per table write ID was introduced where snapshot isolation is > built using ValidWriteIdList on tables which are read with in a txn. > ReadEntity list is referred to decide which table is read within a txn. > For insert operation, table will be found only in WriteEntity, but the table > is read to collect stats. > So, it is needed to build the ValidWriteIdList for tables/partition part of > WriteEntity as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384008#comment-16384008
]
Sergey Shelukhin commented on HIVE-18848:
-
Less than ten, the value must be...
> Improve readability of filter conditions in explain plan when CBO is run
>
>
> Key: HIVE-18848
> URL: https://issues.apache.org/jira/browse/HIVE-18848
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18848.01.patch, HIVE-18848.02.patch
>
>
> CBO might return comparison operands in any non-deterministic order. Try to
> show {{ }} when possible, i.e., {{c < 10}} rather
> than {{10 > c}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384008#comment-16384008
]
Sergey Shelukhin edited comment on HIVE-18848 at 3/2/18 7:15 PM:
-
Be less than ten, the value must...
was (Author: sershe):
Less than ten, the value must be...
> Improve readability of filter conditions in explain plan when CBO is run
>
>
> Key: HIVE-18848
> URL: https://issues.apache.org/jira/browse/HIVE-18848
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18848.01.patch, HIVE-18848.02.patch
>
>
> CBO might return comparison operands in any non-deterministic order. Try to
> show {{ }} when possible, i.e., {{c < 10}} rather
> than {{10 > c}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18750) Exchange partition should be disabled on ACID/Insert-only tables with per table write ID.
[ https://issues.apache.org/jira/browse/HIVE-18750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384005#comment-16384005 ] ASF GitHub Bot commented on HIVE-18750: --- Github user sankarh closed the pull request at: https://github.com/apache/hive/pull/314 > Exchange partition should be disabled on ACID/Insert-only tables with per > table write ID. > - > > Key: HIVE-18750 > URL: https://issues.apache.org/jira/browse/HIVE-18750 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, DDL, TODOC3.0, pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-18750.01.patch, HIVE-18750.02.patch > > > Per table write id implementation (HIVE-18192) have introduced write ID per > table and used write ID to name the delta/base files and also as primary key > for each row. > Now, exchange partition have to move delta/base files across tables without > changing the write ID which causes incorrect results. > Also, this exchange partition feature is there to support the use-case of > atomic updates. But with ACID updates, we shall support atomic-updates and > hence it makes sense to not support exchange partition for ACID and MM tables. > The qtest file mm_exchangepartition.q test results to be updated after this > change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18738) LLAP IO ACID - includes handling is broken
[
https://issues.apache.org/jira/browse/HIVE-18738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383994#comment-16383994
]
Hive QA commented on HIVE-18738:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
42s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} llap-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} llap-server: The patch generated 29 new + 291
unchanged - 23 fixed = 320 total (was 314) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 8 new + 802 unchanged - 10
fixed = 810 total (was 812) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9443/dev-support/hive-personality.sh
|
| git revision | master / fc8a45b |
| Default Java | 1.8.0_111 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9443/yetus/patch-mvninstall-llap-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9443/yetus/diff-checkstyle-llap-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9443/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9443/yetus/whitespace-eol.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9443/yetus/patch-asflicense-problems.txt
|
| modules | C: storage-api itests llap-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9443/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP IO ACID - includes handling is broken
> --
>
> Key: HIVE-18738
> URL: https://issues.apache.org/jira/browse/HIVE-18738
> Project: Hive
> Issue Type: Bug
>Reporter: Deepesh Khandelwal
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18738.01.patch, HIVE-18738.02.patch,
> HIVE-18738.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18825) Define ValidTxnList before starting query optimization
[
https://issues.apache.org/jira/browse/HIVE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383978#comment-16383978
]
Jesus Camacho Rodriguez commented on HIVE-18825:
The following test failures seem related:
{code}
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking11
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testDynamicPartitionInsert
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.updateSelectUpdate
{code}
Taking a look right now.
> Define ValidTxnList before starting query optimization
> --
>
> Key: HIVE-18825
> URL: https://issues.apache.org/jira/browse/HIVE-18825
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-18825.01.patch, HIVE-18825.02.patch,
> HIVE-18825.patch
>
>
> Consider a set of tables used by a materialized view where inserts happened
> after the materialization was created. To compute incremental view
> maintenance, we need to be able to filter only new rows from those base
> tables. That can be done by inserting a filter operator with condition e.g.
> {{ROW\_\_ID.transactionId < highwatermark and ROW\_\_ID.transactionId NOT
> IN()}} on top of the MVs query definition and triggering the
> rewriting (which should in turn produce a partial rewriting). However, to do
> that, we need to have a value for {{ValidTxnList}} during query compilation
> so we know the snapshot that we are querying.
> This patch aims to generate {{ValidTxnList}} before query optimization. There
> should not be any visible changes for end user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18797) ExprConstNodeDesc's getExprString should put appropriate qualifier with literals
[
https://issues.apache.org/jira/browse/HIVE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18797:
---
Fix Version/s: 3.0.0
> ExprConstNodeDesc's getExprString should put appropriate qualifier with
> literals
>
>
> Key: HIVE-18797
> URL: https://issues.apache.org/jira/browse/HIVE-18797
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18797.1.patch, HIVE-18797.2.patch,
> HIVE-18797.3.patch, HIVE-18797.4.patch, HIVE-18797.5.patch, HIVE-18797.6.patch
>
>
> e.g. instead of just {{2016-01-23 00:00:00.0)}} it should be
> {{TIMESTAMP'2016-01-23 00:00:00.0)'}}.
> This method is used by explain task and also HIVE-18726 needs it to reparse
> the string for default value.
> I am creating a separate ticket for this instead of lumping it with
> HIVE-18726 because this will need quite a few golden files update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18797) ExprConstNodeDesc's getExprString should put appropriate qualifier with literals
[
https://issues.apache.org/jira/browse/HIVE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383966#comment-16383966
]
Vineet Garg commented on HIVE-18797:
Pushed to master.
> ExprConstNodeDesc's getExprString should put appropriate qualifier with
> literals
>
>
> Key: HIVE-18797
> URL: https://issues.apache.org/jira/browse/HIVE-18797
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18797.1.patch, HIVE-18797.2.patch,
> HIVE-18797.3.patch, HIVE-18797.4.patch, HIVE-18797.5.patch, HIVE-18797.6.patch
>
>
> e.g. instead of just {{2016-01-23 00:00:00.0)}} it should be
> {{TIMESTAMP'2016-01-23 00:00:00.0)'}}.
> This method is used by explain task and also HIVE-18726 needs it to reparse
> the string for default value.
> I am creating a separate ticket for this instead of lumping it with
> HIVE-18726 because this will need quite a few golden files update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18264) CachedStore: Store cached partitions/col stats within the table cache and make prewarm non-blocking
[ https://issues.apache.org/jira/browse/HIVE-18264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383961#comment-16383961 ] Alexander Kolbasov commented on HIVE-18264: --- I have some higher-level concerns with the pre-warn mechanism. HMS data can be pretty big and pre-warm keeps it all in memory whether it is needed or not. Essentially now we have in-memory HMS which uses RDBMS for persistency. I can understand the use of pre-warm in the case where a table is accessed - it is likely that its partition data will be accessed as well. But doing this willy-nilly for all objects seems dangerous in terms of potential memory consumption impact. [~pvary] [~alex.behm] What do you think? > CachedStore: Store cached partitions/col stats within the table cache and > make prewarm non-blocking > --- > > Key: HIVE-18264 > URL: https://issues.apache.org/jira/browse/HIVE-18264 > Project: Hive > Issue Type: Bug >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > Attachments: HIVE-18264.1.patch, HIVE-18264.2.patch, > HIVE-18264.3.patch, HIVE-18264.4.patch, HIVE-18264.5.patch > > > Currently we have a separate cache for partitions and partition col stats > which results in some calls iterating through each of these for > retrieving/updating. We can get better performance by organizing > hierarchically. We should also make prewarm non-blocking -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18797) ExprConstNodeDesc's getExprString should put appropriate qualifier with literals
[
https://issues.apache.org/jira/browse/HIVE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18797:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> ExprConstNodeDesc's getExprString should put appropriate qualifier with
> literals
>
>
> Key: HIVE-18797
> URL: https://issues.apache.org/jira/browse/HIVE-18797
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18797.1.patch, HIVE-18797.2.patch,
> HIVE-18797.3.patch, HIVE-18797.4.patch, HIVE-18797.5.patch, HIVE-18797.6.patch
>
>
> e.g. instead of just {{2016-01-23 00:00:00.0)}} it should be
> {{TIMESTAMP'2016-01-23 00:00:00.0)'}}.
> This method is used by explain task and also HIVE-18726 needs it to reparse
> the string for default value.
> I am creating a separate ticket for this instead of lumping it with
> HIVE-18726 because this will need quite a few golden files update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18797) ExprConstNodeDesc's getExprString should put appropriate qualifier with literals
[
https://issues.apache.org/jira/browse/HIVE-18797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383954#comment-16383954
]
Vineet Garg commented on HIVE-18797:
Test failure are unrelated
> ExprConstNodeDesc's getExprString should put appropriate qualifier with
> literals
>
>
> Key: HIVE-18797
> URL: https://issues.apache.org/jira/browse/HIVE-18797
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18797.1.patch, HIVE-18797.2.patch,
> HIVE-18797.3.patch, HIVE-18797.4.patch, HIVE-18797.5.patch, HIVE-18797.6.patch
>
>
> e.g. instead of just {{2016-01-23 00:00:00.0)}} it should be
> {{TIMESTAMP'2016-01-23 00:00:00.0)'}}.
> This method is used by explain task and also HIVE-18726 needs it to reparse
> the string for default value.
> I am creating a separate ticket for this instead of lumping it with
> HIVE-18726 because this will need quite a few golden files update.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18825) Define ValidTxnList before starting query optimization
[
https://issues.apache.org/jira/browse/HIVE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383952#comment-16383952
]
Hive QA commented on HIVE-18825:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912659/HIVE-18825.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 26 failed/errored test(s), 13024 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=170)
[vector_windowing_expressions.q,tez_union_group_by.q,vector_like_2.q,auto_join1.q,llap_acid.q,sqlmerge.q,tez_dynpart_hashjoin_1.q,schema_evol_orc_acid_part_update_llap_io.q,vector_windowing_gby.q,vectorized_timestamp.q,cbo_subq_exists.q,lateral_view.q,schema_evol_orc_vec_table_llap_io.q,optimize_nullscan.q,vectorization_decimal_date.q,schema_evol_orc_nonvec_table_llap_io.q,udaf_all_keyword.q,tez_self_join.q,vector_partitioned_date_time.q,acid_vectorization_original.q,tez_fsstat.q,vector_mapjoin_reduce.q,join_acid_non_acid.q,empty_join.q,vector_groupby_grouping_window.q,auto_join21.q,tez_input_counters.q,schema_evol_orc_nonvec_part_all_complex_llap_io.q,orc_ppd_timestamp.q,vector_decimal_1.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Updated] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18848:
---
Attachment: HIVE-18848.02.patch
> Improve readability of filter conditions in explain plan when CBO is run
>
>
> Key: HIVE-18848
> URL: https://issues.apache.org/jira/browse/HIVE-18848
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18848.01.patch, HIVE-18848.02.patch
>
>
> CBO might return comparison operands in any non-deterministic order. Try to
> show {{ }} when possible, i.e., {{c < 10}} rather
> than {{10 > c}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18848) Improve readability of filter conditions in explain plan when CBO is run
[
https://issues.apache.org/jira/browse/HIVE-18848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383949#comment-16383949
]
Jesus Camacho Rodriguez commented on HIVE-18848:
Thanks [~gopalv], I will apply the change. I expect a 'few' q file changes :)
> Improve readability of filter conditions in explain plan when CBO is run
>
>
> Key: HIVE-18848
> URL: https://issues.apache.org/jira/browse/HIVE-18848
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Minor
> Attachments: HIVE-18848.01.patch
>
>
> CBO might return comparison operands in any non-deterministic order. Try to
> show {{ }} when possible, i.e., {{c < 10}} rather
> than {{10 > c}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18826) fix TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
[
https://issues.apache.org/jira/browse/HIVE-18826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-18826:
---
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master, one failure less, thanks for reviewing [~kgyrtkirk].
> fix TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
> -
>
> Key: HIVE-18826
> URL: https://issues.apache.org/jira/browse/HIVE-18826
> Project: Hive
> Issue Type: Bug
> Components: Tests
>Reporter: Zoltan Haindrich
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18826.patch
>
>
> It seems like this test have started failing after HIVE-18386.
> removing a small part of the patch fixes this test:
> {code}
> git show 80e6f7b0f13c134642763b0a692c3abbf3ffe106
> ./itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java | patch
> -p0 -R
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18716) Delete unnecessary parameters from TaskFactory
[
https://issues.apache.org/jira/browse/HIVE-18716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383883#comment-16383883
]
Zoltan Haindrich commented on HIVE-18716:
-
I think so...I'm not sure if anyone is relying on this or not...
> Delete unnecessary parameters from TaskFactory
> --
>
> Key: HIVE-18716
> URL: https://issues.apache.org/jira/browse/HIVE-18716
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Gergely Hajós
>Assignee: Gergely Hajós
>Priority: Trivial
> Attachments: HIVE-18716.1.patch, HIVE-18716.2.patch,
> HIVE-18716.3.patch
>
>
> * In _TaskFactory class conf_ parameter is not used here
> {code:java}
> public static Task get(Class workClass,
> HiveConf conf)
> {code}
> * _get_ functions never called with vararg parameter _Task Serializable>... tasklist_
> * the following version of _get_ is always called with _setConf_ true from
> outside of _TaskFactory_ class
> {code:java}
> public static Task get(T work, HiveConf conf,
> boolean setConf, Task... tasklist)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18716) Delete unnecessary parameters from TaskFactory
[
https://issues.apache.org/jira/browse/HIVE-18716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383865#comment-16383865
]
Gergely Hajós commented on HIVE-18716:
--
[~kgyrtkirk] I'm sorry! Now I completely understand your comment. May I change
back the constructor to two parameters in case of
HiveAuthorizationTaskFactoryImpl?
> Delete unnecessary parameters from TaskFactory
> --
>
> Key: HIVE-18716
> URL: https://issues.apache.org/jira/browse/HIVE-18716
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0
>Reporter: Gergely Hajós
>Assignee: Gergely Hajós
>Priority: Trivial
> Attachments: HIVE-18716.1.patch, HIVE-18716.2.patch,
> HIVE-18716.3.patch
>
>
> * In _TaskFactory class conf_ parameter is not used here
> {code:java}
> public static Task get(Class workClass,
> HiveConf conf)
> {code}
> * _get_ functions never called with vararg parameter _Task Serializable>... tasklist_
> * the following version of _get_ is always called with _setConf_ true from
> outside of _TaskFactory_ class
> {code:java}
> public static Task get(T work, HiveConf conf,
> boolean setConf, Task... tasklist)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18825) Define ValidTxnList before starting query optimization
[
https://issues.apache.org/jira/browse/HIVE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383863#comment-16383863
]
Hive QA commented on HIVE-18825:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
27s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-9442/dev-support/hive-personality.sh
|
| git revision | master / 61e21d6 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9442/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9442/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Define ValidTxnList before starting query optimization
> --
>
> Key: HIVE-18825
> URL: https://issues.apache.org/jira/browse/HIVE-18825
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-18825.01.patch, HIVE-18825.02.patch,
> HIVE-18825.patch
>
>
> Consider a set of tables used by a materialized view where inserts happened
> after the materialization was created. To compute incremental view
> maintenance, we need to be able to filter only new rows from those base
> tables. That can be done by inserting a filter operator with condition e.g.
> {{ROW\_\_ID.transactionId < highwatermark and ROW\_\_ID.transactionId NOT
> IN()}} on top of the MVs query definition and triggering the
> rewriting (which should in turn produce a partial rewriting). However, to do
> that, we need to have a value for {{ValidTxnList}} during query compilation
> so we know the snapshot that we are querying.
> This patch aims to generate {{ValidTxnList}} before query optimization. There
> should not be any visible changes for end user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17552) Enable bucket map join by default
[
https://issues.apache.org/jira/browse/HIVE-17552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383830#comment-16383830
]
Hive QA commented on HIVE-17552:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12912654/HIVE-17552.8.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 13043 tests
executed
*Failed tests:*
{noformat}
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=91)
[infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q,bucket_num_reducers_acid2.q]
TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=93)
[jira] [Updated] (HIVE-18817) ArrayIndexOutOfBounds exception during read of ACID table.
[
https://issues.apache.org/jira/browse/HIVE-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-18817:
--
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
committed to master
thanks Jason for the review
> ArrayIndexOutOfBounds exception during read of ACID table.
> --
>
> Key: HIVE-18817
> URL: https://issues.apache.org/jira/browse/HIVE-18817
> Project: Hive
> Issue Type: Bug
> Components: ORC, Transactions
>Reporter: Jason Dere
>Assignee: Eugene Koifman
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18817.01.patch, HIVE-18817.02.patch,
> HIVE-18817.03.patch, HIVE-18817.04.patch, repro.patch
>
>
> Seeing some users hitting the following stack trace:
> {noformat}
> 2018-02-26 05:49:45,876 [ERROR] [TezChild] |tez.TezProcessor|:
> java.lang.RuntimeException: java.io.IOException:
> java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:196)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:142)
> at
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:113)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:66)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:325)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:150)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:139)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:347)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:194)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:185)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:185)
> at
> org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:181)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> at
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:258)
> at
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:193)
> ... 19 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 7
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.discoverKeyBounds(OrcRawRecordMerger.java:388)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:457)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1456)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1342)
> at
> org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:255)
> ... 20 more
> {noformat}
> Have a JUnit test that appears to produce a similar stack trace - looks like
> this occurs if there is an OrcSplit of an ACID table where the split offset
> is beyond the starting offset of the last stripe in the ORC file.
> cc [~ekoifman]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
