[jira] [Updated] (HIVE-22235) CommandProcessorResponse should not be an exception
[ https://issues.apache.org/jira/browse/HIVE-22235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22235: -- Status: Patch Available (was: Open) > CommandProcessorResponse should not be an exception > --- > > Key: HIVE-22235 > URL: https://issues.apache.org/jira/browse/HIVE-22235 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22235.01.patch, HIVE-22235.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The CommandProcessorResponse class extends Exception. This may be convenient, > but it is wrong, as a response is not an exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22235) CommandProcessorResponse should not be an exception
[ https://issues.apache.org/jira/browse/HIVE-22235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22235: -- Status: Open (was: Patch Available) > CommandProcessorResponse should not be an exception > --- > > Key: HIVE-22235 > URL: https://issues.apache.org/jira/browse/HIVE-22235 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22235.01.patch, HIVE-22235.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The CommandProcessorResponse class extends Exception. This may be convenient, > but it is wrong, as a response is not an exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22207) Tez: SplitGenerator throws NumberFormatException when "dfs.block.size" on cluster is "128m"
[
https://issues.apache.org/jira/browse/HIVE-22207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939149#comment-16939149
]
Naresh P R commented on HIVE-22207:
---
[~praneetsharma]
I am also facing this, let me know if you can share a patch for the same or
shall i work on it.
> Tez: SplitGenerator throws NumberFormatException when "dfs.block.size" on
> cluster is "128m"
> ---
>
> Key: HIVE-22207
> URL: https://issues.apache.org/jira/browse/HIVE-22207
> Project: Hive
> Issue Type: Bug
> Components: Hive, Tez
>Affects Versions: 2.1.0, 3.1.0
>Reporter: Praneet Sharma
>Priority: Major
>
> When the value of "dfs.block.size" on the cluster is "128m" in place of
> actual bytes as Long, then Tez job fails with NumberFormatException:
> {code:java}
> java.lang.NumberFormatException: For input string: "128m" at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
> at java.lang.Long.parseLong(Long.java:589) at
> java.lang.Long.parseLong(Long.java:631) at
> org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1539) at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:194)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22235) CommandProcessorResponse should not be an exception
[ https://issues.apache.org/jira/browse/HIVE-22235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22235: -- Attachment: HIVE-22235.02.patch > CommandProcessorResponse should not be an exception > --- > > Key: HIVE-22235 > URL: https://issues.apache.org/jira/browse/HIVE-22235 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22235.01.patch, HIVE-22235.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The CommandProcessorResponse class extends Exception. This may be convenient, > but it is wrong, as a response is not an exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22235) CommandProcessorResponse should not be an exception
[ https://issues.apache.org/jira/browse/HIVE-22235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22235: -- Attachment: (was: HIVE-22235.02.patch) > CommandProcessorResponse should not be an exception > --- > > Key: HIVE-22235 > URL: https://issues.apache.org/jira/browse/HIVE-22235 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22235.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The CommandProcessorResponse class extends Exception. This may be convenient, > but it is wrong, as a response is not an exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22234) Hive replication fails with table already exist error when replicating from old version of hive.
[ https://issues.apache.org/jira/browse/HIVE-22234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22234: --- Status: Open (was: Patch Available) > Hive replication fails with table already exist error when replicating from > old version of hive. > > > Key: HIVE-22234 > URL: https://issues.apache.org/jira/browse/HIVE-22234 > Project: Hive > Issue Type: Bug >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22234.01.patch, HIVE-22234.02.patch, > HIVE-22234.03.patch > > Time Spent: 40m > Remaining Estimate: 0h > > HIve replication from old version where HIVE-22046 is not patched will not > have engine column set in the table column stats. This causes "ERROR: null > value in column "ENGINE" violates not-null constraint" error during create > table while updating the column stats. As the column stats are updated after > the create table txn is committed, the next retry by HMS client throws table > already exist error. Need to update the ENGINE column to default value while > importing the table if the column value is not set. The column stat and > create table in same txn can be done as part of separate Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22234) Hive replication fails with table already exist error when replicating from old version of hive.
[ https://issues.apache.org/jira/browse/HIVE-22234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22234: --- Status: Patch Available (was: Open) > Hive replication fails with table already exist error when replicating from > old version of hive. > > > Key: HIVE-22234 > URL: https://issues.apache.org/jira/browse/HIVE-22234 > Project: Hive > Issue Type: Bug >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22234.01.patch, HIVE-22234.02.patch, > HIVE-22234.03.patch > > Time Spent: 40m > Remaining Estimate: 0h > > HIve replication from old version where HIVE-22046 is not patched will not > have engine column set in the table column stats. This causes "ERROR: null > value in column "ENGINE" violates not-null constraint" error during create > table while updating the column stats. As the column stats are updated after > the create table txn is committed, the next retry by HMS client throws table > already exist error. Need to update the ENGINE column to default value while > importing the table if the column value is not set. The column stat and > create table in same txn can be done as part of separate Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22247) HiveHFileOutputFormat throws FileNotFoundException when partition's task output empty
[
https://issues.apache.org/jira/browse/HIVE-22247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22247:
--
Description:
When partition's task output empty, HiveHFileOutputFormat throws
FileNotFoundException like this:
{code:java}
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: 1 finished. closing...
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: FS[1]: records written - 0
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: Final Path: FS
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: Writing to temp file: FS
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: New Final Path: FS
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
2019-09-24 19:15:55,915 INFO [main]
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output
Committer Algorithm version is 1
2019-09-24 19:15:55,954 INFO [main]
org.apache.hadoop.conf.Configuration.deprecation: hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
2019-09-24 19:15:56,089 ERROR [main] ExecReducer: Hit error while closing
operators - failing tree
2019-09-24 19:15:56,090 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.lang.RuntimeException: Hive Runtime Error while
closing operators: java.io.FileNotFoundException: File
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
does not exist.
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1923)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.io.FileNotFoundException: File
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
does not exist.
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:200)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1016)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:278)
... 7 more
Caused by: java.io.FileNotFoundException: File
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
does not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:880)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:109)
at
org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:938)
at
org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:934)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:945)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1592)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1632)
at
org.apache.hadoop.hive.hbase.HiveHFileOutputFormat$1.close(HiveHFileOutputFormat.java:153)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:197)
... 11 more
2019-09-24 19:15:56,093 INFO [main] org.apache.hadoop.mapred.Task: Runnning
cleanup for the task
{code}
I think we should skip it if srcDir do not exist, fix like this:
{code:java}
@Override
public void close(boolean abort) throws IOException {
try {
...
FileStatus [] files = null;
for (;;) {
try {
files = fs.listStatus(srcDir, FileUtils.STAGING_DIR_PATH_FILTER);
} catch (FileNotFoundException fnfe) {
LOG.error(String.format("Output data is empty, please check Task [ %s
]", tac.getTaskAttemptID().toString()), fnfe);
break;
}
}
if (files != null ) {
for
[jira] [Commented] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[
https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939083#comment-16939083
]
Hive QA commented on HIVE-22042:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981461/HIVE-22042.4.patch
{color:green}SUCCESS:{color} +1 due to 296 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17009 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18757/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18757/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18757/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981461 - PreCommit-HIVE-Build
> Set hive.exec.dynamic.partition.mode=nonstrict by default
> -
>
> Key: HIVE-22042
> URL: https://issues.apache.org/jira/browse/HIVE-22042
> Project: Hive
> Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: John Sherman
>Priority: Minor
> Attachments: HIVE-22042.1.patch, HIVE-22042.2.patch,
> HIVE-22042.3.patch, HIVE-22042.4.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[
https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939074#comment-16939074
]
Hive QA commented on HIVE-22042:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 1566 extant Findbugs warnings.
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
38s{color} | {color:red} branch/hcatalog no findbugs output file
(hcatalog/target/findbugsXml.xml) {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
47s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} The patch hcatalog passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch streaming passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hive-blobstore passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 533
unchanged - 21 fixed = 533 total (was 554) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
0s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
36s{color} | {color:red} patch/hcatalog no findbugs output file
(hcatalog/target/findbugsXml.xml) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5
[jira] [Updated] (HIVE-22234) Hive replication fails with table already exist error when replicating from old version of hive.
[ https://issues.apache.org/jira/browse/HIVE-22234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22234: --- Status: Patch Available (was: Open) > Hive replication fails with table already exist error when replicating from > old version of hive. > > > Key: HIVE-22234 > URL: https://issues.apache.org/jira/browse/HIVE-22234 > Project: Hive > Issue Type: Bug >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22234.01.patch, HIVE-22234.02.patch, > HIVE-22234.03.patch > > Time Spent: 40m > Remaining Estimate: 0h > > HIve replication from old version where HIVE-22046 is not patched will not > have engine column set in the table column stats. This causes "ERROR: null > value in column "ENGINE" violates not-null constraint" error during create > table while updating the column stats. As the column stats are updated after > the create table txn is committed, the next retry by HMS client throws table > already exist error. Need to update the ENGINE column to default value while > importing the table if the column value is not set. The column stat and > create table in same txn can be done as part of separate Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22234) Hive replication fails with table already exist error when replicating from old version of hive.
[ https://issues.apache.org/jira/browse/HIVE-22234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22234: --- Status: Open (was: Patch Available) > Hive replication fails with table already exist error when replicating from > old version of hive. > > > Key: HIVE-22234 > URL: https://issues.apache.org/jira/browse/HIVE-22234 > Project: Hive > Issue Type: Bug >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22234.01.patch, HIVE-22234.02.patch, > HIVE-22234.03.patch > > Time Spent: 40m > Remaining Estimate: 0h > > HIve replication from old version where HIVE-22046 is not patched will not > have engine column set in the table column stats. This causes "ERROR: null > value in column "ENGINE" violates not-null constraint" error during create > table while updating the column stats. As the column stats are updated after > the create table txn is committed, the next retry by HMS client throws table > already exist error. Need to update the ENGINE column to default value while > importing the table if the column value is not set. The column stat and > create table in same txn can be done as part of separate Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22234) Hive replication fails with table already exist error when replicating from old version of hive.
[ https://issues.apache.org/jira/browse/HIVE-22234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22234: --- Attachment: HIVE-22234.03.patch > Hive replication fails with table already exist error when replicating from > old version of hive. > > > Key: HIVE-22234 > URL: https://issues.apache.org/jira/browse/HIVE-22234 > Project: Hive > Issue Type: Bug >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22234.01.patch, HIVE-22234.02.patch, > HIVE-22234.03.patch > > Time Spent: 40m > Remaining Estimate: 0h > > HIve replication from old version where HIVE-22046 is not patched will not > have engine column set in the table column stats. This causes "ERROR: null > value in column "ENGINE" violates not-null constraint" error during create > table while updating the column stats. As the column stats are updated after > the create table txn is committed, the next retry by HMS client throws table > already exist error. Need to update the ENGINE column to default value while > importing the table if the column value is not set. The column stat and > create table in same txn can be done as part of separate Jira. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939055#comment-16939055
]
Hive QA commented on HIVE-21449:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981460/HIVE-21449.6.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17010 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18756/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18756/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18756/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981460 - PreCommit-HIVE-Build
> implement 'WITHIN GROUP' clause
> ---
>
> Key: HIVE-21449
> URL: https://issues.apache.org/jira/browse/HIVE-21449
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch,
> HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch,
> HIVE-21449.5.patch, HIVE-21449.6.patch, HIVE-21449.6.patch,
> HIVE-21449.6.patch, HIVE-21449.6.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22247) HiveHFileOutputFormat throws FileNotFoundException when partition's task output empty
[
https://issues.apache.org/jira/browse/HIVE-22247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22247:
--
Component/s: HBase Handler
> HiveHFileOutputFormat throws FileNotFoundException when partition's task
> output empty
> -
>
> Key: HIVE-22247
> URL: https://issues.apache.org/jira/browse/HIVE-22247
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler
>Affects Versions: 2.2.0, 3.0.0
>Reporter: xiepengjie
>Assignee: xiepengjie
>Priority: Major
>
> When partition's task output empty, HiveHFileOutputFormat throws
> FileNotFoundException like this:
> {code:java}
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: 1 finished. closing...
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: FS[1]: records written - 0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: Final Path: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: Writing to temp file: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: New Final Path: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
> 2019-09-24 19:15:55,915 INFO [main]
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output
> Committer Algorithm version is 1
> 2019-09-24 19:15:55,954 INFO [main]
> org.apache.hadoop.conf.Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 2019-09-24 19:15:56,089 ERROR [main] ExecReducer: Hit error while closing
> operators - failing tree
> 2019-09-24 19:15:56,090 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators: java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
> at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1923)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:200)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1016)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:278)
> ... 7 more
> Caused by: java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:880)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:109)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:938)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:934)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:945)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1592)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1632)
> at
> org.apache.hadoop.hive.hbase.HiveHFileOutputFormat$1.close(HiveHFileOutputFormat.java:153)
> at
>
[jira] [Updated] (HIVE-22247) HiveHFileOutputFormat throws FileNotFoundException when partition's task output empty
[
https://issues.apache.org/jira/browse/HIVE-22247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie updated HIVE-22247:
--
Description:
When partition's task output empty, HiveHFileOutputFormat throws
FileNotFoundException like this:
{code:java}
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: 1 finished. closing...
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: FS[1]: records written - 0
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: Final Path: FS
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: Writing to temp file: FS
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
2019-09-24 19:15:55,886 INFO [main]
org.apache.hadoop.hive.ql.exec.FileSinkOperator: New Final Path: FS
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
2019-09-24 19:15:55,915 INFO [main]
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output
Committer Algorithm version is 1
2019-09-24 19:15:55,954 INFO [main]
org.apache.hadoop.conf.Configuration.deprecation: hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
2019-09-24 19:15:56,089 ERROR [main] ExecReducer: Hit error while closing
operators - failing tree
2019-09-24 19:15:56,090 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : java.lang.RuntimeException: Hive Runtime Error while
closing operators: java.io.FileNotFoundException: File
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
does not exist.
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1923)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.io.FileNotFoundException: File
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
does not exist.
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:200)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1016)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:278)
... 7 more
Caused by: java.io.FileNotFoundException: File
hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
does not exist.
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:880)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:109)
at
org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:938)
at
org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:934)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:945)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1592)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1632)
at
org.apache.hadoop.hive.hbase.HiveHFileOutputFormat$1.close(HiveHFileOutputFormat.java:153)
at
org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:197)
... 11 more
2019-09-24 19:15:56,093 INFO [main] org.apache.hadoop.mapred.Task: Runnning
cleanup for the task
{code}
I think we should skip it if srcDir do not exist, fix like this:
{code:java}
@Override
public void close(boolean abort) throws IOException {
try {
...
FileStatus [] files = null;
for (;;) {
try {
files = fs.listStatus(srcDir, FileUtils.STAGING_DIR_PATH_FILTER);
} catch (FileNotFoundException fnfe) {
LOG.error(String.format("Output data is empty, please check Task [ %s
]", tac.getTaskAttemptID().toString()), fnfe);
break;
}
...
} catch (InterruptedException ex)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939042#comment-16939042
]
Hive QA commented on HIVE-21449:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 1566 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18756/dev-support/hive-personality.sh
|
| git revision | master / 4c8b374 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18756/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> implement 'WITHIN GROUP' clause
> ---
>
> Key: HIVE-21449
> URL: https://issues.apache.org/jira/browse/HIVE-21449
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch,
> HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch,
> HIVE-21449.5.patch, HIVE-21449.6.patch, HIVE-21449.6.patch,
> HIVE-21449.6.patch, HIVE-21449.6.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22137) Implement alter/rename partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939035#comment-16939035
]
Hive QA commented on HIVE-22137:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981452/HIVE-22137.02.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17147 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_rewrite_ssb]
(batchId=171)
org.apache.hive.jdbc.TestNoSaslAuth.testConnection (batchId=286)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18755/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18755/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18755/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981452 - PreCommit-HIVE-Build
> Implement alter/rename partition related methods on temporary tables
>
>
> Key: HIVE-22137
> URL: https://issues.apache.org/jira/browse/HIVE-22137
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22137.01.patch, HIVE-22137.02.patch
>
>
> IMetaStoreClient exposes the following methods related to altering of
> partitions:
> {code:java}
> void alter_partition(String dbName, String tblName, Partition newPart);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart);
> void alter_partition(String dbName, String tblName, Partition newPart,
> EnvironmentContext environmentContext);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext, String writeIdList);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext,String writeIdList, long
> writeId);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts, EnvironmentContext environmentContext, String
> writeIdList, long writeId);
> void renamePartition(final String dbname, final String tableName, final
> List part_vals, final Partition newPart);
> void renamePartition(String catName, String dbname, String tableName,
> List part_vals, Partition newPart, String validWriteIds){code}
> These should be implemented, in order to completely support partition on
> temporary tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21924) Split text files even if header/footer exists
[
https://issues.apache.org/jira/browse/HIVE-21924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-21924:
--
Labels: pull-request-available (was: )
> Split text files even if header/footer exists
> -
>
> Key: HIVE-21924
> URL: https://issues.apache.org/jira/browse/HIVE-21924
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 2.4.0, 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21924.patch
>
>
> https://github.com/apache/hive/blob/967a1cc98beede8e6568ce750ebeb6e0d048b8ea/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L494-L503
>
> {code}
> int headerCount = 0;
> int footerCount = 0;
> if (table != null) {
> headerCount = Utilities.getHeaderCount(table);
> footerCount = Utilities.getFooterCount(table, conf);
> if (headerCount != 0 || footerCount != 0) {
> // Input file has header or footer, cannot be splitted.
> HiveConf.setLongVar(conf, ConfVars.MAPREDMINSPLITSIZE,
> Long.MAX_VALUE);
> }
> }
> {code}
> this piece of code makes the CSV (or any text files with header/footer) files
> not splittable if header or footer is present.
> If only header is present, we can find the offset after first line break and
> use that to split. Similarly for footer, may be read few KB's of data at the
> end and find the last line break offset. Use that to determine the data range
> which can be used for splitting. Few reads during split generation are
> cheaper than not splitting the file at all.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21924) Split text files even if header/footer exists
[
https://issues.apache.org/jira/browse/HIVE-21924?focusedWorklogId=319283=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-319283
]
ASF GitHub Bot logged work on HIVE-21924:
-
Author: ASF GitHub Bot
Created on: 26/Sep/19 23:52
Start Date: 26/Sep/19 23:52
Worklog Time Spent: 10m
Work Description: mustafaiman commented on pull request #791: HIVE-21924
URL: https://github.com/apache/hive/pull/791
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 319283)
Remaining Estimate: 0h
Time Spent: 10m
> Split text files even if header/footer exists
> -
>
> Key: HIVE-21924
> URL: https://issues.apache.org/jira/browse/HIVE-21924
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 2.4.0, 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21924.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> https://github.com/apache/hive/blob/967a1cc98beede8e6568ce750ebeb6e0d048b8ea/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L494-L503
>
> {code}
> int headerCount = 0;
> int footerCount = 0;
> if (table != null) {
> headerCount = Utilities.getHeaderCount(table);
> footerCount = Utilities.getFooterCount(table, conf);
> if (headerCount != 0 || footerCount != 0) {
> // Input file has header or footer, cannot be splitted.
> HiveConf.setLongVar(conf, ConfVars.MAPREDMINSPLITSIZE,
> Long.MAX_VALUE);
> }
> }
> {code}
> this piece of code makes the CSV (or any text files with header/footer) files
> not splittable if header or footer is present.
> If only header is present, we can find the offset after first line break and
> use that to split. Similarly for footer, may be read few KB's of data at the
> end and find the last line break offset. Use that to determine the data range
> which can be used for splitting. Few reads during split generation are
> cheaper than not splitting the file at all.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21924) Split text files even if header/footer exists
[
https://issues.apache.org/jira/browse/HIVE-21924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mustafa Iman updated HIVE-21924:
Attachment: HIVE-21924.patch
Status: Patch Available (was: Open)
> Split text files even if header/footer exists
> -
>
> Key: HIVE-21924
> URL: https://issues.apache.org/jira/browse/HIVE-21924
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 2.4.0, 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Mustafa Iman
>Priority: Major
> Attachments: HIVE-21924.patch
>
>
> https://github.com/apache/hive/blob/967a1cc98beede8e6568ce750ebeb6e0d048b8ea/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L494-L503
>
> {code}
> int headerCount = 0;
> int footerCount = 0;
> if (table != null) {
> headerCount = Utilities.getHeaderCount(table);
> footerCount = Utilities.getFooterCount(table, conf);
> if (headerCount != 0 || footerCount != 0) {
> // Input file has header or footer, cannot be splitted.
> HiveConf.setLongVar(conf, ConfVars.MAPREDMINSPLITSIZE,
> Long.MAX_VALUE);
> }
> }
> {code}
> this piece of code makes the CSV (or any text files with header/footer) files
> not splittable if header or footer is present.
> If only header is present, we can find the offset after first line break and
> use that to split. Similarly for footer, may be read few KB's of data at the
> end and find the last line break offset. Use that to determine the data range
> which can be used for splitting. Few reads during split generation are
> cheaper than not splitting the file at all.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-19746) Hadoop credential provider allows to read passwords for every user
[
https://issues.apache.org/jira/browse/HIVE-19746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939030#comment-16939030
]
Thejas Nair edited comment on HIVE-19746 at 9/26/19 11:32 PM:
--
The jceks file can be stored in local filesystem with only permissions for the
hive service user, to avoid this issue.
was (Author: thejas):
The jceks file can be stored in local filesystem, to avoid this issue.
> Hadoop credential provider allows to read passwords for every user
> --
>
> Key: HIVE-19746
> URL: https://issues.apache.org/jira/browse/HIVE-19746
> Project: Hive
> Issue Type: Bug
> Components: Security
>Reporter: Oleksiy Sayankin
>Assignee: Oleksiy Sayankin
>Priority: Blocker
>
> This simple program allows to read any password from any {{jceks}} file:
> {code}
> package com.test.app;
> import java.util.List;
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.security.alias.CredentialProvider;
> import org.apache.hadoop.security.alias.CredentialProviderFactory;
> import java.io.IOException;
> public class PasswordReader {
> public static void main(String[] args) throws IOException {
> if (args == null || args.length == 0){
> throw new IllegalArgumentException("Credential provider path is to
> set");
> }
> String credentialProviderPath = args[0];
> Configuration configuration = new Configuration();
> configuration.set(CredentialProviderFactory.CREDENTIAL_PROVIDER_PATH,
> credentialProviderPath);
> CredentialProvider credentialProvider =
> CredentialProviderFactory.getProviders(configuration).get(0);
> List aliases = credentialProvider.getAliases();
> for(String alias : aliases){
> System.out.println(alias + " = " + new
> String(configuration.getPassword(alias)));
> }
> }
> }
> {code}
> {code}
> java -cp $(hadoop classpath):password-reader.jar com.test.app.PasswordReader
> jceks://hdfs/user/hive/hivemetastore.jceks
> {code}
> *RESULT*
> {code}
> javax.jdo.option.connectionpassword = 123456
> {code}
> File {{jceks://hdfs/user/hive/hivemetastore.jceks}} has {{\-rw\-r\-\-r\-\-}}
> permissions and {{hdfs:hdfs}} owner:group. We can't remove world readable
> permissions here, because Hive is configured for impersonation to allow users
> user than {{hdfs}} connect to HiveServer2. When removed world readable
> permissions and tried to connect as other user, I got the exception:
> {code}
> 2018-05-31T10:08:40,191 ERROR [pool-7-thread-41] fs.Inode: Marking failure
> for: /user/hive/hivemetastore.jceks, error: Input/output error
> 2018-05-31T10:08:40,192 ERROR [pool-7-thread-41] fs.Inode: Throwing exception
> for: /user/hive/hivemetastore.jceks, error: Input/output error
> 2018-05-31T10:08:40,192 ERROR [pool-7-thread-41]
> metastore.RetryingHMSHandler: java.lang.RuntimeException: Error getting
> metastore password: null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:485)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:279)
> {code}
> Any ideas how to protect passwords (let's say restrict access to
> {{jceks://hdfs/user/hive/hivemetastore.jceks}} to {{600}}) and at the same
> time be available to connect to HS2 as not admin user?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-19746) Hadoop credential provider allows to read passwords for every user
[
https://issues.apache.org/jira/browse/HIVE-19746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939030#comment-16939030
]
Thejas Nair commented on HIVE-19746:
The jceks file can be stored in local filesystem, to avoid this issue.
> Hadoop credential provider allows to read passwords for every user
> --
>
> Key: HIVE-19746
> URL: https://issues.apache.org/jira/browse/HIVE-19746
> Project: Hive
> Issue Type: Bug
> Components: Security
>Reporter: Oleksiy Sayankin
>Assignee: Oleksiy Sayankin
>Priority: Blocker
>
> This simple program allows to read any password from any {{jceks}} file:
> {code}
> package com.test.app;
> import java.util.List;
> import org.apache.hadoop.conf.Configuration;
> import org.apache.hadoop.security.alias.CredentialProvider;
> import org.apache.hadoop.security.alias.CredentialProviderFactory;
> import java.io.IOException;
> public class PasswordReader {
> public static void main(String[] args) throws IOException {
> if (args == null || args.length == 0){
> throw new IllegalArgumentException("Credential provider path is to
> set");
> }
> String credentialProviderPath = args[0];
> Configuration configuration = new Configuration();
> configuration.set(CredentialProviderFactory.CREDENTIAL_PROVIDER_PATH,
> credentialProviderPath);
> CredentialProvider credentialProvider =
> CredentialProviderFactory.getProviders(configuration).get(0);
> List aliases = credentialProvider.getAliases();
> for(String alias : aliases){
> System.out.println(alias + " = " + new
> String(configuration.getPassword(alias)));
> }
> }
> }
> {code}
> {code}
> java -cp $(hadoop classpath):password-reader.jar com.test.app.PasswordReader
> jceks://hdfs/user/hive/hivemetastore.jceks
> {code}
> *RESULT*
> {code}
> javax.jdo.option.connectionpassword = 123456
> {code}
> File {{jceks://hdfs/user/hive/hivemetastore.jceks}} has {{\-rw\-r\-\-r\-\-}}
> permissions and {{hdfs:hdfs}} owner:group. We can't remove world readable
> permissions here, because Hive is configured for impersonation to allow users
> user than {{hdfs}} connect to HiveServer2. When removed world readable
> permissions and tried to connect as other user, I got the exception:
> {code}
> 2018-05-31T10:08:40,191 ERROR [pool-7-thread-41] fs.Inode: Marking failure
> for: /user/hive/hivemetastore.jceks, error: Input/output error
> 2018-05-31T10:08:40,192 ERROR [pool-7-thread-41] fs.Inode: Throwing exception
> for: /user/hive/hivemetastore.jceks, error: Input/output error
> 2018-05-31T10:08:40,192 ERROR [pool-7-thread-41]
> metastore.RetryingHMSHandler: java.lang.RuntimeException: Error getting
> metastore password: null
> at
> org.apache.hadoop.hive.metastore.ObjectStore.getDataSourceProps(ObjectStore.java:485)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:279)
> {code}
> Any ideas how to protect passwords (let's say restrict access to
> {{jceks://hdfs/user/hive/hivemetastore.jceks}} to {{600}}) and at the same
> time be available to connect to HS2 as not admin user?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22137) Implement alter/rename partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939029#comment-16939029
]
Hive QA commented on HIVE-22137:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
41s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 5m
16s{color} | {color:blue} ql in master has 1566 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} standalone-metastore/metastore-server: The patch
generated 0 new + 4 unchanged - 2 fixed = 4 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 5 new + 17 unchanged - 0 fixed
= 22 total (was 17) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 37m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18755/dev-support/hive-personality.sh
|
| git revision | master / 4c8b374 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18755/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18755/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18755/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement alter/rename partition related methods on temporary tables
>
>
> Key: HIVE-22137
> URL: https://issues.apache.org/jira/browse/HIVE-22137
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22137.01.patch, HIVE-22137.02.patch
>
>
> IMetaStoreClient exposes the following methods related to altering of
> partitions:
> {code:java}
> void alter_partition(String dbName, String tblName, Partition newPart);
> void alter_partition(String catName,
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939025#comment-16939025
]
Hive QA commented on HIVE-21884:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
30s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
26s{color} | {color:blue} standalone-metastore/metastore-common in master has
32 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
5s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 1566 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} service in master has 49 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} standalone-metastore: The patch generated 19 new +
1679 unchanged - 15 fixed = 1698 total (was 1694) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 1 new + 493 unchanged - 0 fixed = 494 total (was 493) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} common: The patch generated 2 new + 367 unchanged - 0
fixed = 369 total (was 367) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 18 new + 1186 unchanged - 15 fixed = 1204 total (was 1201) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 59 new + 656 unchanged - 0
fixed = 715 total (was 656) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} service: The patch generated 0 new + 38 unchanged -
1 fixed = 38 total (was 39) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
19s{color} | {color:red} root: The patch generated 108 new + 2835 unchanged -
21 fixed = 2943 total (was 2856) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hive-minikdc passed checkstyle {color} |
|
[jira] [Updated] (HIVE-22246) Beeline reflector should handle map types
[
https://issues.apache.org/jira/browse/HIVE-22246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-22246:
Attachment: HIVE-22246.2.patch
> Beeline reflector should handle map types
> -
>
> Key: HIVE-22246
> URL: https://issues.apache.org/jira/browse/HIVE-22246
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22246.1.patch, HIVE-22246.2.patch
>
>
> Since beeline {{Reflector}} is not handling Map types, it ends up converting
> values from {{beeline.properties}} to "null" and throws NPE with {{"}}beeline
> --hivevar x=1 --hivevar y=1".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939009#comment-16939009
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981449/HIVE-21844.19.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18754/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18754/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18754/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12981449/HIVE-21844.19.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981449 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch,
> HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch,
> HIVE-21884.09.patch, HIVE-21884.10.patch, HIVE-21884.10.patch,
> HIVE-21884.11.patch, HIVE-21884.12.patch, HIVE-21884.13.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.14.patch,
> HIVE-21884.16.patch, HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled
> queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939007#comment-16939007
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981449/HIVE-21844.19.patch
{color:green}SUCCESS:{color} +1 due to 20 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 17040 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestHiveMetaStoreAlterColumnPar.org.apache.hadoop.hive.metastore.TestHiveMetaStoreAlterColumnPar
(batchId=246)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.org.apache.hadoop.hive.ql.TestWarehouseExternalDir
(batchId=274)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testExternalDefaultPaths
(batchId=274)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths
(batchId=274)
org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.testScheduledQueryExecutionImpersonation
(batchId=277)
org.apache.hive.service.cli.session.TestSessionManagerMetrics.testAbandonedSessionMetrics
(batchId=237)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18753/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18753/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18753/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981449 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch,
> HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch,
> HIVE-21884.09.patch, HIVE-21884.10.patch, HIVE-21884.10.patch,
> HIVE-21884.11.patch, HIVE-21884.12.patch, HIVE-21884.13.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.14.patch,
> HIVE-21884.16.patch, HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled
> queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22235) CommandProcessorResponse should not be an exception
[ https://issues.apache.org/jira/browse/HIVE-22235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22235: -- Attachment: HIVE-22235.02.patch > CommandProcessorResponse should not be an exception > --- > > Key: HIVE-22235 > URL: https://issues.apache.org/jira/browse/HIVE-22235 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22235.01.patch, HIVE-22235.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The CommandProcessorResponse class extends Exception. This may be convenient, > but it is wrong, as a response is not an exception. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22244) Added default ACLs for znodes on a non-kerberized cluster
[ https://issues.apache.org/jira/browse/HIVE-22244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22244: -- Attachment: HIVE-22244.3.patch > Added default ACLs for znodes on a non-kerberized cluster > - > > Key: HIVE-22244 > URL: https://issues.apache.org/jira/browse/HIVE-22244 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22244.1.patch, HIVE-22244.2.patch, > HIVE-22244.3.patch > > > Set default ACLs for znodes on a non-kerberized cluster: > Create/Read/Delete/Write/Admin to the world -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22241) Implement UDF to interpret date/timestamp using its internal representation and Gregorian-Julian hybrid calendar
[ https://issues.apache.org/jira/browse/HIVE-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938993#comment-16938993 ] Jesus Camacho Rodriguez commented on HIVE-22241: [~szita], I have updated the patch after our conversation offline. I changed the names and descriptions (in the JIRA and UDF) to make clearer the intent of the new UDF. I also added new test cases. Can you take a look again? Thanks > Implement UDF to interpret date/timestamp using its internal representation > and Gregorian-Julian hybrid calendar > > > Key: HIVE-22241 > URL: https://issues.apache.org/jira/browse/HIVE-22241 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22241.01.patch, HIVE-22241.02.patch, > HIVE-22241.patch > > Time Spent: 10m > Remaining Estimate: 0h > > UDF that converts a date/timestamp to new *proleptic Gregorian calendar* (ISO > 8601 standard), which is produced by extending the Gregorian calendar > backward to dates preceding its official introduction in 1582, assuming that > its internal days/milliseconds since epoch is calculated using legacy > *Gregorian-Julian hybrid* calendar, i.e., calendar that supports both the > Julian and Gregorian calendar systems with the support of a single > discontinuity, which corresponds by default to the Gregorian date when the > Gregorian calendar was instituted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22241) Implement UDF to interpret date/timestamp using its internal representation and Gregorian-Julian hybrid calendar
[ https://issues.apache.org/jira/browse/HIVE-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22241: --- Attachment: HIVE-22241.02.patch > Implement UDF to interpret date/timestamp using its internal representation > and Gregorian-Julian hybrid calendar > > > Key: HIVE-22241 > URL: https://issues.apache.org/jira/browse/HIVE-22241 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22241.01.patch, HIVE-22241.02.patch, > HIVE-22241.patch > > Time Spent: 10m > Remaining Estimate: 0h > > UDF that converts a date/timestamp to new *proleptic Gregorian calendar* (ISO > 8601 standard), which is produced by extending the Gregorian calendar > backward to dates preceding its official introduction in 1582, assuming that > its internal days/milliseconds since epoch is calculated using legacy > *Gregorian-Julian hybrid* calendar, i.e., calendar that supports both the > Julian and Gregorian calendar systems with the support of a single > discontinuity, which corresponds by default to the Gregorian date when the > Gregorian calendar was instituted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22241) Implement UDF to interpret date/timestamp using its internal representation and Gregorian-Julian hybrid calendar
[ https://issues.apache.org/jira/browse/HIVE-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22241: --- Description: UDF that converts a date/timestamp to new *proleptic Gregorian calendar* (ISO 8601 standard), which is produced by extending the Gregorian calendar backward to dates preceding its official introduction in 1582, assuming that its internal days/milliseconds since epoch is calculated using legacy *Gregorian-Julian hybrid* calendar, i.e., calendar that supports both the Julian and Gregorian calendar systems with the support of a single discontinuity, which corresponds by default to the Gregorian date when the Gregorian calendar was instituted. was:UDF that converts a date/timestamp from *Gregorian-Julian hybrid* calendar, i.e., calendar that supports both the Julian and Gregorian calendar systems with the support of a single discontinuity, which corresponds by default to the Gregorian date when the Gregorian calendar was instituted, to *proleptic Gregorian calendar* (ISO 8601 standard), which is produced by extending the Gregorian calendar backward to dates preceding its official introduction in 1582. > Implement UDF to interpret date/timestamp using its internal representation > and Gregorian-Julian hybrid calendar > > > Key: HIVE-22241 > URL: https://issues.apache.org/jira/browse/HIVE-22241 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22241.01.patch, HIVE-22241.patch > > Time Spent: 10m > Remaining Estimate: 0h > > UDF that converts a date/timestamp to new *proleptic Gregorian calendar* (ISO > 8601 standard), which is produced by extending the Gregorian calendar > backward to dates preceding its official introduction in 1582, assuming that > its internal days/milliseconds since epoch is calculated using legacy > *Gregorian-Julian hybrid* calendar, i.e., calendar that supports both the > Julian and Gregorian calendar systems with the support of a single > discontinuity, which corresponds by default to the Gregorian date when the > Gregorian calendar was instituted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22241) Implement UDF to interpret date/timestamp using its internal representation and Gregorian-Julian hybrid calendar
[ https://issues.apache.org/jira/browse/HIVE-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22241: --- Summary: Implement UDF to interpret date/timestamp using its internal representation and Gregorian-Julian hybrid calendar (was: Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid calendar to proleptic Gregorian calendar) > Implement UDF to interpret date/timestamp using its internal representation > and Gregorian-Julian hybrid calendar > > > Key: HIVE-22241 > URL: https://issues.apache.org/jira/browse/HIVE-22241 > Project: Hive > Issue Type: Improvement > Components: UDF >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22241.01.patch, HIVE-22241.patch > > Time Spent: 10m > Remaining Estimate: 0h > > UDF that converts a date/timestamp from *Gregorian-Julian hybrid* calendar, > i.e., calendar that supports both the Julian and Gregorian calendar systems > with the support of a single discontinuity, which corresponds by default to > the Gregorian date when the Gregorian calendar was instituted, to *proleptic > Gregorian calendar* (ISO 8601 standard), which is produced by extending the > Gregorian calendar backward to dates preceding its official introduction in > 1582. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22245) Make qtest feature parser reuseable
[
https://issues.apache.org/jira/browse/HIVE-22245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938971#comment-16938971
]
Hive QA commented on HIVE-22245:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981443/HIVE-22245.04.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17003 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18752/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18752/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18752/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981443 - PreCommit-HIVE-Build
> Make qtest feature parser reuseable
> ---
>
> Key: HIVE-22245
> URL: https://issues.apache.org/jira/browse/HIVE-22245
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22245.01.patch, HIVE-22245.02.patch,
> HIVE-22245.03.patch, HIVE-22245.03.patch, HIVE-22245.04.patch
>
>
> right now we have a parser for {{--! qt:dataset}} ; to enable further
> addition of things (I would like to run scheduled query service for some
> qtests )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22245) Make qtest feature parser reuseable
[
https://issues.apache.org/jira/browse/HIVE-22245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938938#comment-16938938
]
Hive QA commented on HIVE-22245:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} itests/util: The patch generated 5 new + 30 unchanged
- 5 fixed = 35 total (was 35) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
53s{color} | {color:red} itests/util generated 1 new + 52 unchanged - 1 fixed =
53 total (was 53) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:itests/util |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.qoption.QTestOptionDispatcher.parse(File):in
org.apache.hadoop.hive.ql.qoption.QTestOptionDispatcher.parse(File): new
java.io.FileReader(File) At QTestOptionDispatcher.java:[line 51] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18752/dev-support/hive-personality.sh
|
| git revision | master / 4c8b374 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18752/yetus/diff-checkstyle-itests_util.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18752/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18752/yetus/new-findbugs-itests_util.html
|
| modules | C: itests/util U: itests/util |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18752/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Make qtest feature parser reuseable
> ---
>
> Key: HIVE-22245
> URL: https://issues.apache.org/jira/browse/HIVE-22245
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22245.01.patch, HIVE-22245.02.patch,
> HIVE-22245.03.patch, HIVE-22245.03.patch, HIVE-22245.04.patch
>
>
> right now we have a parser for {{--! qt:dataset}} ; to enable further
> addition of things (I would like to run scheduled query service for some
> qtests )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22244) Added default ACLs for znodes on a non-kerberized cluster
[
https://issues.apache.org/jira/browse/HIVE-22244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938922#comment-16938922
]
Hive QA commented on HIVE-22244:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981440/HIVE-22244.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17007 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=363)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18751/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18751/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18751/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981440 - PreCommit-HIVE-Build
> Added default ACLs for znodes on a non-kerberized cluster
> -
>
> Key: HIVE-22244
> URL: https://issues.apache.org/jira/browse/HIVE-22244
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22244.1.patch, HIVE-22244.2.patch
>
>
> Set default ACLs for znodes on a non-kerberized cluster:
> Create/Read/Delete/Write/Admin to the world
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22246) Beeline reflector should handle map types
[
https://issues.apache.org/jira/browse/HIVE-22246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938897#comment-16938897
]
Gopal Vijayaraghavan commented on HIVE-22246:
-
{code}
2019-09-26T11:02:02,410 INFO [main] txn.TxnHandler: Non-retryable error in
commitTxn(CommitTxnRequest(txnid:1)) : Table/View
'MATERIALIZATION_REBUILD_LOCKS' does not exist. (SQLState=42X05,
ErrorCode=2)
2019-09-26T11:02:02,413 ERROR [main] metastore.RetryingHMSHandler:
MetaException(message:Unable to update transaction database
java.sql.SQLSyntaxErrorException: Table/View 'MATERIALIZATION_REBUILD_LOCKS'
does not exist.
at
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown
Source)
{code}
failures look incidental.
> Beeline reflector should handle map types
> -
>
> Key: HIVE-22246
> URL: https://issues.apache.org/jira/browse/HIVE-22246
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22246.1.patch
>
>
> Since beeline {{Reflector}} is not handling Map types, it ends up converting
> values from {{beeline.properties}} to "null" and throws NPE with {{"}}beeline
> --hivevar x=1 --hivevar y=1".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22205: - Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Fix has been committed to master. Thank you for the review [~samuelan] [~thejas] > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22205.2.patch, HIVE-22205.3.patch, > HIVE-22205.4.patch, HIVE-22205.6.patch, HIVE-22205.7.patch, HIVE-22205.patch > > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22136) Turn on tez.bucket.pruning
[ https://issues.apache.org/jira/browse/HIVE-22136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22136: --- Status: Patch Available (was: Open) > Turn on tez.bucket.pruning > --- > > Key: HIVE-22136 > URL: https://issues.apache.org/jira/browse/HIVE-22136 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22136.1.patch, HIVE-22136.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22136) Turn on tez.bucket.pruning
[ https://issues.apache.org/jira/browse/HIVE-22136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22136: --- Status: Open (was: Patch Available) > Turn on tez.bucket.pruning > --- > > Key: HIVE-22136 > URL: https://issues.apache.org/jira/browse/HIVE-22136 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22136.1.patch, HIVE-22136.2.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22079:
---
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch, HIVE-22079.4.patch, HIVE-22079.5.patch,
> HIVE-22079.6.patch, HIVE-22079.7.patch, HIVE-22079.8.patch, HIVE-22079.9.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22244) Added default ACLs for znodes on a non-kerberized cluster
[
https://issues.apache.org/jira/browse/HIVE-22244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938868#comment-16938868
]
Hive QA commented on HIVE-22244:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
6s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18751/dev-support/hive-personality.sh
|
| git revision | master / be1d957 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18751/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Added default ACLs for znodes on a non-kerberized cluster
> -
>
> Key: HIVE-22244
> URL: https://issues.apache.org/jira/browse/HIVE-22244
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22244.1.patch, HIVE-22244.2.patch
>
>
> Set default ACLs for znodes on a non-kerberized cluster:
> Create/Read/Delete/Write/Admin to the world
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-22042: Attachment: HIVE-22042.4.patch > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: John Sherman >Priority: Minor > Attachments: HIVE-22042.1.patch, HIVE-22042.2.patch, > HIVE-22042.3.patch, HIVE-22042.4.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22246) Beeline reflector should handle map types
[
https://issues.apache.org/jira/browse/HIVE-22246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938862#comment-16938862
]
Hive QA commented on HIVE-22246:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981421/HIVE-22246.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 40 failed/errored test(s), 16975 tests
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
org.apache.hadoop.hive.ql.TestTxnCommands2.testACIDwithSchemaEvolutionAndCompaction
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testAcidOrcWritePreservesFieldNames
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testAcidWithSchemaEvolution
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testAlterTable (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testBucketCodec (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testBucketizedInputFormat
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testCleanerForTxnToWriteId
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testCompactWithDelete (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDeleteEventsCompaction
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDeleteIn (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testDynamicPartitionsMerge2
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testETLSplitStrategyForACID
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testEmptyInTblproperties
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testFailHeartbeater (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testFailureOnAlteringTransactionalProperties
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testFileSystemUnCaching (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInitiatorWithMultipleFailedCompactions
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwrite1 (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwrite2 (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwriteWithSelfJoin
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge2 (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMerge3 (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMergeWithPredicate (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMmTableCompaction (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMultiInsert (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testMultiInsertStatement
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidInsert (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion02
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOpenTxnsCounter (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOrcNoPPD (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOrcPPD (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testOriginalFileReaderWhenNonAcidConvertedToAcid
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testUpdateMixedCase (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.testValidTxnsBookkeeping
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.updateDeletePartitioned (batchId=334)
org.apache.hadoop.hive.ql.TestTxnCommands2.writeBetweenWorkerAndCleaner
(batchId=334)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18750/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18750/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18750/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 40 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981421 - PreCommit-HIVE-Build
> Beeline reflector should handle map types
> -
>
> Key: HIVE-22246
> URL: https://issues.apache.org/jira/browse/HIVE-22246
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22246.1.patch
>
>
> Since beeline {{Reflector}} is not handling Map types, it
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Status: Patch Available (was: Open) > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch, > HIVE-21449.5.patch, HIVE-21449.6.patch, HIVE-21449.6.patch, > HIVE-21449.6.patch, HIVE-21449.6.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Attachment: HIVE-21449.6.patch > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch, > HIVE-21449.5.patch, HIVE-21449.6.patch, HIVE-21449.6.patch, > HIVE-21449.6.patch, HIVE-21449.6.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21449) implement 'WITHIN GROUP' clause
[ https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-21449: -- Status: Open (was: Patch Available) > implement 'WITHIN GROUP' clause > --- > > Key: HIVE-21449 > URL: https://issues.apache.org/jira/browse/HIVE-21449 > Project: Hive > Issue Type: Improvement >Reporter: László Bodor >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch, > HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch, > HIVE-21449.5.patch, HIVE-21449.6.patch, HIVE-21449.6.patch, > HIVE-21449.6.patch, HIVE-21449.6.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22246) Beeline reflector should handle map types
[
https://issues.apache.org/jira/browse/HIVE-22246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938827#comment-16938827
]
Gopal Vijayaraghavan commented on HIVE-22246:
-
Change to split("=",2), so that the strings can contain their own =.
LGTM - +1
> Beeline reflector should handle map types
> -
>
> Key: HIVE-22246
> URL: https://issues.apache.org/jira/browse/HIVE-22246
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22246.1.patch
>
>
> Since beeline {{Reflector}} is not handling Map types, it ends up converting
> values from {{beeline.properties}} to "null" and throws NPE with {{"}}beeline
> --hivevar x=1 --hivevar y=1".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22236) Fail to create View selecting View containing NOT IN subquery
[
https://issues.apache.org/jira/browse/HIVE-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938822#comment-16938822
]
Zoltan Haindrich commented on HIVE-22236:
-
+1
> Fail to create View selecting View containing NOT IN subquery
> -
>
> Key: HIVE-22236
> URL: https://issues.apache.org/jira/browse/HIVE-22236
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Minor
> Attachments: HIVE-22236.01.patch, HIVE-22236.02.patch,
> HIVE-22236.03.patch, HIVE-22236.q
>
>
> * Given a complicated view with a select statement that has subquery
> containing "{{NOT IN}}"
> * Hive fails to create a simple view as {{SELECT * FROM complicated_view}}
> * (with CBO disabled).
> The unparse replacements of the complicated view will be applied to the text
> of the simple view, resulting in {{IllegalArgumentException: replace: range
> invalid}} exceptions from {{org.antlr.runtime.TokenRewriteStream.replace}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22246) Beeline reflector should handle map types
[
https://issues.apache.org/jira/browse/HIVE-22246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938821#comment-16938821
]
Hive QA commented on HIVE-22246:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18750/dev-support/hive-personality.sh
|
| git revision | master / be1d957 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: beeline U: beeline |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18750/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Beeline reflector should handle map types
> -
>
> Key: HIVE-22246
> URL: https://issues.apache.org/jira/browse/HIVE-22246
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22246.1.patch
>
>
> Since beeline {{Reflector}} is not handling Map types, it ends up converting
> values from {{beeline.properties}} to "null" and throws NPE with {{"}}beeline
> --hivevar x=1 --hivevar y=1".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.9.1
[
https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938805#comment-16938805
]
Hive QA commented on HIVE-21737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981427/0002-HIVE-21737-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18749/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18749/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18749/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12981427/0002-HIVE-21737-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981427 - PreCommit-HIVE-Build
> Upgrade Avro to version 1.9.1
> -
>
> Key: HIVE-21737
> URL: https://issues.apache.org/jira/browse/HIVE-21737
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Ismaël Mejía
>Assignee: Fokko Driesprong
>Priority: Minor
> Labels: pull-request-available
> Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch,
> 0002-HIVE-21737-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner
> version of Avro without Jackson in the public API. Worth the update.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.9.1
[
https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938803#comment-16938803
]
Hive QA commented on HIVE-21737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981427/0002-HIVE-21737-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18748/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18748/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18748/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-26 16:56:12.402
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18748/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-26 16:56:12.405
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
50862a6..be1d957 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 50862a6 HIVE-22231: Hive query with big size via knox fails with
Broken pipe Write failed (Denys Kuzmenko via Peter Vary)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at be1d957 HIVE-22242: Move TempTable and PartitionTree out of
SessionHiveMetastoreClient (Laszlo Pinter via Peter Vary)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-26 16:56:13.355
+ rm -rf ../yetus_PreCommit-HIVE-Build-18748
+ mkdir ../yetus_PreCommit-HIVE-Build-18748
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18748
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18748/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/serde/src/java/org/apache/hadoop/hive/serde2/avro/TypeInfoToSchema.java: does
not exist in index
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc7247152872755620838.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc7247152872755620838.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc2428708437839839083.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger (DataNucleus.Persistence).
log4j:WARN Please initialize the log4j system properly.
DataNucleus Enhancer (version 4.1.17) for API "JDO"
[ERROR] COMPILATION ERROR :
[ERROR]
[jira] [Commented] (HIVE-22236) Fail to create View selecting View containing NOT IN subquery
[
https://issues.apache.org/jira/browse/HIVE-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938800#comment-16938800
]
Hive QA commented on HIVE-22236:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981417/HIVE-22236.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17008 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18747/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18747/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18747/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981417 - PreCommit-HIVE-Build
> Fail to create View selecting View containing NOT IN subquery
> -
>
> Key: HIVE-22236
> URL: https://issues.apache.org/jira/browse/HIVE-22236
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Minor
> Attachments: HIVE-22236.01.patch, HIVE-22236.02.patch,
> HIVE-22236.03.patch, HIVE-22236.q
>
>
> * Given a complicated view with a select statement that has subquery
> containing "{{NOT IN}}"
> * Hive fails to create a simple view as {{SELECT * FROM complicated_view}}
> * (with CBO disabled).
> The unparse replacements of the complicated view will be applied to the text
> of the simple view, resulting in {{IllegalArgumentException: replace: range
> invalid}} exceptions from {{org.antlr.runtime.TokenRewriteStream.replace}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22137) Implement alter/rename partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-22137:
-
Attachment: HIVE-22137.02.patch
> Implement alter/rename partition related methods on temporary tables
>
>
> Key: HIVE-22137
> URL: https://issues.apache.org/jira/browse/HIVE-22137
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22137.01.patch, HIVE-22137.02.patch
>
>
> IMetaStoreClient exposes the following methods related to altering of
> partitions:
> {code:java}
> void alter_partition(String dbName, String tblName, Partition newPart);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart);
> void alter_partition(String dbName, String tblName, Partition newPart,
> EnvironmentContext environmentContext);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext, String writeIdList);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext,String writeIdList, long
> writeId);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts, EnvironmentContext environmentContext, String
> writeIdList, long writeId);
> void renamePartition(final String dbname, final String tableName, final
> List part_vals, final Partition newPart);
> void renamePartition(String catName, String dbname, String tableName,
> List part_vals, Partition newPart, String validWriteIds){code}
> These should be implemented, in order to completely support partition on
> temporary tables.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22242) Move TempTable and PartitionTree out of SessionHiveMetastoreClient
[ https://issues.apache.org/jira/browse/HIVE-22242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-22242: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks for the patch [~lpinter]! > Move TempTable and PartitionTree out of SessionHiveMetastoreClient > -- > > Key: HIVE-22242 > URL: https://issues.apache.org/jira/browse/HIVE-22242 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22242.01.patch, HIVE-22242.02.patch, > HIVE-22242.03.patch > > > SessionHiveMetastoreClient is getting too crowded and unreadable. TempTable > and PartitionTree should moved into a separate class. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22236) Fail to create View selecting View containing NOT IN subquery
[
https://issues.apache.org/jira/browse/HIVE-22236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938757#comment-16938757
]
Hive QA commented on HIVE-22236:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18747/dev-support/hive-personality.sh
|
| git revision | master / 50862a6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18747/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fail to create View selecting View containing NOT IN subquery
> -
>
> Key: HIVE-22236
> URL: https://issues.apache.org/jira/browse/HIVE-22236
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Matyus
>Assignee: Zoltan Matyus
>Priority: Minor
> Attachments: HIVE-22236.01.patch, HIVE-22236.02.patch,
> HIVE-22236.03.patch, HIVE-22236.q
>
>
> * Given a complicated view with a select statement that has subquery
> containing "{{NOT IN}}"
> * Hive fails to create a simple view as {{SELECT * FROM complicated_view}}
> * (with CBO disabled).
> The unparse replacements of the complicated view will be applied to the text
> of the simple view, resulting in {{IllegalArgumentException: replace: range
> invalid}} exceptions from {{org.antlr.runtime.TokenRewriteStream.replace}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22242) Move TempTable and PartitionTree out of SessionHiveMetastoreClient
[
https://issues.apache.org/jira/browse/HIVE-22242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938733#comment-16938733
]
Hive QA commented on HIVE-22242:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981414/HIVE-22242.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17007 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18746/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18746/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18746/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981414 - PreCommit-HIVE-Build
> Move TempTable and PartitionTree out of SessionHiveMetastoreClient
> --
>
> Key: HIVE-22242
> URL: https://issues.apache.org/jira/browse/HIVE-22242
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22242.01.patch, HIVE-22242.02.patch,
> HIVE-22242.03.patch
>
>
> SessionHiveMetastoreClient is getting too crowded and unreadable. TempTable
> and PartitionTree should moved into a separate class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22248) Min value for column in stats is not set correctly for some data types
[ https://issues.apache.org/jira/browse/HIVE-22248?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-22248: -- > Min value for column in stats is not set correctly for some data types > -- > > Key: HIVE-22248 > URL: https://issues.apache.org/jira/browse/HIVE-22248 > Project: Hive > Issue Type: Bug > Components: Statistics >Reporter: Jesus Camacho Rodriguez >Assignee: Miklos Gergely >Priority: Major > > I am not sure whether the problem is printing the value or in the value > stored in the metastore itself, but for some types (e.g. tinyint, smallint, > int, bigint, double or float), the min value does not seem to be set > correctly (set to 0). > https://github.com/apache/hive/blob/master/ql/src/test/results/clientpositive/alter_table_update_status.q.out#L342 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: (was: HIVE-21844.19.patch) > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch, > HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch, > HIVE-21884.09.patch, HIVE-21884.10.patch, HIVE-21884.10.patch, > HIVE-21884.11.patch, HIVE-21884.12.patch, HIVE-21884.13.patch, > HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.14.patch, > HIVE-21884.16.patch, HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled > queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21844.19.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch, > HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch, > HIVE-21884.09.patch, HIVE-21884.10.patch, HIVE-21884.10.patch, > HIVE-21884.11.patch, HIVE-21884.12.patch, HIVE-21884.13.patch, > HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.14.patch, > HIVE-21884.16.patch, HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled > queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: (was: HIVE-21844.19.patch) > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch, > HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch, > HIVE-21884.09.patch, HIVE-21884.10.patch, HIVE-21884.10.patch, > HIVE-21884.11.patch, HIVE-21884.12.patch, HIVE-21884.13.patch, > HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.14.patch, > HIVE-21884.16.patch, HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled > queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22242) Move TempTable and PartitionTree out of SessionHiveMetastoreClient
[
https://issues.apache.org/jira/browse/HIVE-22242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938682#comment-16938682
]
Hive QA commented on HIVE-22242:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
5s{color} | {color:green} ql generated 0 new + 1566 unchanged - 1 fixed = 1566
total (was 1567) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18746/dev-support/hive-personality.sh
|
| git revision | master / 50862a6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18746/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Move TempTable and PartitionTree out of SessionHiveMetastoreClient
> --
>
> Key: HIVE-22242
> URL: https://issues.apache.org/jira/browse/HIVE-22242
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22242.01.patch, HIVE-22242.02.patch,
> HIVE-22242.03.patch
>
>
> SessionHiveMetastoreClient is getting too crowded and unreadable. TempTable
> and PartitionTree should moved into a separate class.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21844.19.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch, > HIVE-21844.19.patch, HIVE-21884.01.patch, HIVE-21884.02.patch, > HIVE-21884.03.patch, HIVE-21884.09.patch, HIVE-21884.10.patch, > HIVE-21884.10.patch, HIVE-21884.11.patch, HIVE-21884.12.patch, > HIVE-21884.13.patch, HIVE-21884.14.patch, HIVE-21884.14.patch, > HIVE-21884.14.patch, HIVE-21884.16.patch, HIVE-21884.17.patch, > HIVE-21884.18.patch, Scheduled queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21844.19.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch, > HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch, > HIVE-21884.09.patch, HIVE-21884.10.patch, HIVE-21884.10.patch, > HIVE-21884.11.patch, HIVE-21884.12.patch, HIVE-21884.13.patch, > HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.14.patch, > HIVE-21884.16.patch, HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled > queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21844.19.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch, > HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch, > HIVE-21884.09.patch, HIVE-21884.10.patch, HIVE-21884.10.patch, > HIVE-21884.11.patch, HIVE-21884.12.patch, HIVE-21884.13.patch, > HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.14.patch, > HIVE-21884.16.patch, HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled > queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: (was: HIVE-21844.19.patch) > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21844.19.patch, > HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch, > HIVE-21884.09.patch, HIVE-21884.10.patch, HIVE-21884.10.patch, > HIVE-21884.11.patch, HIVE-21884.12.patch, HIVE-21884.13.patch, > HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.14.patch, > HIVE-21884.16.patch, HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled > queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22245) Make qtest feature parser reuseable
[
https://issues.apache.org/jira/browse/HIVE-22245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22245:
Attachment: HIVE-22245.04.patch
> Make qtest feature parser reuseable
> ---
>
> Key: HIVE-22245
> URL: https://issues.apache.org/jira/browse/HIVE-22245
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22245.01.patch, HIVE-22245.02.patch,
> HIVE-22245.03.patch, HIVE-22245.03.patch, HIVE-22245.04.patch
>
>
> right now we have a parser for {{--! qt:dataset}} ; to enable further
> addition of things (I would like to run scheduled query service for some
> qtests )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22244) Added default ACLs for znodes on a non-kerberized cluster
[
https://issues.apache.org/jira/browse/HIVE-22244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938658#comment-16938658
]
Hive QA commented on HIVE-22244:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981407/HIVE-22244.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17007 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18745/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18745/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18745/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981407 - PreCommit-HIVE-Build
> Added default ACLs for znodes on a non-kerberized cluster
> -
>
> Key: HIVE-22244
> URL: https://issues.apache.org/jira/browse/HIVE-22244
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22244.1.patch, HIVE-22244.2.patch
>
>
> Set default ACLs for znodes on a non-kerberized cluster:
> Create/Read/Delete/Write/Admin to the world
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-8838) Support Parquet through HCatalog
[ https://issues.apache.org/jira/browse/HIVE-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938611#comment-16938611 ] Jay Green-Stevens edited comment on HIVE-8838 at 9/26/19 1:37 PM: -- Hi all - I’ve been working on backporting the patch 8838.4 onto Hive to allow for parquet tables to be used in versions of Hive *pre* 3.0.0. Would it be best for me to reopen this ticket, or create a new one? [^HIVE-8838.4.patch] was (Author: jay_grn): Hi all - I’ve been working on backporting the patch HIVE-8838.4 onto Hive to allow for parquet tables to be used in versions of Hive *pre* 3.0.0. Would it be best for me to reopen this ticket, or create a new one? > Support Parquet through HCatalog > > > Key: HIVE-8838 > URL: https://issues.apache.org/jira/browse/HIVE-8838 > Project: Hive > Issue Type: New Feature >Reporter: Brock Noland >Assignee: Ádám Szita >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-8838.0.patch, HIVE-8838.1.patch, HIVE-8838.2.patch, > HIVE-8838.3.patch, HIVE-8838.4.patch > > > Similar to HIVE-8687 for Avro we need to fix Parquet with HCatalog. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-8838) Support Parquet through HCatalog
[ https://issues.apache.org/jira/browse/HIVE-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938611#comment-16938611 ] Jay Green-Stevens commented on HIVE-8838: - Hi all - I’ve been working on backporting the patch HIVE-8838.4 onto Hive to allow for parquet tables to be used in versions of Hive *pre* 3.0.0. Would it be best for me to reopen this ticket, or create a new one? > Support Parquet through HCatalog > > > Key: HIVE-8838 > URL: https://issues.apache.org/jira/browse/HIVE-8838 > Project: Hive > Issue Type: New Feature >Reporter: Brock Noland >Assignee: Ádám Szita >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-8838.0.patch, HIVE-8838.1.patch, HIVE-8838.2.patch, > HIVE-8838.3.patch, HIVE-8838.4.patch > > > Similar to HIVE-8687 for Avro we need to fix Parquet with HCatalog. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22244) Added default ACLs for znodes on a non-kerberized cluster
[ https://issues.apache.org/jira/browse/HIVE-22244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22244: -- Attachment: HIVE-22244.2.patch > Added default ACLs for znodes on a non-kerberized cluster > - > > Key: HIVE-22244 > URL: https://issues.apache.org/jira/browse/HIVE-22244 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22244.1.patch, HIVE-22244.2.patch > > > Set default ACLs for znodes on a non-kerberized cluster: > Create/Read/Delete/Write/Admin to the world -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22247) HiveHFileOutputFormat throws FileNotFoundException when partition's task output empty
[
https://issues.apache.org/jira/browse/HIVE-22247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie reassigned HIVE-22247:
-
> HiveHFileOutputFormat throws FileNotFoundException when partition's task
> output empty
> -
>
> Key: HIVE-22247
> URL: https://issues.apache.org/jira/browse/HIVE-22247
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0, 2.2.0
>Reporter: xiepengjie
>Assignee: xiepengjie
>Priority: Major
>
> When partition's task output empty, HiveHFileOutputFormat throws
> FileNotFoundException like this:
> {code:java}
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: 1 finished. closing...
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: FS[1]: records written - 0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: Final Path: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: Writing to temp file: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> 2019-09-24 19:15:55,886 INFO [main]
> org.apache.hadoop.hive.ql.exec.FileSinkOperator: New Final Path: FS
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_tmp.-ext-10002/02_0
> 2019-09-24 19:15:55,915 INFO [main]
> org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output
> Committer Algorithm version is 1
> 2019-09-24 19:15:55,954 INFO [main]
> org.apache.hadoop.conf.Configuration.deprecation: hadoop.native.lib is
> deprecated. Instead, use io.native.lib.available
> 2019-09-24 19:15:56,089 ERROR [main] ExecReducer: Hit error while closing
> operators - failing tree
> 2019-09-24 19:15:56,090 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators: java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
> at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1923)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:200)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1016)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:617)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:631)
> at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:278)
> ... 7 more
> Caused by: java.io.FileNotFoundException: File
> hdfs://Hdptest-mini-nmg/tmp/hive-staging/hadoop_hive_2019-09-24_19-15-26_453_1697529445006435790-5/_task_tmp.-ext-10002/_tmp.02_0
> does not exist.
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:880)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$700(DistributedFileSystem.java:109)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:938)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$19.doCall(DistributedFileSystem.java:934)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:945)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1592)
> at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1632)
> at
> org.apache.hadoop.hive.hbase.HiveHFileOutputFormat$1.close(HiveHFileOutputFormat.java:153)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator$FSPaths.closeWriters(FileSinkOperator.java:197)
> ... 11 more
> 2019-09-24 19:15:56,093
[jira] [Commented] (HIVE-22244) Added default ACLs for znodes on a non-kerberized cluster
[
https://issues.apache.org/jira/browse/HIVE-22244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938587#comment-16938587
]
Hive QA commented on HIVE-22244:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 1 new + 32 unchanged - 0 fixed = 33 total (was 32) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18745/dev-support/hive-personality.sh
|
| git revision | master / 50862a6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18745/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18745/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Added default ACLs for znodes on a non-kerberized cluster
> -
>
> Key: HIVE-22244
> URL: https://issues.apache.org/jira/browse/HIVE-22244
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22244.1.patch
>
>
> Set default ACLs for znodes on a non-kerberized cluster:
> Create/Read/Delete/Write/Admin to the world
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22241) Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid calendar to proleptic Gregorian calendar
[
https://issues.apache.org/jira/browse/HIVE-22241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938586#comment-16938586
]
Ádám Szita commented on HIVE-22241:
---
[~jcamachorodriguez] I think this change is doing something the other way
around.
For example
{code:java}
select to_proleptic_gregorian(CAST('1582-10-02' AS DATE));{code}
gives 1582-09-22. Which would mean that 1582-10-02 being a Julian date which it
is, corresponds to an earlier date in proleptic Gregorian, where in reality it
should be later one: 1582-10-12.
AFAIK 1582-10-04 was the last Julian date, a day followed by 1582-10-15 and
this latter is already a (true) Gregorian calendar day. So 1582-10-02
interpreted as Julian date, should be 3 days before 1582-10-15 (true)
Gregorian, making it a 1582-10-12 proleptic Gregorian date.
> Implement UDF to convert a date/timestamp from Gregorian-Julian hybrid
> calendar to proleptic Gregorian calendar
> ---
>
> Key: HIVE-22241
> URL: https://issues.apache.org/jira/browse/HIVE-22241
> Project: Hive
> Issue Type: Improvement
> Components: UDF
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22241.01.patch, HIVE-22241.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> UDF that converts a date/timestamp from *Gregorian-Julian hybrid* calendar,
> i.e., calendar that supports both the Julian and Gregorian calendar systems
> with the support of a single discontinuity, which corresponds by default to
> the Gregorian date when the Gregorian calendar was instituted, to *proleptic
> Gregorian calendar* (ISO 8601 standard), which is produced by extending the
> Gregorian calendar backward to dates preceding its official introduction in
> 1582.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22243) Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well
[ https://issues.apache.org/jira/browse/HIVE-22243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938570#comment-16938570 ] László Bodor commented on HIVE-22243: - +1 > Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well > -- > > Key: HIVE-22243 > URL: https://issues.apache.org/jira/browse/HIVE-22243 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22243.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Thrift was bumped to 0.9.3-1 in HIVE-21173, but standalone-metastore was left > out of this. > Thanks for pointing this out [~ashutosh.bapat]! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938563#comment-16938563
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981399/HIVE-21884.18.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18744/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18744/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18744/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12981399/HIVE-21884.18.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981399 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21884.01.patch,
> HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch,
> HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch,
> HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.16.patch,
> HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22245) Make qtest feature parser reuseable
[
https://issues.apache.org/jira/browse/HIVE-22245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938561#comment-16938561
]
Hive QA commented on HIVE-22245:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981400/HIVE-22245.03.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18743/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18743/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18743/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12981400/HIVE-22245.03.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981400 - PreCommit-HIVE-Build
> Make qtest feature parser reuseable
> ---
>
> Key: HIVE-22245
> URL: https://issues.apache.org/jira/browse/HIVE-22245
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22245.01.patch, HIVE-22245.02.patch,
> HIVE-22245.03.patch, HIVE-22245.03.patch
>
>
> right now we have a parser for {{--! qt:dataset}} ; to enable further
> addition of things (I would like to run scheduled query service for some
> qtests )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938560#comment-16938560
]
Hive QA commented on HIVE-21449:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981396/HIVE-21449.6.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17010 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_avro]
(batchId=298)
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=363)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18742/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18742/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18742/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981396 - PreCommit-HIVE-Build
> implement 'WITHIN GROUP' clause
> ---
>
> Key: HIVE-21449
> URL: https://issues.apache.org/jira/browse/HIVE-21449
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch,
> HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch,
> HIVE-21449.5.patch, HIVE-21449.6.patch, HIVE-21449.6.patch, HIVE-21449.6.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22231) Hive query with big size via knox fails with Broken pipe Write failed
[ https://issues.apache.org/jira/browse/HIVE-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938539#comment-16938539 ] Denys Kuzmenko commented on HIVE-22231: --- Thank you, [~pvary]! > Hive query with big size via knox fails with Broken pipe Write failed > - > > Key: HIVE-22231 > URL: https://issues.apache.org/jira/browse/HIVE-22231 > Project: Hive > Issue Type: Bug >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22231.1.patch, HIVE-22231.2.patch > > > Posting large data through knox is causing the Broken pipe (Write failed) > java.net.SocketException. Issue here that HS2 is responding with 401 even > before complete query is transferred. > HS2 has to wait until all the data is received and then respond with 401. > That way knox can re-open the connection and send the negotiate header with > the data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22231) Hive query with big size via knox fails with Broken pipe Write failed
[ https://issues.apache.org/jira/browse/HIVE-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-22231: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks for the patch [~dkuzmenko]! > Hive query with big size via knox fails with Broken pipe Write failed > - > > Key: HIVE-22231 > URL: https://issues.apache.org/jira/browse/HIVE-22231 > Project: Hive > Issue Type: Bug >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22231.1.patch, HIVE-22231.2.patch > > > Posting large data through knox is causing the Broken pipe (Write failed) > java.net.SocketException. Issue here that HS2 is responding with 401 even > before complete query is transferred. > HS2 has to wait until all the data is received and then respond with 401. > That way knox can re-open the connection and send the negotiate header with > the data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22231) Hive query with big size via knox fails with Broken pipe Write failed
[ https://issues.apache.org/jira/browse/HIVE-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-22231: -- Summary: Hive query with big size via knox fails with Broken pipe Write failed (was: Hive query with big size via knox fails with 'Broken pipe (Write failed)) > Hive query with big size via knox fails with Broken pipe Write failed > - > > Key: HIVE-22231 > URL: https://issues.apache.org/jira/browse/HIVE-22231 > Project: Hive > Issue Type: Bug >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22231.1.patch, HIVE-22231.2.patch > > > Posting large data through knox is causing the Broken pipe (Write failed) > java.net.SocketException. Issue here that HS2 is responding with 401 even > before complete query is transferred. > HS2 has to wait until all the data is received and then respond with 401. > That way knox can re-open the connection and send the negotiate header with > the data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21449) implement 'WITHIN GROUP' clause
[
https://issues.apache.org/jira/browse/HIVE-21449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938519#comment-16938519
]
Hive QA commented on HIVE-21449:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1567 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18742/dev-support/hive-personality.sh
|
| git revision | master / c660cba |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18742/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> implement 'WITHIN GROUP' clause
> ---
>
> Key: HIVE-21449
> URL: https://issues.apache.org/jira/browse/HIVE-21449
> Project: Hive
> Issue Type: Improvement
>Reporter: László Bodor
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21449.1.patch, HIVE-21449.2.patch,
> HIVE-21449.3.patch, HIVE-21449.4.patch, HIVE-21449.5.patch,
> HIVE-21449.5.patch, HIVE-21449.6.patch, HIVE-21449.6.patch, HIVE-21449.6.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22234) Hive replication fails with table already exist error when replicating from old version of hive.
[
https://issues.apache.org/jira/browse/HIVE-22234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938505#comment-16938505
]
Hive QA commented on HIVE-22234:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981394/HIVE-22234.02.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18741/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18741/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18741/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-26 11:21:25.808
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18741/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-26 11:21:25.811
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at c660cba HIVE-22205: Upgrade Zookeeper and Curator libraries in
Hive. (Naveen Gangam, reviewed by Sam An)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at c660cba HIVE-22205: Upgrade Zookeeper and Curator libraries in
Hive. (Naveen Gangam, reviewed by Sam An)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-26 11:21:26.493
+ rm -rf ../yetus_PreCommit-HIVE-Build-18741
+ mkdir ../yetus_PreCommit-HIVE-Build-18741
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18741
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18741/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
fatal: git diff header lacks filename information when removing 0 leading
pathname components (line 126)
error: cannot apply binary patch to
'data/files/test_hcube_2.db/tbl/delta_001_001_/bucket_0'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to
'data/files/test_hcube_2.db/tbl/delta_001_001_/bucket_0'
without full index line
error: data/files/test_hcube_2.db/tbl/delta_001_001_/bucket_0:
patch does not apply
error: cannot apply binary patch to
'data/files/test_hcube_2.db/tbl1/delta_001_001_/bucket_0'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to
'data/files/test_hcube_2.db/tbl1/delta_001_001_/bucket_0'
without full index line
error: data/files/test_hcube_2.db/tbl1/delta_001_001_/bucket_0:
patch does not apply
error: cannot apply binary patch to
'data/files/test_hcube_2.db/tbl1/delta_002_002_/bucket_0'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to
'data/files/test_hcube_2.db/tbl1/delta_002_002_/bucket_0'
without full index line
error: data/files/test_hcube_2.db/tbl1/delta_002_002_/bucket_0:
patch does not apply
error: cannot apply binary patch to
'data/files/test_hcube_2.db/tbl1/delta_003_003_/bucket_0'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to
'data/files/test_hcube_2.db/tbl1/delta_003_003_/bucket_0'
without full index line
error: data/files/test_hcube_2.db/tbl1/delta_003_003_/bucket_0:
patch does not apply
error: cannot apply binary patch to
'data/files/test_hcube_2.db/tbl2/delta_001_001_/bucket_0'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to
[jira] [Commented] (HIVE-22225) add owner name for create database if missing
[
https://issues.apache.org/jira/browse/HIVE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938504#comment-16938504
]
Hive QA commented on HIVE-5:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981382/HIVE-5.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 16966 tests
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestJdbcWithMiniLlapArrow - did not produce a TEST-*.xml file (likely timed
out) (batchId=283)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18740/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18740/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18740/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981382 - PreCommit-HIVE-Build
> add owner name for create database if missing
> -
>
> Key: HIVE-5
> URL: https://issues.apache.org/jira/browse/HIVE-5
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Sam An
>Assignee: Sam An
>Priority: Minor
> Attachments: HIVE-5.1.patch, HIVE-5.2.patch,
> HIVE-5.4.patch, Hive-5.3.patch
>
>
> When Spark connects to HMS, the database owner name is not filled. This could
> happen to other clients as well. We shall fill this in HMS. Ownership info is
> useful for authorizer component in Ranger, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22246) Beeline reflector should handle map types
[
https://issues.apache.org/jira/browse/HIVE-22246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-22246:
Status: Patch Available (was: Open)
> Beeline reflector should handle map types
> -
>
> Key: HIVE-22246
> URL: https://issues.apache.org/jira/browse/HIVE-22246
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22246.1.patch
>
>
> Since beeline {{Reflector}} is not handling Map types, it ends up converting
> values from {{beeline.properties}} to "null" and throws NPE with {{"}}beeline
> --hivevar x=1 --hivevar y=1".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated HIVE-21737: Attachment: 0002-HIVE-21737-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Minor > Labels: pull-request-available > Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch, > 0002-HIVE-21737-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated HIVE-21737: Attachment: (was: HIVE-21737.2-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch) > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Minor > Labels: pull-request-available > Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22225) add owner name for create database if missing
[
https://issues.apache.org/jira/browse/HIVE-5?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938458#comment-16938458
]
Hive QA commented on HIVE-5:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
4s{color} | {color:blue} standalone-metastore/metastore-server in master has
170 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18740/dev-support/hive-personality.sh
|
| git revision | master / c660cba |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18740/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> add owner name for create database if missing
> -
>
> Key: HIVE-5
> URL: https://issues.apache.org/jira/browse/HIVE-5
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Sam An
>Assignee: Sam An
>Priority: Minor
> Attachments: HIVE-5.1.patch, HIVE-5.2.patch,
> HIVE-5.4.patch, Hive-5.3.patch
>
>
> When Spark connects to HMS, the database owner name is not filled. This could
> happen to other clients as well. We shall fill this in HMS. Ownership info is
> useful for authorizer component in Ranger, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=318901=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318901 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 26/Sep/19 09:55 Start Date: 26/Sep/19 09:55 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #785: [WIP] HIVE-21737: Bump Apache Avro to 1.9.0 URL: https://github.com/apache/hive/pull/785 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 318901) Time Spent: 1h (was: 50m) > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Minor > Labels: pull-request-available > Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch, > HIVE-21737.2-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch > > Time Spent: 1h > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=318900=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318900 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 26/Sep/19 09:55 Start Date: 26/Sep/19 09:55 Worklog Time Spent: 10m Work Description: iemejia commented on issue #785: [WIP] HIVE-21737: Bump Apache Avro to 1.9.0 URL: https://github.com/apache/hive/pull/785#issuecomment-535430988 I am closing this one in favor of the attached patch in [HIVE-21737](https://issues.apache.org/jira/browse/HIVE-21737) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 318900) Time Spent: 50m (was: 40m) > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Minor > Labels: pull-request-available > Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch, > HIVE-21737.2-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch > > Time Spent: 50m > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21737) Upgrade Avro to version 1.9.1
[ https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismaël Mejía updated HIVE-21737: Attachment: HIVE-21737.2-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch > Upgrade Avro to version 1.9.1 > - > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Minor > Labels: pull-request-available > Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch, > HIVE-21737.2-Use-NULL_DEFAULT_VALUE-for-Avro-fields-in.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner > version of Avro without Jackson in the public API. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938442#comment-16938442
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981399/HIVE-21884.18.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18739/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18739/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18739/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-26 09:41:56.079
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18739/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-26 09:41:56.083
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at c660cba HIVE-22205: Upgrade Zookeeper and Curator libraries in
Hive. (Naveen Gangam, reviewed by Sam An)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at c660cba HIVE-22205: Upgrade Zookeeper and Curator libraries in
Hive. (Naveen Gangam, reviewed by Sam An)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-26 09:41:57.191
+ rm -rf ../yetus_PreCommit-HIVE-Build-18739
+ mkdir ../yetus_PreCommit-HIVE-Build-18739
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18739
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18739/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed: standalone-metastore/pom.xml:95
Falling back to three-way merge...
Applied patch to 'standalone-metastore/pom.xml' with conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:1971: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:2667: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:2677: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:2688: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:2944: trailing whitespace.
error: patch failed: standalone-metastore/pom.xml:95
Falling back to three-way merge...
Applied patch to 'standalone-metastore/pom.xml' with conflicts.
U standalone-metastore/pom.xml
warning: squelched 179 whitespace errors
warning: 184 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-18739
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981399 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21844.09.patch, HIVE-21844.15.patch, HIVE-21884.01.patch,
> HIVE-21884.02.patch, HIVE-21884.03.patch, HIVE-21884.09.patch,
> HIVE-21884.10.patch, HIVE-21884.10.patch, HIVE-21884.11.patch,
> HIVE-21884.12.patch, HIVE-21884.13.patch, HIVE-21884.14.patch,
> HIVE-21884.14.patch, HIVE-21884.14.patch, HIVE-21884.16.patch,
> HIVE-21884.17.patch, HIVE-21884.18.patch, Scheduled queries2.pdf
>
>
> design document:
>
[jira] [Commented] (HIVE-22245) Make qtest feature parser reuseable
[
https://issues.apache.org/jira/browse/HIVE-22245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938441#comment-16938441
]
Hive QA commented on HIVE-22245:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981400/HIVE-22245.03.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 17003 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCompareCliDriver.testCliDriver[llap_0]
(batchId=294)
org.apache.hadoop.hive.cli.TestCompareCliDriver.testCliDriver[vectorized_math_funcs]
(batchId=294)
org.apache.hadoop.hive.metastore.TestPartitionManagement.testNoPartitionRetentionForReplTarget
(batchId=223)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18738/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18738/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18738/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981400 - PreCommit-HIVE-Build
> Make qtest feature parser reuseable
> ---
>
> Key: HIVE-22245
> URL: https://issues.apache.org/jira/browse/HIVE-22245
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22245.01.patch, HIVE-22245.02.patch,
> HIVE-22245.03.patch, HIVE-22245.03.patch
>
>
> right now we have a parser for {{--! qt:dataset}} ; to enable further
> addition of things (I would like to run scheduled query service for some
> qtests )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22243) Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well
[ https://issues.apache.org/jira/browse/HIVE-22243?focusedWorklogId=318881=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-318881 ] ASF GitHub Bot logged work on HIVE-22243: - Author: ASF GitHub Bot Created on: 26/Sep/19 09:01 Start Date: 26/Sep/19 09:01 Worklog Time Spent: 10m Work Description: dlavati commented on pull request #790: HIVE-22243 Align Apache Thrift version to 0.9.3-1 in standalone-metas… URL: https://github.com/apache/hive/pull/790 …tore as well Change-Id: Ia82684e7c69684080da7aa6669566bf5fc999cb1 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 318881) Remaining Estimate: 0h Time Spent: 10m > Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well > -- > > Key: HIVE-22243 > URL: https://issues.apache.org/jira/browse/HIVE-22243 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22243.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Thrift was bumped to 0.9.3-1 in HIVE-21173, but standalone-metastore was left > out of this. > Thanks for pointing this out [~ashutosh.bapat]! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22243) Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well
[ https://issues.apache.org/jira/browse/HIVE-22243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22243: -- Labels: pull-request-available (was: ) > Align Apache Thrift version to 0.9.3-1 in standalone-metastore as well > -- > > Key: HIVE-22243 > URL: https://issues.apache.org/jira/browse/HIVE-22243 > Project: Hive > Issue Type: Bug >Reporter: David Lavati >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22243.patch > > > Thrift was bumped to 0.9.3-1 in HIVE-21173, but standalone-metastore was left > out of this. > Thanks for pointing this out [~ashutosh.bapat]! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22246) Beeline reflector should handle map types
[
https://issues.apache.org/jira/browse/HIVE-22246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-22246:
Attachment: HIVE-22246.1.patch
> Beeline reflector should handle map types
> -
>
> Key: HIVE-22246
> URL: https://issues.apache.org/jira/browse/HIVE-22246
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22246.1.patch
>
>
> Since beeline {{Reflector}} is not handling Map types, it ends up converting
> values from {{beeline.properties}} to "null" and throws NPE with {{"}}beeline
> --hivevar x=1 --hivevar y=1".
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22245) Make qtest feature parser reuseable
[
https://issues.apache.org/jira/browse/HIVE-22245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938408#comment-16938408
]
Hive QA commented on HIVE-22245:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} itests/util: The patch generated 5 new + 26 unchanged
- 5 fixed = 31 total (was 31) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
51s{color} | {color:red} itests/util generated 1 new + 52 unchanged - 1 fixed =
53 total (was 53) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:itests/util |
| | Found reliance on default encoding in
org.apache.hadoop.hive.ql.qoption.QTestOptionDispatcher.parse(File):in
org.apache.hadoop.hive.ql.qoption.QTestOptionDispatcher.parse(File): new
java.io.FileReader(File) At QTestOptionDispatcher.java:[line 51] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18738/dev-support/hive-personality.sh
|
| git revision | master / c660cba |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18738/yetus/diff-checkstyle-itests_util.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18738/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18738/yetus/new-findbugs-itests_util.html
|
| modules | C: itests/util U: itests/util |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18738/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Make qtest feature parser reuseable
> ---
>
> Key: HIVE-22245
> URL: https://issues.apache.org/jira/browse/HIVE-22245
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22245.01.patch, HIVE-22245.02.patch,
> HIVE-22245.03.patch, HIVE-22245.03.patch
>
>
> right now we have a parser for {{--! qt:dataset}} ; to enable further
> addition of things (I would like to run scheduled query service for some
> qtests )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22208) Column name with reserved keyword is unescaped when query including join on table with mask column is re-written
[
https://issues.apache.org/jira/browse/HIVE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16938396#comment-16938396
]
Hive QA commented on HIVE-22208:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12981373/HIVE-22208.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17008 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_pos_alias]
(batchId=58)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_with_masking]
(batchId=181)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18737/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18737/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18737/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12981373 - PreCommit-HIVE-Build
> Column name with reserved keyword is unescaped when query including join on
> table with mask column is re-written
>
>
> Key: HIVE-22208
> URL: https://issues.apache.org/jira/browse/HIVE-22208
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Riju Trivedi
>Assignee: Zoltan Matyus
>Priority: Critical
> Attachments: HIVE-22208.01.patch
>
>
> Join query involving table with mask column and other having reserved
> keyword as column name fails with SemanticException during parsing re-written
> query :
> Original Query :
> {code:java}
> select a.`date`, b.nm
> from sample_keyword a
> join sample_mask b
> on b.id = a.id;
> {code}
> Re-written Query :
>
> {code:java}
> select a.date, b.nm
> from sample_keyword a
> join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
> BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
> `default`.`sample_mask` )`b`
> on b.id = a.id;
> {code}
> Re-written query does not have escape quotes for date column which cause
> SemanticException while parsing :
> {code:java}
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near 'a' '.' 'date' in selection target
>
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
> SemanticAnalyzer.java:12084)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
> SemanticAnalyzer.java:12298)
> at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
> CalcitePlanner.java:360)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
> BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-14650) Select fails when ORC file has more columns than table schema
[
https://issues.apache.org/jira/browse/HIVE-14650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Saijin Huang updated HIVE-14650:
Description: (was: When SELECTing from a Hive ORC table, the following
IndexOutOfBoundsException is thrown if the underlying ORC file has 4 or more
columns than the Hive schema (where N is the number of columns in the ORC file).
{noformat}
Failed with exception java.io.IOException:java.lang.IndexOutOfBoundsException:
toIndex = N
16/08/25 15:22:19 ERROR CliDriver: Failed with exception
java.io.IOException:java.lang.IndexOutOfBoundsException: toIndex = N
java.io.IOException: java.lang.IndexOutOfBoundsException: toIndex = N
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:507)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:414)
at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1686)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.IndexOutOfBoundsException: toIndex = 6
at java.util.ArrayList.subListRangeCheck(ArrayList.java:1004)
at java.util.ArrayList.subList(ArrayList.java:996)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderFactory.getSchemaOnRead(RecordReaderFactory.java:161)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderFactory.createTreeReader(RecordReaderFactory.java:66)
at
org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.(RecordReaderImpl.java:202)
at
org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:541)
at
org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.(OrcRawRecordMerger.java:183)
at
org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$OriginalReaderPair.(OrcRawRecordMerger.java:226)
at
org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:437)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1216)
at
org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1113)
at
org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:673)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:323)
at
org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:445)
... 15 more
{noformat}
This error appears to be related to the patch of HIVE-10591.
Steps to reproduce (Hive QL):
{noformat}
DROP TABLE IF EXISTS orc_drop_column;
CREATE TABLE orc_drop_column (`id` int, `name` string, `description` string,
`somevalue` double, `someflag` boolean, `somedate` timestamp) STORED AS ORC;
INSERT INTO TABLE orc_drop_column select * from (select 1, 'my_name',
'my_desc', 5.5, true, '2016-08-25 06:00:00') a;
ALTER TABLE orc_drop_column SET SERDE
'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe';
ALTER TABLE orc_drop_column REPLACE COLUMNS (
`id` int,
`name` string
);
ALTER TABLE orc_drop_column SET SERDE
'org.apache.hadoop.hive.ql.io.orc.OrcSerde';
SELECT id, name FROM orc_drop_column;
{noformat}
)
> Select fails when ORC file has more columns than table schema
> -
>
> Key: HIVE-14650
> URL: https://issues.apache.org/jira/browse/HIVE-14650
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.2.1
>Reporter: Jeff Mink
>Assignee: xiepengjie
>Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
