[jira] [Commented] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355084#comment-16355084
]
Hive QA commented on HIVE-16605:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909328/HIVE-16605.5.patch
{color:green}SUCCESS:{color} +1 due to 10 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 12987 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9063/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9063/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9063/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909328 - PreCommit-HIVE-Build
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch, HIVE-16605.5.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18464) Fix TestSSL test failures
[ https://issues.apache.org/jira/browse/HIVE-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar reassigned HIVE-18464: -- Assignee: Vihang Karajgaonkar > Fix TestSSL test failures > - > > Key: HIVE-18464 > URL: https://issues.apache.org/jira/browse/HIVE-18464 > Project: Hive > Issue Type: Test >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > > TestSSL test has been failing for a while now. Following three tests fail > everytime precommit job runs. > org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=232) > org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=232) > org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=232) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18421) Vectorized execution handles overflows in a different manner than non-vectorized execution
[
https://issues.apache.org/jira/browse/HIVE-18421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-18421:
---
Attachment: (was: HIVE-18421.06.patch)
> Vectorized execution handles overflows in a different manner than
> non-vectorized execution
> --
>
> Key: HIVE-18421
> URL: https://issues.apache.org/jira/browse/HIVE-18421
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 2.1.1, 2.2.0, 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-18421.01.patch, HIVE-18421.02.patch,
> HIVE-18421.03.patch, HIVE-18421.04.patch, HIVE-18421.05.patch,
> HIVE-18421.06.patch
>
>
> In vectorized execution arithmetic operations which cause integer overflows
> can give wrong results. Issue is reproducible in both Orc and parquet.
> Simple test case to reproduce this issue
> {noformat}
> set hive.vectorized.execution.enabled=true;
> create table parquettable (t1 tinyint, t2 tinyint) stored as parquet;
> insert into parquettable values (-104, 25), (-112, 24), (54, 9);
> select t1, t2, (t1-t2) as diff from parquettable where (t1-t2) < 50 order by
> diff desc;
> +---+-+---+
> | t1 | t2 | diff |
> +---+-+---+
> | -104 | 25 | 127 |
> | -112 | 24 | 120 |
> | 54| 9 | 45|
> +---+-+---+
> {noformat}
> When vectorization is turned off the same query produces only one row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18421) Vectorized execution handles overflows in a different manner than non-vectorized execution
[
https://issues.apache.org/jira/browse/HIVE-18421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-18421:
---
Attachment: HIVE-18421.06.patch
> Vectorized execution handles overflows in a different manner than
> non-vectorized execution
> --
>
> Key: HIVE-18421
> URL: https://issues.apache.org/jira/browse/HIVE-18421
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 2.1.1, 2.2.0, 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-18421.01.patch, HIVE-18421.02.patch,
> HIVE-18421.03.patch, HIVE-18421.04.patch, HIVE-18421.05.patch,
> HIVE-18421.06.patch
>
>
> In vectorized execution arithmetic operations which cause integer overflows
> can give wrong results. Issue is reproducible in both Orc and parquet.
> Simple test case to reproduce this issue
> {noformat}
> set hive.vectorized.execution.enabled=true;
> create table parquettable (t1 tinyint, t2 tinyint) stored as parquet;
> insert into parquettable values (-104, 25), (-112, 24), (54, 9);
> select t1, t2, (t1-t2) as diff from parquettable where (t1-t2) < 50 order by
> diff desc;
> +---+-+---+
> | t1 | t2 | diff |
> +---+-+---+
> | -104 | 25 | 127 |
> | -112 | 24 | 120 |
> | 54| 9 | 45|
> +---+-+---+
> {noformat}
> When vectorization is turned off the same query produces only one row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18622:
Attachment: (was: HIVE-18622.02.patch)
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.03.patch
>
>
>
> Many vector expression classes are missing guards around setting noNulls
> among other things.
> {code:java}
> // Carefully update noNulls...
> if (outputColVector.noNulls) {
> outputColVector.noNulls = inputColVector.noNulls;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18622:
Attachment: HIVE-18622.03.patch
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.03.patch
>
>
>
> Many vector expression classes are missing guards around setting noNulls
> among other things.
> {code:java}
> // Carefully update noNulls...
> if (outputColVector.noNulls) {
> outputColVector.noNulls = inputColVector.noNulls;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18622:
Status: Patch Available (was: In Progress)
Still a Work-In-Progress.
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.03.patch
>
>
>
> Many vector expression classes are missing guards around setting noNulls
> among other things.
> {code:java}
> // Carefully update noNulls...
> if (outputColVector.noNulls) {
> outputColVector.noNulls = inputColVector.noNulls;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18622:
Attachment: (was: HIVE-18622.01.patch)
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.03.patch
>
>
>
> Many vector expression classes are missing guards around setting noNulls
> among other things.
> {code:java}
> // Carefully update noNulls...
> if (outputColVector.noNulls) {
> outputColVector.noNulls = inputColVector.noNulls;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18622) Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly
[
https://issues.apache.org/jira/browse/HIVE-18622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-18622:
Status: In Progress (was: Patch Available)
> Vectorization: IF Statements, Comparisons, and more do not handle NULLs
> correctly
> -
>
> Key: HIVE-18622
> URL: https://issues.apache.org/jira/browse/HIVE-18622
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18622.01.patch, HIVE-18622.02.patch
>
>
>
> Many vector expression classes are missing guards around setting noNulls
> among other things.
> {code:java}
> // Carefully update noNulls...
> if (outputColVector.noNulls) {
> outputColVector.noNulls = inputColVector.noNulls;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355042#comment-16355042
]
Hive QA commented on HIVE-16605:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
47s{color} | {color:red} ql: The patch generated 60 new + 981 unchanged - 271
fixed = 1041 total (was 1252) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 2422e18 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9063/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9063/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch, HIVE-16605.5.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18626) Repl load "with" clause does not pass config to tasks
[ https://issues.apache.org/jira/browse/HIVE-18626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355025#comment-16355025 ] Daniel Dai commented on HIVE-18626: --- Address checkstyle issue. > Repl load "with" clause does not pass config to tasks > - > > Key: HIVE-18626 > URL: https://issues.apache.org/jira/browse/HIVE-18626 > Project: Hive > Issue Type: Bug > Components: repl >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-18626.1.patch, HIVE-18626.2.patch > > > The "with" clause in repl load suppose to pass custom hive config entries to > replication. However, the config is only effective in > BootstrapEventsIterator, but not the generated tasks (such as MoveTask, > DDLTask). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18626) Repl load "with" clause does not pass config to tasks
[ https://issues.apache.org/jira/browse/HIVE-18626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-18626: -- Attachment: HIVE-18626.2.patch > Repl load "with" clause does not pass config to tasks > - > > Key: HIVE-18626 > URL: https://issues.apache.org/jira/browse/HIVE-18626 > Project: Hive > Issue Type: Bug > Components: repl >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-18626.1.patch, HIVE-18626.2.patch > > > The "with" clause in repl load suppose to pass custom hive config entries to > replication. However, the config is only effective in > BootstrapEventsIterator, but not the generated tasks (such as MoveTask, > DDLTask). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18611) Avoid memory allocation of aggregation buffer during stats computation
[
https://issues.apache.org/jira/browse/HIVE-18611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355014#comment-16355014
]
Hive QA commented on HIVE-18611:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909331/HIVE-18611.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 12956 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9062/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9062/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9062/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909331 - PreCommit-HIVE-Build
> Avoid memory allocation of aggregation buffer during stats computation
> ---
>
> Key: HIVE-18611
> URL: https://issues.apache.org/jira/browse/HIVE-18611
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Statistics
>Affects Versions: 3.0.0, 2.3.2
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Labels: performance
> Attachments: HIVE-18611.2.patch, HIVE-18611.patch
>
>
> Bloom filter aggregation buffer may result in allocation of upto ~594MB array
> which is unnecessary.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18626) Repl load "with" clause does not pass config to tasks
[ https://issues.apache.org/jira/browse/HIVE-18626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354985#comment-16354985 ] Thejas M Nair commented on HIVE-18626: -- +1 pending above comments > Repl load "with" clause does not pass config to tasks > - > > Key: HIVE-18626 > URL: https://issues.apache.org/jira/browse/HIVE-18626 > Project: Hive > Issue Type: Bug > Components: repl >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-18626.1.patch > > > The "with" clause in repl load suppose to pass custom hive config entries to > replication. However, the config is only effective in > BootstrapEventsIterator, but not the generated tasks (such as MoveTask, > DDLTask). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18611) Avoid memory allocation of aggregation buffer during stats computation
[
https://issues.apache.org/jira/browse/HIVE-18611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354984#comment-16354984
]
Hive QA commented on HIVE-18611:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} storage-api: The patch generated 2 new + 7 unchanged -
2 fixed = 9 total (was 9) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 1 new + 146 unchanged - 1
fixed = 147 total (was 147) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
56s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 2422e18 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9062/yetus/diff-checkstyle-storage-api.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9062/yetus/diff-checkstyle-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9062/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: storage-api ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9062/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Avoid memory allocation of aggregation buffer during stats computation
> ---
>
> Key: HIVE-18611
> URL: https://issues.apache.org/jira/browse/HIVE-18611
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Statistics
>Affects Versions: 3.0.0, 2.3.2
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Labels: performance
> Attachments: HIVE-18611.2.patch, HIVE-18611.patch
>
>
> Bloom filter aggregation buffer may result in allocation of upto ~594MB array
> which is unnecessary.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354971#comment-16354971
]
Hive QA commented on HIVE-18625:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909329/HIVE-18625.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 12976 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.TestAcidTableSetup.testTransactionalValidation
(batchId=223)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConnectionSchemaAPIs (batchId=238)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testHttpHeaderSize (batchId=238)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerDAGTotalTasks
(batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9061/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9061/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9061/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 27 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909329 - PreCommit-HIVE-Build
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation
[jira] [Updated] (HIVE-18467) support whole warehouse dump / load + create/drop database events
[ https://issues.apache.org/jira/browse/HIVE-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] anishek updated HIVE-18467: --- Resolution: Fixed Status: Resolved (was: Patch Available) > support whole warehouse dump / load + create/drop database events > - > > Key: HIVE-18467 > URL: https://issues.apache.org/jira/browse/HIVE-18467 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-18467.0.patch, HIVE-18467.1.patch, > HIVE-18467.2.patch, HIVE-18467.5.patch, HIVE-18467.6.patch > > > A complete hive warehouse might be required to replicate to a DR site for > certain use cases and rather than allowing only a database name in the REPL > DUMP commands, we should allow dumping of all databases using the "*" option > as in > _REPL DUMP *_ > On the repl load side there will not be an option to specify the database > name when loading from a location used to dump multiple databases, hence only > _REPL LOAD FROM [location]_ would be supported when dumping via _REPL DUMP *_ > Additionally, incremental dumps will go through all events across databases > in a warehouse and hence CREATE / DROP Database events have to be serialized > correctly to allow repl load to create them correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18467) support whole warehouse dump / load + create/drop database events
[ https://issues.apache.org/jira/browse/HIVE-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354952#comment-16354952 ] anishek commented on HIVE-18467: Thanks [~sankarh] for the review, the tests failures are not relevant to this patch. Patch is committed to master ! > support whole warehouse dump / load + create/drop database events > - > > Key: HIVE-18467 > URL: https://issues.apache.org/jira/browse/HIVE-18467 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-18467.0.patch, HIVE-18467.1.patch, > HIVE-18467.2.patch, HIVE-18467.5.patch, HIVE-18467.6.patch > > > A complete hive warehouse might be required to replicate to a DR site for > certain use cases and rather than allowing only a database name in the REPL > DUMP commands, we should allow dumping of all databases using the "*" option > as in > _REPL DUMP *_ > On the repl load side there will not be an option to specify the database > name when loading from a location used to dump multiple databases, hence only > _REPL LOAD FROM [location]_ would be supported when dumping via _REPL DUMP *_ > Additionally, incremental dumps will go through all events across databases > in a warehouse and hence CREATE / DROP Database events have to be serialized > correctly to allow repl load to create them correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18192) Introduce WriteID per table rather than using global transaction ID
[ https://issues.apache.org/jira/browse/HIVE-18192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354062#comment-16354062 ] Sankar Hariappan edited comment on HIVE-18192 at 2/7/18 4:18 AM: - Added 09.patch with below updates. # Remove entries from TXNS_TO_WRITE_ID table when drop a table/database. Also, split db and table names into 2 columns. # Multi-statement txn support by setting ValidTxnList for open txn and get/set ValidWriteId list for each operation within txn. Pending changes for this feature # Scripts to add Metastore tables for write id management. Add for other non-derby databases too. # CompactionTxnHandler cleanup COMPLETED_TXN_COMPONENTS based on highest write id instead of highest txn id. # Address review comments from Eugene. # Address few more test failures. # Fix check style issues. Out of scope of HIVE-18192: # Cleaner for TXNS_TO_WRITE_ID table entries. Also, need to maintain the LWM for each table write id. # Update classes to use WriteID instead of TxnId as methods and variable names. was (Author: sankarh): Added 09.patch with below updates. # Remove entries from TXNS_TO_WRITE_ID table when drop a table/database. Also, split db and table names into 2 columns. # Multi-statement txn support by setting ValidTxnList for open txn and get/set ValidWriteId list for each operation within txn. Pending changes for this feature # Scripts to add Metastore tables for write id management. Add for other non-derby databases too. # CompactionTxnHandler cleanup COMPLETED_TXN_COMPONENTS based on highest write id instead of highest txn id. # Address few more test failures. # Fix check style issues. Out of scope of HIVE-18192: # Cleaner for TXNS_TO_WRITE_ID table entries. Also, need to maintain the LWM for each table write id. # Update classes to use WriteID instead of TxnId as methods and variable names. > Introduce WriteID per table rather than using global transaction ID > --- > > Key: HIVE-18192 > URL: https://issues.apache.org/jira/browse/HIVE-18192 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, DR, pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-18192.01.patch, HIVE-18192.02.patch, > HIVE-18192.03.patch, HIVE-18192.04.patch, HIVE-18192.05.patch, > HIVE-18192.06.patch, HIVE-18192.07.patch, HIVE-18192.08.patch, > HIVE-18192.09.patch > > > To support ACID replication, we will be introducing a per table write Id > which will replace the transaction id in the primary key for each row in a > ACID table. > The current primary key is determined via > > which will move to > > For each table modified by the given transaction will have a table level > write ID allocated and a persisted map of global txn id -> to table -> write > id for that table has to be maintained to allow Snapshot isolation. > Readers should use the combination of ValidTxnList and > ValidWriteIdList(Table) for snapshot isolation. > > [Hive Replication - ACID > Tables.pdf|https://issues.apache.org/jira/secure/attachment/12903157/Hive%20Replication-%20ACID%20Tables.pdf] > has a section "Per Table Sequences (Write-Id)" with more detials -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18636) fix TestTxnNoBuckets.testCTAS - keeps failing on ptest
[ https://issues.apache.org/jira/browse/HIVE-18636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18636: -- Status: Patch Available (was: Open) [~sershe] could you review please > fix TestTxnNoBuckets.testCTAS - keeps failing on ptest > -- > > Key: HIVE-18636 > URL: https://issues.apache.org/jira/browse/HIVE-18636 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Blocker > Attachments: HIVE-18636.01.patch > > > need to update expected result -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18636) fix TestTxnNoBuckets.testCTAS - keeps failing on ptest
[ https://issues.apache.org/jira/browse/HIVE-18636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-18636: -- Attachment: HIVE-18636.01.patch > fix TestTxnNoBuckets.testCTAS - keeps failing on ptest > -- > > Key: HIVE-18636 > URL: https://issues.apache.org/jira/browse/HIVE-18636 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Eugene Koifman >Priority: Blocker > Attachments: HIVE-18636.01.patch > > > need to update expected result -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18625) SessionState Not Checking For Directory Creation Result
[
https://issues.apache.org/jira/browse/HIVE-18625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354943#comment-16354943
]
Hive QA commented on HIVE-18625:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 1 new + 75 unchanged - 0 fixed
= 76 total (was 75) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3972bf0 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9061/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9061/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SessionState Not Checking For Directory Creation Result
> ---
>
> Key: HIVE-18625
> URL: https://issues.apache.org/jira/browse/HIVE-18625
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: BELUGA BEHR
>Assignee: Andrew Sherman
>Priority: Minor
> Attachments: HIVE-18625.1.patch
>
>
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java#L773
> {code:java}
> private static void createPath(HiveConf conf, Path path, String permission,
> boolean isLocal,
> boolean isCleanUp) throws IOException {
> FsPermission fsPermission = new FsPermission(permission);
> FileSystem fs;
> if (isLocal) {
> fs = FileSystem.getLocal(conf);
> } else {
> fs = path.getFileSystem(conf);
> }
> if (!fs.exists(path)) {
> fs.mkdirs(path, fsPermission);
> String dirType = isLocal ? "local" : "HDFS";
> LOG.info("Created " + dirType + " directory: " + path.toString());
> }
> if (isCleanUp) {
> fs.deleteOnExit(path);
> }
> }
> {code}
> The method {{fs.mkdirs(path, fsPermission)}} returns a boolean value
> indicating if the directory creation was successful or not. Hive ignores
> this return value and therefore could be acting on a directory that doesn't
> exist.
> Please capture the result, check it, and throw an Exception if it failed
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18467) support whole warehouse dump / load + create/drop database events
[ https://issues.apache.org/jira/browse/HIVE-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354925#comment-16354925 ] Sankar Hariappan commented on HIVE-18467: - +1 Patch looks good to me. Also, the test failures seems irrelevant. > support whole warehouse dump / load + create/drop database events > - > > Key: HIVE-18467 > URL: https://issues.apache.org/jira/browse/HIVE-18467 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: anishek >Assignee: anishek >Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > Attachments: HIVE-18467.0.patch, HIVE-18467.1.patch, > HIVE-18467.2.patch, HIVE-18467.5.patch, HIVE-18467.6.patch > > > A complete hive warehouse might be required to replicate to a DR site for > certain use cases and rather than allowing only a database name in the REPL > DUMP commands, we should allow dumping of all databases using the "*" option > as in > _REPL DUMP *_ > On the repl load side there will not be an option to specify the database > name when loading from a location used to dump multiple databases, hence only > _REPL LOAD FROM [location]_ would be supported when dumping via _REPL DUMP *_ > Additionally, incremental dumps will go through all events across databases > in a warehouse and hence CREATE / DROP Database events have to be serialized > correctly to allow repl load to create them correctly. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18553) VectorizedParquetReader fails after adding a new column to table
[
https://issues.apache.org/jira/browse/HIVE-18553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354916#comment-16354916
]
Hive QA commented on HIVE-18553:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909348/test_result_based_on_HIVE-18553.xlsx
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9060/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9060/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9060/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-02-07 03:40:44.316
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-9060/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-02-07 03:40:44.320
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 3972bf0 HIVE-18613: Extend JsonSerDe to support BINARY type
(Jesus Camacho Rodriguez, reviewed by Prasanth Jayachandran)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 3972bf0 HIVE-18613: Extend JsonSerDe to support BINARY type
(Jesus Camacho Rodriguez, reviewed by Prasanth Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-02-07 03:40:46.829
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-9060/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
fatal: unrecognized input
fatal: unrecognized input
fatal: unrecognized input
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909348 - PreCommit-HIVE-Build
> VectorizedParquetReader fails after adding a new column to table
>
>
> Key: HIVE-18553
> URL: https://issues.apache.org/jira/browse/HIVE-18553
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0, 2.4.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Ferdinand Xu
>Priority: Major
> Attachments: HIVE-18553.2.patch, HIVE-18553.3.patch,
> HIVE-18553.4.patch, HIVE-18553.patch, test_result_based_on_HIVE-18553.xlsx
>
>
> VectorizedParquetReader throws an exception when trying to reading from a
> parquet table on which new columns are added. Steps to reproduce below:
> {code}
> 0: jdbc:hive2://localhost:1/default> desc test_p;
> +---++--+
> | col_name | data_type | comment |
> +---++--+
> | t1| tinyint| |
> | t2| tinyint| |
> | i1| int| |
> | i2| int| |
> +---++--+
> 0: jdbc:hive2://localhost:1/default> set hive.fetch.task.conversion=none;
> 0: jdbc:hive2://localhost:1/default> set
> hive.vectorized.execution.enabled=true;
> 0: jdbc:hive2://localhost:1/default> alter table test_p add columns (ts
> timestamp);
> 0: jdbc:hive2://localhost:1/default> select * from test_p;
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2)
> {code}
> Following exception is seen in the logs
> {code}
> Caused by: java.lang.IllegalArgumentException: [ts] BINARY is not in the
> store: [[i1] INT32, [i2] INT32, [t1] INT32,
[jira] [Commented] (HIVE-18629) copyValues in BytesColumnVector may be missing null checks
[
https://issues.apache.org/jira/browse/HIVE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354914#comment-16354914
]
Hive QA commented on HIVE-18629:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909347/HIVE-18629.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 22 failed/errored test(s), 12974 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestAddAlterDropIndexes.testDropIndexInvalidDB[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9059/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9059/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9059/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 22 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909347 - PreCommit-HIVE-Build
> copyValues in BytesColumnVector may be missing null checks
> --
>
> Key: HIVE-18629
> URL: https://issues.apache.org/jira/browse/HIVE-18629
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18629.01.patch, HIVE-18629.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:333)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions..evaluate(IfExprStringGroupColumnStringGroupColumn.java:83)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:133)
> {noformat}
> IfExprStringGroupColumnStringGroupColumn code below the v1.isRepeating case
> has isNull checks for v2/v3 buffers that copySelected is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18623) Hive throws an exception "Renames across Mount points not supported" when running in a federated cluster
[ https://issues.apache.org/jira/browse/HIVE-18623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangfang updated HIVE-18623: Affects Version/s: (was: 2.3.1) (was: 2.3.0) 2.3.3 3.0.0 > Hive throws an exception "Renames across Mount points not supported" when > running in a federated cluster > > > Key: HIVE-18623 > URL: https://issues.apache.org/jira/browse/HIVE-18623 > Project: Hive > Issue Type: Bug > Components: Query Processor >Affects Versions: 2.2.0, 3.0.0, 2.3.2, 2.3.3 > Environment: hadoop 2.7.5, HDFS Federation enabled > hive 3.0.0 >Reporter: yangfang >Assignee: yangfang >Priority: Major > Attachments: HIVE-18623.1.patch > > > > I run a sql query in in a federated cluster and I have two namespaces: > nameservice and nameservice1. I set > hive.exec.stagingdir=/nameservice1/hive_tmp in hive-site.xml and my data > tables are located in the directory of nameservice, then I got the exception > as below: > hive> create external table test_par6(id int,name string) partitioned by(p > int); > OK > Time taken: 1.527 seconds > hive> insert into table test_par6 partition(p = 1) values(1,'Jack'); > Moving data to directory > viewfs://nsX/nameservice1/hive_tmp_hive_2018-02-05_14-09-36_416_3075179128063595297-1/-ext-1 > Loading data to table default.test_par6 partition (p=1) > Failed with exception java.io.IOException: Renames across Mount points not > supported > FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.MoveTask. java.io.IOException: Renames across > Mount points not supported > MapReduce Jobs Launched: > Stage-Stage-1: Map: 1 Cumulative CPU: 2.08 sec HDFS Read: 3930 HDFS Write: 7 > SUCCESS > Total MapReduce CPU Time Spent: 2 seconds 80 msec -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15911) Create a view based on another view throws an exception “FAILED: NullPointerException null”
[ https://issues.apache.org/jira/browse/HIVE-15911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangfang updated HIVE-15911: Affects Version/s: (was: 3.0.0) 2.1.1 > Create a view based on another view throws an exception “FAILED: > NullPointerException null” > --- > > Key: HIVE-15911 > URL: https://issues.apache.org/jira/browse/HIVE-15911 > Project: Hive > Issue Type: Bug > Components: Views >Affects Versions: 2.1.1 > Environment: hive2.1.0 >Reporter: yangfang >Assignee: yangfang >Priority: Major > Attachments: 0001-Hive-15911.patch > > > when I create a new view based on another view I an exception “FAILED: > NullPointerException null”: > hive> create view view2(a,b) as select a, b from view1; //view1 is another > view > FAILED: NullPointerException null > hive> > The hive log show error stack： > 2017-02-15T15:40:25,816 ERROR ql.Driver (SessionState.java:printError(1116)) > - FAILED: NullPointerException null > java.lang.NullPointerException > at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:863) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:552) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1319) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1459) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1239) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1229) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) > at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-15911) Create a view based on another view throws an exception “FAILED: NullPointerException null”
[ https://issues.apache.org/jira/browse/HIVE-15911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yangfang updated HIVE-15911: Affects Version/s: (was: 2.1.1) (was: 2.1.0) 3.0.0 > Create a view based on another view throws an exception “FAILED: > NullPointerException null” > --- > > Key: HIVE-15911 > URL: https://issues.apache.org/jira/browse/HIVE-15911 > Project: Hive > Issue Type: Bug > Components: Views >Affects Versions: 3.0.0 > Environment: hive2.1.0 >Reporter: yangfang >Assignee: yangfang >Priority: Major > Attachments: 0001-Hive-15911.patch > > > when I create a new view based on another view I an exception “FAILED: > NullPointerException null”: > hive> create view view2(a,b) as select a, b from view1; //view1 is another > view > FAILED: NullPointerException null > hive> > The hive log show error stack： > 2017-02-15T15:40:25,816 ERROR ql.Driver (SessionState.java:printError(1116)) > - FAILED: NullPointerException null > java.lang.NullPointerException > at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:863) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:552) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1319) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1459) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1239) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1229) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) > at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18629) copyValues in BytesColumnVector may be missing null checks
[
https://issues.apache.org/jira/browse/HIVE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354888#comment-16354888
]
Hive QA commented on HIVE-18629:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} storage-api: The patch generated 1 new + 4 unchanged -
2 fixed = 5 total (was 6) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 1 new + 3 unchanged - 0 fixed
= 4 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3972bf0 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9059/yetus/diff-checkstyle-storage-api.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9059/yetus/diff-checkstyle-ql.txt
|
| modules | C: storage-api ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9059/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> copyValues in BytesColumnVector may be missing null checks
> --
>
> Key: HIVE-18629
> URL: https://issues.apache.org/jira/browse/HIVE-18629
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18629.01.patch, HIVE-18629.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:333)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions..evaluate(IfExprStringGroupColumnStringGroupColumn.java:83)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:133)
> {noformat}
> IfExprStringGroupColumnStringGroupColumn code below the v1.isRepeating case
> has isNull checks for v2/v3 buffers that
[jira] [Updated] (HIVE-18629) copyValues in BytesColumnVector may be missing null checks
[
https://issues.apache.org/jira/browse/HIVE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18629:
Attachment: HIVE-18629.01.patch
> copyValues in BytesColumnVector may be missing null checks
> --
>
> Key: HIVE-18629
> URL: https://issues.apache.org/jira/browse/HIVE-18629
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18629.01.patch, HIVE-18629.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:333)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions..evaluate(IfExprStringGroupColumnStringGroupColumn.java:83)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:133)
> {noformat}
> IfExprStringGroupColumnStringGroupColumn code below the v1.isRepeating case
> has isNull checks for v2/v3 buffers that copySelected is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18629) copyValues in BytesColumnVector may be missing null checks
[
https://issues.apache.org/jira/browse/HIVE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354880#comment-16354880
]
Sergey Shelukhin commented on HIVE-18629:
-
[~vihangk1] updated the patch.
> copyValues in BytesColumnVector may be missing null checks
> --
>
> Key: HIVE-18629
> URL: https://issues.apache.org/jira/browse/HIVE-18629
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18629.01.patch, HIVE-18629.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:333)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions..evaluate(IfExprStringGroupColumnStringGroupColumn.java:83)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:133)
> {noformat}
> IfExprStringGroupColumnStringGroupColumn code below the v1.isRepeating case
> has isNull checks for v2/v3 buffers that copySelected is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-15353) Metastore throws NPE if StorageDescriptor.cols is null
[
https://issues.apache.org/jira/browse/HIVE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354874#comment-16354874
]
Hive QA commented on HIVE-15353:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909352/HIVE-15353.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 26 failed/errored test(s), 12974 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.TestMarkPartition.testMarkingPartitionSet
(batchId=215)
org.apache.hadoop.hive.metastore.client.TestTablesGetExists.testGetAllTablesCaseInsensitive[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testHttpRetryOnServerIdleTimeout
(batchId=238)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9058/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9058/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9058/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 26 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909352 - PreCommit-HIVE-Build
> Metastore throws NPE if StorageDescriptor.cols is null
> --
>
> Key: HIVE-15353
> URL: https://issues.apache.org/jira/browse/HIVE-15353
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.2.0
>Reporter: Anthony Hsu
>Assignee: Anthony Hsu
>Priority: Major
> Attachments: HIVE-15353.1.patch, HIVE-15353.2.patch,
> HIVE-15353.3.patch, HIVE-15353.4.patch, HIVE-15353.5.patch
>
>
> When using the HiveMetaStoreClient API directly to talk to the metastore, you
> get NullPointerExceptions when StorageDescriptor.cols is null in the
> Table/Partition object in the following calls:
> * create_table
> * alter_table
> * alter_partition
> Calling add_partition with StorageDescriptor.cols set to null causes null to
> be stored in the metastore database and subsequent calls to alter_partition
> for that partition to fail with an NPE.
> Null checks should be added to eliminate the NPEs in the metastore.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18340) Dynamic Min-Max/BloomFilter runtime-filtering in HoS
[ https://issues.apache.org/jira/browse/HIVE-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354871#comment-16354871 ] Ke Jia commented on HIVE-18340: --- [~stakiar] : >Have we done any performance analysis for this feature? We have done the benchmark in TPC-DS with this feature in HoT and get improvement(+20%) in query82/88 and (+10%) in query34/90. Now, design doc and HIVE-18340.1.patch is the initial implementation in HoS. Can you help to review the design doc and HIVE-18340.1.patch? Thanks for your help! > Dynamic Min-Max/BloomFilter runtime-filtering in HoS > > > Key: HIVE-18340 > URL: https://issues.apache.org/jira/browse/HIVE-18340 > Project: Hive > Issue Type: New Feature > Components: Spark >Affects Versions: 3.0.0 >Reporter: Ke Jia >Assignee: Ke Jia >Priority: Major > Attachments: HIVE-18340.1.patch > > > Tez implemented Dynamic Min-Max/BloomFilter runtime-filtering in HIVE-15269 > and we should implement the same in HOS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18619) Verification of temporary Micromanaged table atomicity is needed
[ https://issues.apache.org/jira/browse/HIVE-18619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354857#comment-16354857 ] Steve Yeom commented on HIVE-18619: --- Talked with Vineet regarding generating failure in INSERT. Probably using debugger might be the best option. > Verification of temporary Micromanaged table atomicity is needed > - > > Key: HIVE-18619 > URL: https://issues.apache.org/jira/browse/HIVE-18619 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Minor > > Session based temporary table by HIVE-7090 had no consideration of > Micromanaged table > (MM) since there was no insert-only ACID table at its creation tije. > HIVE-18599 addressed the issue of no writes during CTTAS (Create Temporary > Table As Select) > on Micro-Managed table. But atomicity of temporary MM table is not verified. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18421) Vectorized execution handles overflows in a different manner than non-vectorized execution
[
https://issues.apache.org/jira/browse/HIVE-18421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-18421:
---
Attachment: (was: HIVE-18421.06.patch)
> Vectorized execution handles overflows in a different manner than
> non-vectorized execution
> --
>
> Key: HIVE-18421
> URL: https://issues.apache.org/jira/browse/HIVE-18421
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 2.1.1, 2.2.0, 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-18421.01.patch, HIVE-18421.02.patch,
> HIVE-18421.03.patch, HIVE-18421.04.patch, HIVE-18421.05.patch,
> HIVE-18421.06.patch
>
>
> In vectorized execution arithmetic operations which cause integer overflows
> can give wrong results. Issue is reproducible in both Orc and parquet.
> Simple test case to reproduce this issue
> {noformat}
> set hive.vectorized.execution.enabled=true;
> create table parquettable (t1 tinyint, t2 tinyint) stored as parquet;
> insert into parquettable values (-104, 25), (-112, 24), (54, 9);
> select t1, t2, (t1-t2) as diff from parquettable where (t1-t2) < 50 order by
> diff desc;
> +---+-+---+
> | t1 | t2 | diff |
> +---+-+---+
> | -104 | 25 | 127 |
> | -112 | 24 | 120 |
> | 54| 9 | 45|
> +---+-+---+
> {noformat}
> When vectorization is turned off the same query produces only one row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18421) Vectorized execution handles overflows in a different manner than non-vectorized execution
[
https://issues.apache.org/jira/browse/HIVE-18421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-18421:
---
Attachment: HIVE-18421.06.patch
> Vectorized execution handles overflows in a different manner than
> non-vectorized execution
> --
>
> Key: HIVE-18421
> URL: https://issues.apache.org/jira/browse/HIVE-18421
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 2.1.1, 2.2.0, 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-18421.01.patch, HIVE-18421.02.patch,
> HIVE-18421.03.patch, HIVE-18421.04.patch, HIVE-18421.05.patch,
> HIVE-18421.06.patch
>
>
> In vectorized execution arithmetic operations which cause integer overflows
> can give wrong results. Issue is reproducible in both Orc and parquet.
> Simple test case to reproduce this issue
> {noformat}
> set hive.vectorized.execution.enabled=true;
> create table parquettable (t1 tinyint, t2 tinyint) stored as parquet;
> insert into parquettable values (-104, 25), (-112, 24), (54, 9);
> select t1, t2, (t1-t2) as diff from parquettable where (t1-t2) < 50 order by
> diff desc;
> +---+-+---+
> | t1 | t2 | diff |
> +---+-+---+
> | -104 | 25 | 127 |
> | -112 | 24 | 120 |
> | 54| 9 | 45|
> +---+-+---+
> {noformat}
> When vectorization is turned off the same query produces only one row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18643) don't check for archived partitions for ACID ops
[ https://issues.apache.org/jira/browse/HIVE-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354840#comment-16354840 ] Eugene Koifman commented on HIVE-18643: --- +1 > don't check for archived partitions for ACID ops > > > Key: HIVE-18643 > URL: https://issues.apache.org/jira/browse/HIVE-18643 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18643.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18643) don't check for archived partitions for ACID ops
[ https://issues.apache.org/jira/browse/HIVE-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18643: Attachment: (was: HIVE-18643.patch) > don't check for archived partitions for ACID ops > > > Key: HIVE-18643 > URL: https://issues.apache.org/jira/browse/HIVE-18643 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18643.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18643) don't check for archived partitions for ACID ops
[ https://issues.apache.org/jira/browse/HIVE-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18643: Attachment: HIVE-18643.patch > don't check for archived partitions for ACID ops > > > Key: HIVE-18643 > URL: https://issues.apache.org/jira/browse/HIVE-18643 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18643.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-15353) Metastore throws NPE if StorageDescriptor.cols is null
[
https://issues.apache.org/jira/browse/HIVE-15353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354829#comment-16354829
]
Hive QA commented on HIVE-15353:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3972bf0 |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9058/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore throws NPE if StorageDescriptor.cols is null
> --
>
> Key: HIVE-15353
> URL: https://issues.apache.org/jira/browse/HIVE-15353
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0, 2.2.0
>Reporter: Anthony Hsu
>Assignee: Anthony Hsu
>Priority: Major
> Attachments: HIVE-15353.1.patch, HIVE-15353.2.patch,
> HIVE-15353.3.patch, HIVE-15353.4.patch, HIVE-15353.5.patch
>
>
> When using the HiveMetaStoreClient API directly to talk to the metastore, you
> get NullPointerExceptions when StorageDescriptor.cols is null in the
> Table/Partition object in the following calls:
> * create_table
> * alter_table
> * alter_partition
> Calling add_partition with StorageDescriptor.cols set to null causes null to
> be stored in the metastore database and subsequent calls to alter_partition
> for that partition to fail with an NPE.
> Null checks should be added to eliminate the NPEs in the metastore.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18643) don't check for archived partitions for ACID ops
[ https://issues.apache.org/jira/browse/HIVE-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18643: Status: Patch Available (was: Open) [~ekoifman] can you take a look? tiny patch for some bug > don't check for archived partitions for ACID ops > > > Key: HIVE-18643 > URL: https://issues.apache.org/jira/browse/HIVE-18643 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18643.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18643) don't check for archived partitions for ACID ops
[ https://issues.apache.org/jira/browse/HIVE-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18643: Attachment: HIVE-18643.patch > don't check for archived partitions for ACID ops > > > Key: HIVE-18643 > URL: https://issues.apache.org/jira/browse/HIVE-18643 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18643.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18643) don't check for archived partitions for ACID ops
[ https://issues.apache.org/jira/browse/HIVE-18643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18643: --- > don't check for archived partitions for ACID ops > > > Key: HIVE-18643 > URL: https://issues.apache.org/jira/browse/HIVE-18643 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.
[ https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354821#comment-16354821 ] Alan Gates commented on HIVE-17983: --- Patch 4 is identical to patch 3, uploaded again in order to re-run tests. > Make the standalone metastore generate tarballs etc. > > > Key: HIVE-17983 > URL: https://issues.apache.org/jira/browse/HIVE-17983 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Labels: pull-request-available > Attachments: HIVE-17983.2.patch, HIVE-17983.3.patch, > HIVE-17983.4.patch, HIVE-17983.patch > > > In order to be separately installable the standalone metastore needs its own > tarballs, startup scripts, etc. All of the SQL installation and upgrade > scripts also need to move from metastore to standalone-metastore. > I also plan to create Dockerfiles for different database types so that > developers can test the SQL installation and upgrade scripts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-17983) Make the standalone metastore generate tarballs etc.
[ https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-17983: -- Attachment: HIVE-17983.4.patch > Make the standalone metastore generate tarballs etc. > > > Key: HIVE-17983 > URL: https://issues.apache.org/jira/browse/HIVE-17983 > Project: Hive > Issue Type: Sub-task > Components: Standalone Metastore >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Labels: pull-request-available > Attachments: HIVE-17983.2.patch, HIVE-17983.3.patch, > HIVE-17983.4.patch, HIVE-17983.patch > > > In order to be separately installable the standalone metastore needs its own > tarballs, startup scripts, etc. All of the SQL installation and upgrade > scripts also need to move from metastore to standalone-metastore. > I also plan to create Dockerfiles for different database types so that > developers can test the SQL installation and upgrade scripts. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18573) Use proper Calcite operator instead of UDFs
[ https://issues.apache.org/jira/browse/HIVE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra reassigned HIVE-18573: - Assignee: Ashutosh Chauhan > Use proper Calcite operator instead of UDFs > --- > > Key: HIVE-18573 > URL: https://issues.apache.org/jira/browse/HIVE-18573 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: slim bouguerra >Assignee: Ashutosh Chauhan >Priority: Major > Attachments: HIVE-18573.2.patch, HIVE-18573.3.patch, > HIVE-18573.4.patch, HIVE-18573.5.patch, HIVE-18573.6.patch, > HIVE-18573.7.patch, HIVE-18573.7.patch, HIVE-18573.patch > > > Currently, Hive is mostly using user-defined black box sql operators during > Query planning. It will be more beneficial to use proper calcite operators, > this is to prepare the ground for pushing complex expressions to > Druid-calcite adapter. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18573) Use proper Calcite operator instead of UDFs
[ https://issues.apache.org/jira/browse/HIVE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-18573: -- Attachment: HIVE-18573.7.patch > Use proper Calcite operator instead of UDFs > --- > > Key: HIVE-18573 > URL: https://issues.apache.org/jira/browse/HIVE-18573 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: slim bouguerra >Priority: Major > Attachments: HIVE-18573.2.patch, HIVE-18573.3.patch, > HIVE-18573.4.patch, HIVE-18573.5.patch, HIVE-18573.6.patch, > HIVE-18573.7.patch, HIVE-18573.7.patch, HIVE-18573.patch > > > Currently, Hive is mostly using user-defined black box sql operators during > Query planning. It will be more beneficial to use proper calcite operators, > this is to prepare the ground for pushing complex expressions to > Druid-calcite adapter. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18595) UNIX_TIMESTAMP UDF fails when type is Timestamp with local timezone
[
https://issues.apache.org/jira/browse/HIVE-18595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-18595:
--
Attachment: HIVE-18595.patch
> UNIX_TIMESTAMP UDF fails when type is Timestamp with local timezone
>
>
> Key: HIVE-18595
> URL: https://issues.apache.org/jira/browse/HIVE-18595
> Project: Hive
> Issue Type: Bug
>Reporter: slim bouguerra
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-18595.patch, HIVE-18595.patch
>
>
> {code}
> 2018-01-31T12:59:45,464 ERROR [10e97c86-7f90-406b-a8fa-38be5d3529cc main]
> ql.Driver: FAILED: SemanticException [Error 10014]: Line 3:456 Wrong
> arguments ''-MM-dd HH:mm:ss'': The function UNIX_TIMESTAMP takes only
> string/date/timestamp types
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 3:456 Wrong arguments
> ''-MM-dd HH:mm:ss'': The function UNIX_TIMESTAMP takes only
> string/date/timestamp types
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1394)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
> at
> org.apache.hadoop.hive.ql.lib.ExpressionWalker.walk(ExpressionWalker.java:76)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:235)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:181)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:11847)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:11780)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genGBLogicalPlan(CalcitePlanner.java:3140)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4330)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1407)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1354)
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:1052)
> at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:154)
> at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:111)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1159)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1175)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:422)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11393)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:304)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:268)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:163)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:268)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:639)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1504)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1632)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1395)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1382)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:240)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:343)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1331)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1305)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:173)
> at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> at
> org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver(TestMiniDruidCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
[jira] [Assigned] (HIVE-18595) UNIX_TIMESTAMP UDF fails when type is Timestamp with local timezone
[
https://issues.apache.org/jira/browse/HIVE-18595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra reassigned HIVE-18595:
-
Assignee: Jesus Camacho Rodriguez
> UNIX_TIMESTAMP UDF fails when type is Timestamp with local timezone
>
>
> Key: HIVE-18595
> URL: https://issues.apache.org/jira/browse/HIVE-18595
> Project: Hive
> Issue Type: Bug
>Reporter: slim bouguerra
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-18595.patch, HIVE-18595.patch
>
>
> {code}
> 2018-01-31T12:59:45,464 ERROR [10e97c86-7f90-406b-a8fa-38be5d3529cc main]
> ql.Driver: FAILED: SemanticException [Error 10014]: Line 3:456 Wrong
> arguments ''-MM-dd HH:mm:ss'': The function UNIX_TIMESTAMP takes only
> string/date/timestamp types
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 3:456 Wrong arguments
> ''-MM-dd HH:mm:ss'': The function UNIX_TIMESTAMP takes only
> string/date/timestamp types
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory$DefaultExprProcessor.process(TypeCheckProcFactory.java:1394)
> at
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
> at
> org.apache.hadoop.hive.ql.lib.ExpressionWalker.walk(ExpressionWalker.java:76)
> at
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:120)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:235)
> at
> org.apache.hadoop.hive.ql.parse.TypeCheckProcFactory.genExprNode(TypeCheckProcFactory.java:181)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:11847)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:11780)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genGBLogicalPlan(CalcitePlanner.java:3140)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4330)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1407)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1354)
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:1052)
> at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:154)
> at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:111)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1159)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1175)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:422)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11393)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:304)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:268)
> at
> org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:163)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:268)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:639)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1504)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1632)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1395)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1382)
> at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:240)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:188)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:410)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:343)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1331)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1305)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:173)
> at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:104)
> at
> org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver(TestMiniDruidCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
[jira] [Commented] (HIVE-18467) support whole warehouse dump / load + create/drop database events
[
https://issues.apache.org/jira/browse/HIVE-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354815#comment-16354815
]
Hive QA commented on HIVE-18467:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909355/HIVE-18467.6.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 12973 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[bucketizedhiveinputformat]
(batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hadoop.hive.ql.parse.TestQBCompact.org.apache.hadoop.hive.ql.parse.TestQBCompact
(batchId=275)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9057/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9057/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9057/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909355 - PreCommit-HIVE-Build
> support whole warehouse dump / load + create/drop database events
> -
>
> Key: HIVE-18467
> URL: https://issues.apache.org/jira/browse/HIVE-18467
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0
>
> Attachments: HIVE-18467.0.patch, HIVE-18467.1.patch,
> HIVE-18467.2.patch, HIVE-18467.5.patch, HIVE-18467.6.patch
>
>
> A complete hive warehouse might be required to replicate to a DR site for
> certain use cases and rather than allowing only a database name in the REPL
> DUMP commands, we should allow dumping of all databases using the "*" option
> as in
> _REPL DUMP *_
> On the repl load side there will not be an option to specify the database
> name when loading from a location used to dump multiple databases, hence only
> _REPL LOAD FROM [location]_ would be supported when dumping via _REPL DUMP *_
> Additionally, incremental dumps will go through all events across databases
> in a warehouse and hence CREATE / DROP Database events have to be serialized
> correctly to allow repl load to create them correctly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17735) ObjectStore.addNotificationEvent is leaking queries
[
https://issues.apache.org/jira/browse/HIVE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-17735:
Status: Patch Available (was: In Progress)
> ObjectStore.addNotificationEvent is leaking queries
> ---
>
> Key: HIVE-17735
> URL: https://issues.apache.org/jira/browse/HIVE-17735
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-17735.1.patch, HIVE-17735.2.patch
>
>
> In ObjectStore.addNotificationEvent():
> {code}
> Query objectQuery = pm.newQuery(MNotificationNextId.class);
> Collection ids = (Collection)
> objectQuery.execute();
> {code}
> The query is never closed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17735) ObjectStore.addNotificationEvent is leaking queries
[
https://issues.apache.org/jira/browse/HIVE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-17735:
Status: In Progress (was: Patch Available)
> ObjectStore.addNotificationEvent is leaking queries
> ---
>
> Key: HIVE-17735
> URL: https://issues.apache.org/jira/browse/HIVE-17735
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-17735.1.patch, HIVE-17735.2.patch
>
>
> In ObjectStore.addNotificationEvent():
> {code}
> Query objectQuery = pm.newQuery(MNotificationNextId.class);
> Collection ids = (Collection)
> objectQuery.execute();
> {code}
> The query is never closed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18328) Improve schematool validator to report duplicate rows for column statistics
[
https://issues.apache.org/jira/browse/HIVE-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354796#comment-16354796
]
Aihua Xu commented on HIVE-18328:
-
[~ngangam] Sorry that I missed this one. Do you think we can also add similar
validation for PART_COL_STATS as well?
> Improve schematool validator to report duplicate rows for column statistics
> ---
>
> Key: HIVE-18328
> URL: https://issues.apache.org/jira/browse/HIVE-18328
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 2.1.1
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-18328.patch
>
>
> By design, in the {{TAB_COL_STATS}} table of the HMS schema, there should be
> ONE AND ONLY ONE row, representing its statistics, for each column defined in
> hive. A combination of DB_NAME, TABLE_NAME and COLUMN_NAME constitute a
> primary key/unique row.
> Each time the statistics are computed for a column, this row is updated.
> However, if somehow via BDR/replication process, we end up with multiple
> rows in this table for a given column, HMS server to recompute the statistics
> there after.
> So it would be good to detect this data anamoly via the schema validation
> tool.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18421) Vectorized execution handles overflows in a different manner than non-vectorized execution
[
https://issues.apache.org/jira/browse/HIVE-18421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-18421:
---
Attachment: HIVE-18421.06.patch
> Vectorized execution handles overflows in a different manner than
> non-vectorized execution
> --
>
> Key: HIVE-18421
> URL: https://issues.apache.org/jira/browse/HIVE-18421
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 2.1.1, 2.2.0, 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-18421.01.patch, HIVE-18421.02.patch,
> HIVE-18421.03.patch, HIVE-18421.04.patch, HIVE-18421.05.patch,
> HIVE-18421.06.patch
>
>
> In vectorized execution arithmetic operations which cause integer overflows
> can give wrong results. Issue is reproducible in both Orc and parquet.
> Simple test case to reproduce this issue
> {noformat}
> set hive.vectorized.execution.enabled=true;
> create table parquettable (t1 tinyint, t2 tinyint) stored as parquet;
> insert into parquettable values (-104, 25), (-112, 24), (54, 9);
> select t1, t2, (t1-t2) as diff from parquettable where (t1-t2) < 50 order by
> diff desc;
> +---+-+---+
> | t1 | t2 | diff |
> +---+-+---+
> | -104 | 25 | 127 |
> | -112 | 24 | 120 |
> | 54| 9 | 45|
> +---+-+---+
> {noformat}
> When vectorization is turned off the same query produces only one row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18640) Update q.out for TestSparkCliDriver.testCliDriver[ppd_join5]
[ https://issues.apache.org/jira/browse/HIVE-18640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354797#comment-16354797 ] Sahil Takiar commented on HIVE-18640: - ummm really not sure what is going on here. Test works locally on OSX. Tried on Ubuntu still works. Tried on an actual ptest-slave and it still works. Tried the entire q-test batch, still works. So not sure why this is failing. JVM version maybe? > Update q.out for TestSparkCliDriver.testCliDriver[ppd_join5] > > > Key: HIVE-18640 > URL: https://issues.apache.org/jira/browse/HIVE-18640 > Project: Hive > Issue Type: Test > Components: Test >Reporter: Sahil Takiar >Assignee: Sahil Takiar >Priority: Major > > TestSparkCliDriver.testCliDriver[ppd_join5] has been consistently failing for > 166 builds with a q-file diff. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18626) Repl load "with" clause does not pass config to tasks
[ https://issues.apache.org/jira/browse/HIVE-18626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354787#comment-16354787 ] Thejas M Nair commented on HIVE-18626: -- Changes look good. Can you please take a look at checkstyle warnings (seems like an indentation issue), and also verify that TestMarkPartition.testMarkingPartitionSet is not related to this issue ? (only test failure that has age=1 in jenkins) > Repl load "with" clause does not pass config to tasks > - > > Key: HIVE-18626 > URL: https://issues.apache.org/jira/browse/HIVE-18626 > Project: Hive > Issue Type: Bug > Components: repl >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-18626.1.patch > > > The "with" clause in repl load suppose to pass custom hive config entries to > replication. However, the config is only effective in > BootstrapEventsIterator, but not the generated tasks (such as MoveTask, > DDLTask). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18467) support whole warehouse dump / load + create/drop database events
[
https://issues.apache.org/jira/browse/HIVE-18467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354770#comment-16354770
]
Hive QA commented on HIVE-18467:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} standalone-metastore: The patch generated 6 new + 13
unchanged - 4 fixed = 19 total (was 17) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 6 new + 515 unchanged - 3
fixed = 521 total (was 518) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} itests/hive-unit: The patch generated 18 new + 19
unchanged - 4 fixed = 37 total (was 23) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3972bf0 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9057/yetus/diff-checkstyle-standalone-metastore.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9057/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9057/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: standalone-metastore ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9057/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> support whole warehouse dump / load + create/drop database events
> -
>
> Key: HIVE-18467
> URL: https://issues.apache.org/jira/browse/HIVE-18467
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0
>
> Attachments: HIVE-18467.0.patch, HIVE-18467.1.patch,
> HIVE-18467.2.patch, HIVE-18467.5.patch, HIVE-18467.6.patch
>
>
> A complete hive warehouse might be required to replicate to a DR site for
> certain use cases and rather than allowing only a database name in the REPL
> DUMP
[jira] [Commented] (HIVE-17063) insert overwrite partition onto a external table fail when drop partition first
[
https://issues.apache.org/jira/browse/HIVE-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354768#comment-16354768
]
Ashutosh Chauhan commented on HIVE-17063:
-
+1
> insert overwrite partition onto a external table fail when drop partition
> first
> ---
>
> Key: HIVE-17063
> URL: https://issues.apache.org/jira/browse/HIVE-17063
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.2, 2.1.1, 2.2.0
>Reporter: Wang Haihua
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-17063.1.patch, HIVE-17063.2.patch,
> HIVE-17063.3.patch, HIVE-17063.4.patch
>
>
> The default value of {{hive.exec.stagingdir}} which is a relative path, and
> also drop partition on a external table will not clear the real data. As a
> result, insert overwrite partition twice will happen to fail because of the
> target data to be moved has
> already existed.
> This happened when we reproduce partition data onto a external table.
> I see the target data will not be cleared only when {{immediately generated
> data}} is child of {{the target data directory}}, so my proposal is trying
> to clear target file already existed finally whe doing rename {{immediately
> generated data}} into {{the target data directory}}
> Operation reproduced:
> {code}
> create external table insert_after_drop_partition(key string, val string)
> partitioned by (insertdate string);
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> alter table insert_after_drop_partition drop partition
> (insertdate='2008-01-01');
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> {code}
> Stack trace:
> {code}
> 2017-07-09T08:32:05,212 ERROR [f3bc51c8-2441-4689-b1c1-d60aef86c3aa main]
> exec.Task: Failed with exception java.io.IOException: rename for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename
> for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2992)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3248)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1532)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1461)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:498)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2073)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1744)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1137)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:120)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_after_drop_partition(TestCliDriver.java:103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
>
[jira] [Commented] (HIVE-17063) insert overwrite partition onto a external table fail when drop partition first
[
https://issues.apache.org/jira/browse/HIVE-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354741#comment-16354741
]
Deepak Jaiswal commented on HIVE-17063:
---
[~ashutoshc] can you please review?
> insert overwrite partition onto a external table fail when drop partition
> first
> ---
>
> Key: HIVE-17063
> URL: https://issues.apache.org/jira/browse/HIVE-17063
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.2, 2.1.1, 2.2.0
>Reporter: Wang Haihua
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-17063.1.patch, HIVE-17063.2.patch,
> HIVE-17063.3.patch, HIVE-17063.4.patch
>
>
> The default value of {{hive.exec.stagingdir}} which is a relative path, and
> also drop partition on a external table will not clear the real data. As a
> result, insert overwrite partition twice will happen to fail because of the
> target data to be moved has
> already existed.
> This happened when we reproduce partition data onto a external table.
> I see the target data will not be cleared only when {{immediately generated
> data}} is child of {{the target data directory}}, so my proposal is trying
> to clear target file already existed finally whe doing rename {{immediately
> generated data}} into {{the target data directory}}
> Operation reproduced:
> {code}
> create external table insert_after_drop_partition(key string, val string)
> partitioned by (insertdate string);
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> alter table insert_after_drop_partition drop partition
> (insertdate='2008-01-01');
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> {code}
> Stack trace:
> {code}
> 2017-07-09T08:32:05,212 ERROR [f3bc51c8-2441-4689-b1c1-d60aef86c3aa main]
> exec.Task: Failed with exception java.io.IOException: rename for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename
> for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2992)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3248)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1532)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1461)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:498)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2073)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1744)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1137)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:120)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_after_drop_partition(TestCliDriver.java:103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
[jira] [Updated] (HIVE-17063) insert overwrite partition onto a external table fail when drop partition first
[
https://issues.apache.org/jira/browse/HIVE-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepak Jaiswal updated HIVE-17063:
--
Attachment: HIVE-17063.4.patch
> insert overwrite partition onto a external table fail when drop partition
> first
> ---
>
> Key: HIVE-17063
> URL: https://issues.apache.org/jira/browse/HIVE-17063
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.2, 2.1.1, 2.2.0
>Reporter: Wang Haihua
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-17063.1.patch, HIVE-17063.2.patch,
> HIVE-17063.3.patch, HIVE-17063.4.patch
>
>
> The default value of {{hive.exec.stagingdir}} which is a relative path, and
> also drop partition on a external table will not clear the real data. As a
> result, insert overwrite partition twice will happen to fail because of the
> target data to be moved has
> already existed.
> This happened when we reproduce partition data onto a external table.
> I see the target data will not be cleared only when {{immediately generated
> data}} is child of {{the target data directory}}, so my proposal is trying
> to clear target file already existed finally whe doing rename {{immediately
> generated data}} into {{the target data directory}}
> Operation reproduced:
> {code}
> create external table insert_after_drop_partition(key string, val string)
> partitioned by (insertdate string);
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> alter table insert_after_drop_partition drop partition
> (insertdate='2008-01-01');
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> {code}
> Stack trace:
> {code}
> 2017-07-09T08:32:05,212 ERROR [f3bc51c8-2441-4689-b1c1-d60aef86c3aa main]
> exec.Task: Failed with exception java.io.IOException: rename for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename
> for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2992)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3248)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1532)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1461)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:498)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2073)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1744)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1137)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:120)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_after_drop_partition(TestCliDriver.java:103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
>
[jira] [Assigned] (HIVE-17063) insert overwrite partition onto a external table fail when drop partition first
[
https://issues.apache.org/jira/browse/HIVE-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepak Jaiswal reassigned HIVE-17063:
-
Assignee: Deepak Jaiswal (was: Wang Haihua)
> insert overwrite partition onto a external table fail when drop partition
> first
> ---
>
> Key: HIVE-17063
> URL: https://issues.apache.org/jira/browse/HIVE-17063
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.2, 2.1.1, 2.2.0
>Reporter: Wang Haihua
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-17063.1.patch, HIVE-17063.2.patch,
> HIVE-17063.3.patch
>
>
> The default value of {{hive.exec.stagingdir}} which is a relative path, and
> also drop partition on a external table will not clear the real data. As a
> result, insert overwrite partition twice will happen to fail because of the
> target data to be moved has
> already existed.
> This happened when we reproduce partition data onto a external table.
> I see the target data will not be cleared only when {{immediately generated
> data}} is child of {{the target data directory}}, so my proposal is trying
> to clear target file already existed finally whe doing rename {{immediately
> generated data}} into {{the target data directory}}
> Operation reproduced:
> {code}
> create external table insert_after_drop_partition(key string, val string)
> partitioned by (insertdate string);
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> alter table insert_after_drop_partition drop partition
> (insertdate='2008-01-01');
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> {code}
> Stack trace:
> {code}
> 2017-07-09T08:32:05,212 ERROR [f3bc51c8-2441-4689-b1c1-d60aef86c3aa main]
> exec.Task: Failed with exception java.io.IOException: rename for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename
> for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2992)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3248)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1532)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1461)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:498)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2073)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1744)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1137)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:120)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_after_drop_partition(TestCliDriver.java:103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
>
[jira] [Updated] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18456: -- Attachment: HIVE-18456.1.patch > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18456) Add some tests for HIVE-18367 to check that the table information contains the query correctly
[ https://issues.apache.org/jira/browse/HIVE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18456: -- Status: Patch Available (was: Open) > Add some tests for HIVE-18367 to check that the table information contains > the query correctly > -- > > Key: HIVE-18456 > URL: https://issues.apache.org/jira/browse/HIVE-18456 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman >Priority: Major > Attachments: HIVE-18456.1.patch > > > This cannot be tested with a CliDriver test so add a java test to check the > output of 'describe extended', which is changed by HIVE-18367 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18567) ObjectStore.getPartitionNamesNoTxn doesn't handle max param properly
[
https://issues.apache.org/jira/browse/HIVE-18567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354647#comment-16354647
]
Hive QA commented on HIVE-18567:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909416/HIVE-18567.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 12974 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestFunctions.testGetFunctionNullDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9055/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9055/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9055/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909416 - PreCommit-HIVE-Build
> ObjectStore.getPartitionNamesNoTxn doesn't handle max param properly
>
>
> Key: HIVE-18567
> URL: https://issues.apache.org/jira/browse/HIVE-18567
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18567.0.patch, HIVE-18567.1.patch
>
>
> As per [this HMS API test
> case|https://github.com/apache/hive/commit/fa0a8d27d4149cc5cc2dbb49d8eb6b03f46bc279#diff-25c67d898000b53e623a6df9221aad5dR1044]
> listing partition names doesn't check tha max param against
> MetaStoreConf.LIMIT_PARTITION_REQUEST (as other methods do by
> checkLimitNumberOfPartitionsByFilter), and also behaves differently on max=0
> setting compared to other methods.
> We should bring this into consistency.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18629) copyValues in BytesColumnVector may be missing null checks
[
https://issues.apache.org/jira/browse/HIVE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354609#comment-16354609
]
Sergey Shelukhin commented on HIVE-18629:
-
Sorry, yeah, I mixed it up when changing the ifs around.
> copyValues in BytesColumnVector may be missing null checks
> --
>
> Key: HIVE-18629
> URL: https://issues.apache.org/jira/browse/HIVE-18629
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18629.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:333)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions..evaluate(IfExprStringGroupColumnStringGroupColumn.java:83)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:133)
> {noformat}
> IfExprStringGroupColumnStringGroupColumn code below the v1.isRepeating case
> has isNull checks for v2/v3 buffers that copySelected is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18629) copyValues in BytesColumnVector may be missing null checks
[
https://issues.apache.org/jira/browse/HIVE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354585#comment-16354585
]
Vihang Karajgaonkar commented on HIVE-18629:
Hi [~sershe] I took a quick look at the attached patch.
Shouldn't the snippet below also guard against NPE by adding a if condition
{{if (!(output.isNull[i] = isNull[i]))}} after we do a System.arraycopy
{code}
} else {
System.arraycopy(isNull, 0, output.isNull, 0, size);
for (int i = 0; i < size; i++) {
output.setVal(i, vector[i], start[i], length[i]);
}
}
{code}
> copyValues in BytesColumnVector may be missing null checks
> --
>
> Key: HIVE-18629
> URL: https://issues.apache.org/jira/browse/HIVE-18629
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18629.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:333)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions..evaluate(IfExprStringGroupColumnStringGroupColumn.java:83)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:133)
> {noformat}
> IfExprStringGroupColumnStringGroupColumn code below the v1.isRepeating case
> has isNull checks for v2/v3 buffers that copySelected is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18449) Add configurable policy for choosing the HMS URI from hive.metastore.uris
[
https://issues.apache.org/jira/browse/HIVE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janaki Lahorani updated HIVE-18449:
---
Attachment: HIVE-18449.3-branch-2.patch
> Add configurable policy for choosing the HMS URI from hive.metastore.uris
> -
>
> Key: HIVE-18449
> URL: https://issues.apache.org/jira/browse/HIVE-18449
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-18449.1.patch, HIVE-18449.2.patch,
> HIVE-18449.3-branch-2.patch
>
>
> HIVE-10815 added logic to randomly choose a HMS URI from
> {{hive.metastore.uris}}. It would be nice if there was a configurable policy
> that determined how a URI is chosen from this list - e.g. one option can be
> to randomly pick a URI, another option can be to choose the first URI in the
> list (which was the behavior prior to HIVE-10815).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18642) incorrect assertion in TezSessionPool for WM
[ https://issues.apache.org/jira/browse/HIVE-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354572#comment-16354572 ] Prasanth Jayachandran commented on HIVE-18642: -- +1 > incorrect assertion in TezSessionPool for WM > > > Key: HIVE-18642 > URL: https://issues.apache.org/jira/browse/HIVE-18642 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18642.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18627) PPD: Handle FLOAT boxing differently for single/double precision constants
[ https://issues.apache.org/jira/browse/HIVE-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354570#comment-16354570 ] Prasanth Jayachandran commented on HIVE-18627: -- +1. I wonder why the file system counters are removed for some tests. > PPD: Handle FLOAT boxing differently for single/double precision constants > -- > > Key: HIVE-18627 > URL: https://issues.apache.org/jira/browse/HIVE-18627 > Project: Hive > Issue Type: Bug > Components: Types >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18627.1.patch, HIVE-18627.2.patch > > > Constants like 0.1 and 0.3 are differently boxed based on intermediate > precision of the compiler codepath. > Disabling CBO produces 0.1BD constants which fail to box correctly to > Double/Float. > Enabling CBO fixes this issue, but cannot be applied all queries in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18567) ObjectStore.getPartitionNamesNoTxn doesn't handle max param properly
[
https://issues.apache.org/jira/browse/HIVE-18567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354559#comment-16354559
]
Hive QA commented on HIVE-18567:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} standalone-metastore: The patch generated 1 new + 474
unchanged - 2 fixed = 475 total (was 476) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 39s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3972bf0 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9055/yetus/diff-checkstyle-standalone-metastore.txt
|
| modules | C: standalone-metastore U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9055/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ObjectStore.getPartitionNamesNoTxn doesn't handle max param properly
>
>
> Key: HIVE-18567
> URL: https://issues.apache.org/jira/browse/HIVE-18567
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18567.0.patch, HIVE-18567.1.patch
>
>
> As per [this HMS API test
> case|https://github.com/apache/hive/commit/fa0a8d27d4149cc5cc2dbb49d8eb6b03f46bc279#diff-25c67d898000b53e623a6df9221aad5dR1044]
> listing partition names doesn't check tha max param against
> MetaStoreConf.LIMIT_PARTITION_REQUEST (as other methods do by
> checkLimitNumberOfPartitionsByFilter), and also behaves differently on max=0
> setting compared to other methods.
> We should bring this into consistency.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18614) Fix sys db creation in Hive
[ https://issues.apache.org/jira/browse/HIVE-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-18614: -- Attachment: HIVE-18614.patch > Fix sys db creation in Hive > --- > > Key: HIVE-18614 > URL: https://issues.apache.org/jira/browse/HIVE-18614 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18614.patch > > > Sys db can not be created due to several server side issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18614) Fix sys db creation in Hive
[ https://issues.apache.org/jira/browse/HIVE-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-18614: -- Status: Patch Available (was: Open) > Fix sys db creation in Hive > --- > > Key: HIVE-18614 > URL: https://issues.apache.org/jira/browse/HIVE-18614 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18614.patch > > > Sys db can not be created due to several server side issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18614) Fix sys db creation in Hive
[ https://issues.apache.org/jira/browse/HIVE-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-18614: -- Attachment: (was: HIVE-18614.patch) > Fix sys db creation in Hive > --- > > Key: HIVE-18614 > URL: https://issues.apache.org/jira/browse/HIVE-18614 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.0.0 > > > Sys db can not be created due to several server side issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18614) Fix sys db creation in Hive
[ https://issues.apache.org/jira/browse/HIVE-18614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-18614: -- Status: Open (was: Patch Available) > Fix sys db creation in Hive > --- > > Key: HIVE-18614 > URL: https://issues.apache.org/jira/browse/HIVE-18614 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.0.0 > > > Sys db can not be created due to several server side issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18523) Fix summary row in case there are no inputs
[
https://issues.apache.org/jira/browse/HIVE-18523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354533#comment-16354533
]
Hive QA commented on HIVE-18523:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909470/HIVE-18523.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 26 failed/errored test(s), 12950 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.TestAcidTableSetup.testTransactionalValidation
(batchId=223)
org.apache.hadoop.hive.metastore.TestReplChangeManager.testRecycleNonPartTable
(batchId=223)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.authorization.TestJdbcMetadataApiAuth.org.apache.hive.jdbc.authorization.TestJdbcMetadataApiAuth
(batchId=239)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9054/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9054/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9054/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 26 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909470 - PreCommit-HIVE-Build
> Fix summary row in case there are no inputs
> ---
>
> Key: HIVE-18523
> URL: https://issues.apache.org/jira/browse/HIVE-18523
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18523.01.patch, HIVE-18523.03.patch
>
>
> HIVE-17617 added a fix to handle empty groups correctly; however it might be
> possible that the summary row is returned multiple times - in case the
> mappers are "seem" to be optimized away...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-17063) insert overwrite partition onto a external table fail when drop partition first
[
https://issues.apache.org/jira/browse/HIVE-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepak Jaiswal reassigned HIVE-17063:
-
Assignee: Wang Haihua (was: Deepak Jaiswal)
> insert overwrite partition onto a external table fail when drop partition
> first
> ---
>
> Key: HIVE-17063
> URL: https://issues.apache.org/jira/browse/HIVE-17063
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.2, 2.1.1, 2.2.0
>Reporter: Wang Haihua
>Assignee: Wang Haihua
>Priority: Major
> Attachments: HIVE-17063.1.patch, HIVE-17063.2.patch,
> HIVE-17063.3.patch
>
>
> The default value of {{hive.exec.stagingdir}} which is a relative path, and
> also drop partition on a external table will not clear the real data. As a
> result, insert overwrite partition twice will happen to fail because of the
> target data to be moved has
> already existed.
> This happened when we reproduce partition data onto a external table.
> I see the target data will not be cleared only when {{immediately generated
> data}} is child of {{the target data directory}}, so my proposal is trying
> to clear target file already existed finally whe doing rename {{immediately
> generated data}} into {{the target data directory}}
> Operation reproduced:
> {code}
> create external table insert_after_drop_partition(key string, val string)
> partitioned by (insertdate string);
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> alter table insert_after_drop_partition drop partition
> (insertdate='2008-01-01');
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> {code}
> Stack trace:
> {code}
> 2017-07-09T08:32:05,212 ERROR [f3bc51c8-2441-4689-b1c1-d60aef86c3aa main]
> exec.Task: Failed with exception java.io.IOException: rename for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename
> for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2992)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3248)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1532)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1461)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:498)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2073)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1744)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1137)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:120)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_after_drop_partition(TestCliDriver.java:103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
>
[jira] [Commented] (HIVE-18340) Dynamic Min-Max/BloomFilter runtime-filtering in HoS
[ https://issues.apache.org/jira/browse/HIVE-18340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354529#comment-16354529 ] Sahil Takiar commented on HIVE-18340: - Have we done any performance analysis for this feature? > Dynamic Min-Max/BloomFilter runtime-filtering in HoS > > > Key: HIVE-18340 > URL: https://issues.apache.org/jira/browse/HIVE-18340 > Project: Hive > Issue Type: New Feature > Components: Spark >Affects Versions: 3.0.0 >Reporter: Ke Jia >Assignee: Ke Jia >Priority: Major > Attachments: HIVE-18340.1.patch > > > Tez implemented Dynamic Min-Max/BloomFilter runtime-filtering in HIVE-15269 > and we should implement the same in HOS. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18641) Remove MCreationMetadata from MTable class
[ https://issues.apache.org/jira/browse/HIVE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354523#comment-16354523 ] Ashutosh Chauhan commented on HIVE-18641: - +1 > Remove MCreationMetadata from MTable class > -- > > Key: HIVE-18641 > URL: https://issues.apache.org/jira/browse/HIVE-18641 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18641.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18421) Vectorized execution handles overflows in a different manner than non-vectorized execution
[
https://issues.apache.org/jira/browse/HIVE-18421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354516#comment-16354516
]
Vihang Karajgaonkar commented on HIVE-18421:
latest patch fixes TestCliDriver.testCliDriver[vectorization_numeric_overflows]
and some Yetus checkstyle changes.
> Vectorized execution handles overflows in a different manner than
> non-vectorized execution
> --
>
> Key: HIVE-18421
> URL: https://issues.apache.org/jira/browse/HIVE-18421
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 2.1.1, 2.2.0, 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-18421.01.patch, HIVE-18421.02.patch,
> HIVE-18421.03.patch, HIVE-18421.04.patch, HIVE-18421.05.patch
>
>
> In vectorized execution arithmetic operations which cause integer overflows
> can give wrong results. Issue is reproducible in both Orc and parquet.
> Simple test case to reproduce this issue
> {noformat}
> set hive.vectorized.execution.enabled=true;
> create table parquettable (t1 tinyint, t2 tinyint) stored as parquet;
> insert into parquettable values (-104, 25), (-112, 24), (54, 9);
> select t1, t2, (t1-t2) as diff from parquettable where (t1-t2) < 50 order by
> diff desc;
> +---+-+---+
> | t1 | t2 | diff |
> +---+-+---+
> | -104 | 25 | 127 |
> | -112 | 24 | 120 |
> | 54| 9 | 45|
> +---+-+---+
> {noformat}
> When vectorization is turned off the same query produces only one row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18421) Vectorized execution handles overflows in a different manner than non-vectorized execution
[
https://issues.apache.org/jira/browse/HIVE-18421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-18421:
---
Attachment: HIVE-18421.05.patch
> Vectorized execution handles overflows in a different manner than
> non-vectorized execution
> --
>
> Key: HIVE-18421
> URL: https://issues.apache.org/jira/browse/HIVE-18421
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 2.1.1, 2.2.0, 3.0.0, 2.3.2
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-18421.01.patch, HIVE-18421.02.patch,
> HIVE-18421.03.patch, HIVE-18421.04.patch, HIVE-18421.05.patch
>
>
> In vectorized execution arithmetic operations which cause integer overflows
> can give wrong results. Issue is reproducible in both Orc and parquet.
> Simple test case to reproduce this issue
> {noformat}
> set hive.vectorized.execution.enabled=true;
> create table parquettable (t1 tinyint, t2 tinyint) stored as parquet;
> insert into parquettable values (-104, 25), (-112, 24), (54, 9);
> select t1, t2, (t1-t2) as diff from parquettable where (t1-t2) < 50 order by
> diff desc;
> +---+-+---+
> | t1 | t2 | diff |
> +---+-+---+
> | -104 | 25 | 127 |
> | -112 | 24 | 120 |
> | 54| 9 | 45|
> +---+-+---+
> {noformat}
> When vectorization is turned off the same query produces only one row.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18588) Add 'checkin' profile that runs slower tests in standalone-metastore
[ https://issues.apache.org/jira/browse/HIVE-18588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354491#comment-16354491 ] Alan Gates commented on HIVE-18588: --- +1 to running the precommit tests with groups set to "". Do you want to make changes to ptest as part of this patch or do you want to do that separately? > Add 'checkin' profile that runs slower tests in standalone-metastore > > > Key: HIVE-18588 > URL: https://issues.apache.org/jira/browse/HIVE-18588 > Project: Hive > Issue Type: Test > Components: Standalone Metastore >Affects Versions: 3.0.0 >Reporter: Alan Gates >Assignee: Alan Gates >Priority: Major > Attachments: HIVE-18588.2.patch, HIVE-18588.patch > > > Runtime for unit tests in standalone-metastore are now exceeding 25 minutes. > Ideally unit tests should finish within 2-3 minutes so users will run them > frequently. To solve this I propose to carve off many of the slower tests to > run in a new 'checkin' profile. This profile should be run before checkin > and by the ptest infrastructure. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18642) incorrect assertion in TezSessionPool for WM
[ https://issues.apache.org/jira/browse/HIVE-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18642: Status: Patch Available (was: Open) > incorrect assertion in TezSessionPool for WM > > > Key: HIVE-18642 > URL: https://issues.apache.org/jira/browse/HIVE-18642 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18642.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18642) incorrect assertion in TezSessionPool for WM
[ https://issues.apache.org/jira/browse/HIVE-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354487#comment-16354487 ] Sergey Shelukhin commented on HIVE-18642: - [~prasanth_j] can you take a look? a tiny patch > incorrect assertion in TezSessionPool for WM > > > Key: HIVE-18642 > URL: https://issues.apache.org/jira/browse/HIVE-18642 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18642.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18642) incorrect assertion in TezSessionPool for WM
[ https://issues.apache.org/jira/browse/HIVE-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18642: --- > incorrect assertion in TezSessionPool for WM > > > Key: HIVE-18642 > URL: https://issues.apache.org/jira/browse/HIVE-18642 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18642.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18642) incorrect assertion in TezSessionPool for WM
[ https://issues.apache.org/jira/browse/HIVE-18642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18642: Attachment: HIVE-18642.patch > incorrect assertion in TezSessionPool for WM > > > Key: HIVE-18642 > URL: https://issues.apache.org/jira/browse/HIVE-18642 > Project: Hive > Issue Type: Bug >Reporter: Dileep Kumar Chiguruvada >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18642.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18523) Fix summary row in case there are no inputs
[
https://issues.apache.org/jira/browse/HIVE-18523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354484#comment-16354484
]
Hive QA commented on HIVE-18523:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 270 unchanged - 24
fixed = 271 total (was 294) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 3972bf0 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9054/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9054/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix summary row in case there are no inputs
> ---
>
> Key: HIVE-18523
> URL: https://issues.apache.org/jira/browse/HIVE-18523
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18523.01.patch, HIVE-18523.03.patch
>
>
> HIVE-17617 added a fix to handle empty groups correctly; however it might be
> possible that the summary row is returned multiple times - in case the
> mappers are "seem" to be optimized away...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18627) PPD: Handle FLOAT boxing differently for single/double precision constants
[ https://issues.apache.org/jira/browse/HIVE-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-18627: --- Attachment: HIVE-18627.2.patch > PPD: Handle FLOAT boxing differently for single/double precision constants > -- > > Key: HIVE-18627 > URL: https://issues.apache.org/jira/browse/HIVE-18627 > Project: Hive > Issue Type: Bug > Components: Types >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18627.1.patch, HIVE-18627.2.patch > > > Constants like 0.1 and 0.3 are differently boxed based on intermediate > precision of the compiler codepath. > Disabling CBO produces 0.1BD constants which fail to box correctly to > Double/Float. > Enabling CBO fixes this issue, but cannot be applied all queries in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18628) Make tez dag status check interval configurable
[ https://issues.apache.org/jira/browse/HIVE-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354474#comment-16354474 ] Prasanth Jayachandran commented on HIVE-18628: -- [~sershe] still discussing with tez team about aggregated counters when tasks are running. I will create a hive or tez ticket based on the response (not sure if there is a way in tez to get aggregated live counters when task is running, if there is a way it will be hive follow up else tez ticket) > Make tez dag status check interval configurable > --- > > Key: HIVE-18628 > URL: https://issues.apache.org/jira/browse/HIVE-18628 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-18628.1.patch > > > DAG status check interval in TezJobMonitor is not configurable. This can > cause cached data to be returned and fail some trigger validation tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18641) Remove MCreationMetadata from MTable class
[
https://issues.apache.org/jira/browse/HIVE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354472#comment-16354472
]
Jesus Camacho Rodriguez commented on HIVE-18641:
[~ashutoshc], could you take a look? This should have been pushed within
HIVE-18546 since the {{CreationMetadataID}} column was removed from {{TBLS}}
and thus should not be contained in {{MTable}}. Thanks
> Remove MCreationMetadata from MTable class
> --
>
> Key: HIVE-18641
> URL: https://issues.apache.org/jira/browse/HIVE-18641
> Project: Hive
> Issue Type: Bug
> Components: Materialized views, Metastore
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-18641.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18628) Make tez dag status check interval configurable
[ https://issues.apache.org/jira/browse/HIVE-18628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354469#comment-16354469 ] Sergey Shelukhin commented on HIVE-18628: - +1. Should there be a Tez jira/follow up jira for the commented out test? That seems like a problem. > Make tez dag status check interval configurable > --- > > Key: HIVE-18628 > URL: https://issues.apache.org/jira/browse/HIVE-18628 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-18628.1.patch > > > DAG status check interval in TezJobMonitor is not configurable. This can > cause cached data to be returned and fail some trigger validation tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18238) Driver execution may not have configuration changing sideeffects
[
https://issues.apache.org/jira/browse/HIVE-18238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354459#comment-16354459
]
Hive QA commented on HIVE-18238:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909455/HIVE-18238.07.patch
{color:green}SUCCESS:{color} +1 due to 11 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 12975 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testCTAS (batchId=280)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9053/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9053/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9053/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12909455 - PreCommit-HIVE-Build
> Driver execution may not have configuration changing sideeffects
> -
>
> Key: HIVE-18238
> URL: https://issues.apache.org/jira/browse/HIVE-18238
> Project: Hive
> Issue Type: Sub-task
> Components: Logical Optimizer
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-18238.01wip01.patch, HIVE-18238.02.patch,
> HIVE-18238.03.patch, HIVE-18238.04.patch, HIVE-18238.04wip01.patch,
> HIVE-18238.07.patch
>
>
> {{Driver}} executes sql statements which use "hiveconf" settings;
> but the {{Driver}} itself may *not* change the configuration...
> I've found an example; which shows how hazardous this is...
> {code}
> set hive.mapred.mode=strict;
> select "${hiveconf:hive.mapred.mode}";
> create table t (a int);
> analyze table t compute statistics;
> select "${hiveconf:hive.mapred.mode}";
> {code}
> currently; the last select returns {{nonstrict}} because of
> [this|https://github.com/apache/hive/blob/7ddd915bf82a68c8ab73b0c4ca409f1a6d43d227/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L1696]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18641) Remove MCreationMetadata from MTable class
[ https://issues.apache.org/jira/browse/HIVE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18641: --- Attachment: HIVE-18641.patch > Remove MCreationMetadata from MTable class > -- > > Key: HIVE-18641 > URL: https://issues.apache.org/jira/browse/HIVE-18641 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18641.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-18641) Remove MCreationMetadata from MTable class
[ https://issues.apache.org/jira/browse/HIVE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-18641 started by Jesus Camacho Rodriguez. -- > Remove MCreationMetadata from MTable class > -- > > Key: HIVE-18641 > URL: https://issues.apache.org/jira/browse/HIVE-18641 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18641) Remove MCreationMetadata from MTable class
[ https://issues.apache.org/jira/browse/HIVE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18641: --- Status: Patch Available (was: In Progress) > Remove MCreationMetadata from MTable class > -- > > Key: HIVE-18641 > URL: https://issues.apache.org/jira/browse/HIVE-18641 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18641) Remove MCreationMetadata from MTable class
[ https://issues.apache.org/jira/browse/HIVE-18641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-18641: -- > Remove MCreationMetadata from MTable class > -- > > Key: HIVE-18641 > URL: https://issues.apache.org/jira/browse/HIVE-18641 > Project: Hive > Issue Type: Bug > Components: Materialized views, Metastore >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17735) ObjectStore.addNotificationEvent is leaking queries
[
https://issues.apache.org/jira/browse/HIVE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354427#comment-16354427
]
Aihua Xu commented on HIVE-17735:
-
Patch-2: fix one test failure. The others don't seem related.
> ObjectStore.addNotificationEvent is leaking queries
> ---
>
> Key: HIVE-17735
> URL: https://issues.apache.org/jira/browse/HIVE-17735
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-17735.1.patch, HIVE-17735.2.patch
>
>
> In ObjectStore.addNotificationEvent():
> {code}
> Query objectQuery = pm.newQuery(MNotificationNextId.class);
> Collection ids = (Collection)
> objectQuery.execute();
> {code}
> The query is never closed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17735) ObjectStore.addNotificationEvent is leaking queries
[
https://issues.apache.org/jira/browse/HIVE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aihua Xu updated HIVE-17735:
Attachment: HIVE-17735.2.patch
> ObjectStore.addNotificationEvent is leaking queries
> ---
>
> Key: HIVE-17735
> URL: https://issues.apache.org/jira/browse/HIVE-17735
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Alexander Kolbasov
>Assignee: Aihua Xu
>Priority: Major
> Attachments: HIVE-17735.1.patch, HIVE-17735.2.patch
>
>
> In ObjectStore.addNotificationEvent():
> {code}
> Query objectQuery = pm.newQuery(MNotificationNextId.class);
> Collection ids = (Collection)
> objectQuery.execute();
> {code}
> The query is never closed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-17063) insert overwrite partition onto a external table fail when drop partition first
[
https://issues.apache.org/jira/browse/HIVE-17063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepak Jaiswal reassigned HIVE-17063:
-
Assignee: Deepak Jaiswal (was: Wang Haihua)
> insert overwrite partition onto a external table fail when drop partition
> first
> ---
>
> Key: HIVE-17063
> URL: https://issues.apache.org/jira/browse/HIVE-17063
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 1.2.2, 2.1.1, 2.2.0
>Reporter: Wang Haihua
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-17063.1.patch, HIVE-17063.2.patch,
> HIVE-17063.3.patch
>
>
> The default value of {{hive.exec.stagingdir}} which is a relative path, and
> also drop partition on a external table will not clear the real data. As a
> result, insert overwrite partition twice will happen to fail because of the
> target data to be moved has
> already existed.
> This happened when we reproduce partition data onto a external table.
> I see the target data will not be cleared only when {{immediately generated
> data}} is child of {{the target data directory}}, so my proposal is trying
> to clear target file already existed finally whe doing rename {{immediately
> generated data}} into {{the target data directory}}
> Operation reproduced:
> {code}
> create external table insert_after_drop_partition(key string, val string)
> partitioned by (insertdate string);
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> alter table insert_after_drop_partition drop partition
> (insertdate='2008-01-01');
> from src insert overwrite table insert_after_drop_partition partition
> (insertdate='2008-01-01') select *;
> {code}
> Stack trace:
> {code}
> 2017-07-09T08:32:05,212 ERROR [f3bc51c8-2441-4689-b1c1-d60aef86c3aa main]
> exec.Task: Failed with exception java.io.IOException: rename for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename
> for src path:
> pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/.hive-staging_hive_2017-07-09_08-32-03_840_4046825276907030554-1/-ext-1/00_0
> to dest
> path:pfile:/data/haihua/official/hive/itests/qtest/target/warehouse/insert_after_drop_partition/insertdate=2008-01-01/00_0
> returned false
> at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2992)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.replaceFiles(Hive.java:3248)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1532)
> at
> org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1461)
> at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:498)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2073)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1744)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1453)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1171)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1161)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:335)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1137)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:120)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_insert_after_drop_partition(TestCliDriver.java:103)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
>
[jira] [Commented] (HIVE-18627) PPD: Handle FLOAT boxing differently for single/double precision constants
[ https://issues.apache.org/jira/browse/HIVE-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354414#comment-16354414 ] Gopal V commented on HIVE-18627: This is the test that fails if I fix it by Boxing from Decimal -> Double. If I do Decimal -> Float -> Double, it passes. This test fails if I fixed it without the new Enum to box those two differently. I'll add a 1 row Double test, to cover the broken case. > PPD: Handle FLOAT boxing differently for single/double precision constants > -- > > Key: HIVE-18627 > URL: https://issues.apache.org/jira/browse/HIVE-18627 > Project: Hive > Issue Type: Bug > Components: Types >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18627.1.patch > > > Constants like 0.1 and 0.3 are differently boxed based on intermediate > precision of the compiler codepath. > Disabling CBO produces 0.1BD constants which fail to box correctly to > Double/Float. > Enabling CBO fixes this issue, but cannot be applied all queries in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18627) PPD: Handle FLOAT boxing differently for single/double precision constants
[ https://issues.apache.org/jira/browse/HIVE-18627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354395#comment-16354395 ] Prasanth Jayachandran commented on HIVE-18627: -- [~gopalv] how come the tests weren't failing then? can we add a test for the delete with filter 0.1 which was originally causing this issue? > PPD: Handle FLOAT boxing differently for single/double precision constants > -- > > Key: HIVE-18627 > URL: https://issues.apache.org/jira/browse/HIVE-18627 > Project: Hive > Issue Type: Bug > Components: Types >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18627.1.patch > > > Constants like 0.1 and 0.3 are differently boxed based on intermediate > precision of the compiler codepath. > Disabling CBO produces 0.1BD constants which fail to box correctly to > Double/Float. > Enabling CBO fixes this issue, but cannot be applied all queries in Hive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
