[jira] [Commented] (HIVE-20937) Postgres jdbc query fail with "LIMIT must not be negative"
[
https://issues.apache.org/jira/browse/HIVE-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690422#comment-16690422
]
Hive QA commented on HIVE-20937:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948586/HIVE-20937.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15544 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14975/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14975/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14975/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948586 - PreCommit-HIVE-Build
> Postgres jdbc query fail with "LIMIT must not be negative"
> --
>
> Key: HIVE-20937
> URL: https://issues.apache.org/jira/browse/HIVE-20937
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20937.1.patch
>
>
> PostgresDatabaseAccessor does not handle limit=-1. Likely to affect
> Oracle/Mssql as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20937) Postgres jdbc query fail with "LIMIT must not be negative"
[
https://issues.apache.org/jira/browse/HIVE-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690406#comment-16690406
]
Hive QA commented on HIVE-20937:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} jdbc-handler in master has 12 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14975/dev-support/hive-personality.sh
|
| git revision | master / fd5f34f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: jdbc-handler U: jdbc-handler |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14975/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Postgres jdbc query fail with "LIMIT must not be negative"
> --
>
> Key: HIVE-20937
> URL: https://issues.apache.org/jira/browse/HIVE-20937
> Project: Hive
> Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-20937.1.patch
>
>
> PostgresDatabaseAccessor does not handle limit=-1. Likely to affect
> Oracle/Mssql as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20845) Fix TestJdbcWithDBTokenStoreNoDoAs flakiness
[ https://issues.apache.org/jira/browse/HIVE-20845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690403#comment-16690403 ] Peter Vary commented on HIVE-20845: --- Thanks [~sershe]! Will try again then > Fix TestJdbcWithDBTokenStoreNoDoAs flakiness > > > Key: HIVE-20845 > URL: https://issues.apache.org/jira/browse/HIVE-20845 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-20845.2.patch, HIVE-20845.patch > > > Previously did a dirty fix for TestJdbcWithDBTokenStoreNoDoAs and > TestJdbcWithDBTokenStore > Found out the issue is that we do not wait enough for HS2 to come up. > Need to fix in MiniHS2.waitForStartup() -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20937) Postgres jdbc query fail with "LIMIT must not be negative"
[ https://issues.apache.org/jira/browse/HIVE-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20937: -- Attachment: HIVE-20937.1.patch > Postgres jdbc query fail with "LIMIT must not be negative" > -- > > Key: HIVE-20937 > URL: https://issues.apache.org/jira/browse/HIVE-20937 > Project: Hive > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20937.1.patch > > > PostgresDatabaseAccessor does not handle limit=-1. Likely to affect > Oracle/Mssql as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20937) Postgres jdbc query fail with "LIMIT must not be negative"
[ https://issues.apache.org/jira/browse/HIVE-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-20937: -- Status: Patch Available (was: Open) > Postgres jdbc query fail with "LIMIT must not be negative" > -- > > Key: HIVE-20937 > URL: https://issues.apache.org/jira/browse/HIVE-20937 > Project: Hive > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20937.1.patch > > > PostgresDatabaseAccessor does not handle limit=-1. Likely to affect > Oracle/Mssql as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20937) Postgres jdbc query fail with "LIMIT must not be negative"
[ https://issues.apache.org/jira/browse/HIVE-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai reassigned HIVE-20937: - > Postgres jdbc query fail with "LIMIT must not be negative" > -- > > Key: HIVE-20937 > URL: https://issues.apache.org/jira/browse/HIVE-20937 > Project: Hive > Issue Type: Bug >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-20937.1.patch > > > PostgresDatabaseAccessor does not handle limit=-1. Likely to affect > Oracle/Mssql as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20936) Allow the Worker thread in the metastore to run outside of it
[
https://issues.apache.org/jira/browse/HIVE-20936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690385#comment-16690385
]
Hive QA commented on HIVE-20936:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948580/HIVE-20936.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 75 failed/errored test(s), 15544 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck]
(batchId=45)
org.apache.hadoop.hive.ql.TestTxnCommands2.testACIDwithSchemaEvolutionAndCompaction
(batchId=308)
org.apache.hadoop.hive.ql.TestTxnCommands2.testCleanerForTxnToWriteId
(batchId=308)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInitiatorWithMultipleFailedCompactions
(batchId=308)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwrite1 (batchId=308)
org.apache.hadoop.hive.ql.TestTxnCommands2.testInsertOverwrite2 (batchId=308)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1
(batchId=308)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion2
(batchId=308)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion3
(batchId=308)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testACIDwithSchemaEvolutionAndCompaction
(batchId=320)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testCleanerForTxnToWriteId
(batchId=320)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testInitiatorWithMultipleFailedCompactions
(batchId=320)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testInsertOverwrite1
(batchId=320)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testInsertOverwrite2
(batchId=320)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
(batchId=320)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion2
(batchId=320)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion3
(batchId=320)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testInsertOverwriteForPartitionedMmTable
(batchId=284)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testInsertOverwriteWithUnionAll
(batchId=284)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testOperationsOnCompletedTxnComponentsForMmTable
(batchId=284)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testSnapshotIsolationWithAbortedTxnOnMmTable
(batchId=284)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testInsertOverwriteForPartitionedMmTable
(batchId=306)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testInsertOverwriteWithUnionAll
(batchId=306)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testOperationsOnCompletedTxnComponentsForMmTable
(batchId=306)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testSnapshotIsolationWithAbortedTxnOnMmTable
(batchId=306)
org.apache.hadoop.hive.ql.TestTxnNoBuckets.testNoBuckets (batchId=308)
org.apache.hadoop.hive.ql.TestTxnNoBucketsVectorized.testNoBuckets (batchId=308)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMetastoreTablesCleanup
(batchId=323)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMajorPartitionCompaction
(batchId=293)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMajorPartitionCompactionNoBase
(batchId=293)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMajorTableCompaction
(batchId=293)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMinorPartitionCompaction
(batchId=293)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.cleanupAfterMinorTableCompaction
(batchId=293)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.notBlockedBySubsequentLock
(batchId=293)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner.partitionNotBlockedBySubsequentLock
(batchId=293)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMajorPartitionCompaction
(batchId=294)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMajorPartitionCompactionNoBase
(batchId=294)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMajorTableCompaction
(batchId=294)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMinorPartitionCompaction
(batchId=294)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.cleanupAfterMinorTableCompaction
(batchId=294)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.notBlockedBySubsequentLock
(batchId=294)
org.apache.hadoop.hive.ql.txn.compactor.TestCleaner2.partitionNotBlockedBySubsequentLock
(batchId=294)
org.apache.hadoop.hive.ql.txn.compactor.TestCleanerWithReplication.cleanupAfterMajorPartitionCompaction
(batchId=245)
[jira] [Commented] (HIVE-20936) Allow the Worker thread in the metastore to run outside of it
[
https://issues.apache.org/jira/browse/HIVE-20936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690381#comment-16690381
]
Hive QA commented on HIVE-20936:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
16s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
50s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
3s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
51s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
19s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
17s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 17s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 8 new + 108 unchanged - 2
fixed = 116 total (was 110) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} service: The patch generated 1 new + 35 unchanged - 0
fixed = 36 total (was 35) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 96 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
49s{color} | {color:red} ql generated 2 new + 2316 unchanged - 0 fixed = 2318
total (was 2316) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
19s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
16s{color} | {color:red} service generated 1 new + 40 unchanged - 0 fixed = 41
total (was 40) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
13s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 41s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Field MetaStoreCompactorThread.threadId masks field in superclass
org.apache.hadoop.hive.ql.txn.compactor.CompactorThread In
MetaStoreCompactorThread.java:superclass
org.apache.hadoop.hive.ql.txn.compactor.CompactorThread In
MetaStoreCompactorThread.java |
| | Unread public/protected field:At MetaStoreCompactorThread.java:[line 20] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14974/dev-support/hive-personality.sh
|
| git revision | master / fd5f34f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall
[jira] [Updated] (HIVE-20936) Allow the Worker thread in the metastore to run outside of it
[ https://issues.apache.org/jira/browse/HIVE-20936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-20936: -- Component/s: Transactions > Allow the Worker thread in the metastore to run outside of it > - > > Key: HIVE-20936 > URL: https://issues.apache.org/jira/browse/HIVE-20936 > Project: Hive > Issue Type: Improvement > Components: Transactions >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20936.1.patch, HIVE-20936.2.patch > > > Currently the Worker thread in the metastore in bounded to the metastore, > mainly because of the TxnHandler that it has. This thread runs some map > reduce jobs which may not only be an option wherever the metastore is > running. A solution for this can be to run this thread in HS2 depending on a > flag. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20936) Allow the Worker thread in the metastore to run outside of it
[ https://issues.apache.org/jira/browse/HIVE-20936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20936: --- Attachment: HIVE-20936.2.patch Status: Patch Available (was: Open) > Allow the Worker thread in the metastore to run outside of it > - > > Key: HIVE-20936 > URL: https://issues.apache.org/jira/browse/HIVE-20936 > Project: Hive > Issue Type: Improvement >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20936.1.patch, HIVE-20936.2.patch > > > Currently the Worker thread in the metastore in bounded to the metastore, > mainly because of the TxnHandler that it has. This thread runs some map > reduce jobs which may not only be an option wherever the metastore is > running. A solution for this can be to run this thread in HS2 depending on a > flag. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20936) Allow the Worker thread in the metastore to run outside of it
[ https://issues.apache.org/jira/browse/HIVE-20936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20936: --- Status: Open (was: Patch Available) > Allow the Worker thread in the metastore to run outside of it > - > > Key: HIVE-20936 > URL: https://issues.apache.org/jira/browse/HIVE-20936 > Project: Hive > Issue Type: Improvement >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20936.1.patch > > > Currently the Worker thread in the metastore in bounded to the metastore, > mainly because of the TxnHandler that it has. This thread runs some map > reduce jobs which may not only be an option wherever the metastore is > running. A solution for this can be to run this thread in HS2 depending on a > flag. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20936) Allow the Worker thread in the metastore to run outside of it
[
https://issues.apache.org/jira/browse/HIVE-20936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690360#comment-16690360
]
Hive QA commented on HIVE-20936:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948576/HIVE-20936.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14973/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14973/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14973/
Messages:
{noformat}
This message was trimmed, see log for full details
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: interface
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.Iface
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[487,29]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: interface
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.Iface
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[927,48]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: interface
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.AsyncIface
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[931,41]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: interface
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.AsyncIface
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[933,30]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: interface
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.AsyncIface
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[935,32]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: interface
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.AsyncIface
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[937,29]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: interface
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.AsyncIface
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[7137,12]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: class
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.Client
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[7150,12]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: class
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.Client
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[7163,48]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: class
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.Client
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[7169,53]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: class
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.Client
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[7205,49]
cannot find symbol
[ERROR] symbol: class CompactionInfoStruct
[ERROR] location: class
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore.Client
[ERROR]
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:[7211,46]
[jira] [Commented] (HIVE-20932) Vectorize Druid Storage Handler Reader
[
https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690358#comment-16690358
]
Hive QA commented on HIVE-20932:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948574/HIVE-20932.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14972/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14972/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14972/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12948574/HIVE-20932.3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948574 - PreCommit-HIVE-Build
> Vectorize Druid Storage Handler Reader
> --
>
> Key: HIVE-20932
> URL: https://issues.apache.org/jira/browse/HIVE-20932
> Project: Hive
> Issue Type: Improvement
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20932.3.patch, HIVE-20932.patch
>
>
> This patch aims at adding support for vectorize read of data from Druid to
> Hive.
> [~t3rmin4t0r] suggested that this will improve the performance of the top
> level operators that supports vectorization.
> As a first cut am just adding a wrapper around the existing Record Reader to
> read up to 1024 row at a time.
> Future work will be to avoid going via old reader and convert straight the
> Json (smile format) to Vector primitive types.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20932) Vectorize Druid Storage Handler Reader
[
https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690357#comment-16690357
]
Hive QA commented on HIVE-20932:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948574/HIVE-20932.3.patch
{color:green}SUCCESS:{color} +1 due to 15 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15544 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14971/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14971/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14971/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948574 - PreCommit-HIVE-Build
> Vectorize Druid Storage Handler Reader
> --
>
> Key: HIVE-20932
> URL: https://issues.apache.org/jira/browse/HIVE-20932
> Project: Hive
> Issue Type: Improvement
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20932.3.patch, HIVE-20932.patch
>
>
> This patch aims at adding support for vectorize read of data from Druid to
> Hive.
> [~t3rmin4t0r] suggested that this will improve the performance of the top
> level operators that supports vectorization.
> As a first cut am just adding a wrapper around the existing Record Reader to
> read up to 1024 row at a time.
> Future work will be to avoid going via old reader and convert straight the
> Json (smile format) to Vector primitive types.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20932) Vectorize Druid Storage Handler Reader
[
https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690354#comment-16690354
]
Hive QA commented on HIVE-20932:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} druid-handler in master has 4 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} The patch . passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} druid-handler: The patch generated 0 new + 0
unchanged - 193 fixed = 0 total (was 193) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
34s{color} | {color:red} druid-handler generated 1 new + 3 unchanged - 1 fixed
= 4 total (was 4) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 27s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:druid-handler |
| | Boxing/unboxing to parse a primitive
org.apache.hadoop.hive.druid.serde.DruidSerDe.doDeserialize(Writable) At
DruidSerDe.java:org.apache.hadoop.hive.druid.serde.DruidSerDe.doDeserialize(Writable)
At DruidSerDe.java:[line 452] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml javac javadoc findbugs checkstyle
compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14971/dev-support/hive-personality.sh
|
| git revision | master / fd5f34f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14971/yetus/new-findbugs-druid-handler.html
|
| modules | C: . druid-handler ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14971/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorize Druid Storage Handler Reader
> --
>
> Key: HIVE-20932
> URL: https://issues.apache.org/jira/browse/HIVE-20932
> Project: Hive
> Issue Type: Improvement
>
[jira] [Updated] (HIVE-20922) HiveServer2 user is switched to another occasionally
[ https://issues.apache.org/jira/browse/HIVE-20922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cao Junliang updated HIVE-20922: Component/s: Authentication > HiveServer2 user is switched to another occasionally > > > Key: HIVE-20922 > URL: https://issues.apache.org/jira/browse/HIVE-20922 > Project: Hive > Issue Type: Bug > Components: Authentication >Affects Versions: 1.2.1 > Environment: HiveServer2 version: 1.2.1 > Hive and yarn were deployed in same machine. >Reporter: Cao Junliang >Priority: Major > > The HiveServer2（version 1.2.1）authentication configuration is: > authentication type: kerberos > kerberos principal: hive/h...@dc1.fh.com > Other infomation: > client principal: n...@dc1.fh.com > yarn service principal:yarn/_h...@dc1.fh.com > After the hive service was started, application could use jdbc to access hive > service. But sometime later, clients got the exception. The user of the hive > service was switched to another one (yarn). > In my environment, Hive and Yarn were deployed in same machine, the user of > yarn service is yarn/_h...@dc1.fh.com. Before the exception occured, the hive > service was not restarted. > == > Main logs: > 2018-11-11 23:59:35 [HiveServer2-Handler-Pool: Thread-115739] INFO > org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(294) -Client > protocol version: HIVE_CLI_SERVICE_PROTOCOL_V8 > 2018-11-11 23:59:35 [HiveServer2-Handler-Pool: Thread-115739] WARN > org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(308) -Error > opening session: > org.apache.hive.service.cli.HiveSQLException: Failed to open new session: > java.lang.RuntimeException: java.lang.RuntimeException: > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): > User: yarn/mas...@dc1.fh.com is not allowed to impersonate nbhd > at > org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:266) > at > org.apache.hive.service.cli.CLIService.openSessionWithImpersonation(CLIService.java:202) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:402) > at > org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:297) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1253) > at > org.apache.hive.service.cli.thrift.TCLIService$Processor$OpenSession.getResult(TCLIService.java:1238) > at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) > at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) > at > org.apache.hadoop.hive.thrift.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:692) > at > org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException: java.lang.RuntimeException: > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): > User: yarn/mas...@dc1.fh.com is not allowed to impersonate nbhd > at > org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:83) > at > org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36) > at > org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) > at > org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59) > at com.sun.proxy.$Proxy19.open(Unknown Source) > at > org.apache.hive.service.cli.session.SessionManager.openSession(SessionManager.java:258) > ... 12 more > Caused by: java.lang.RuntimeException: > org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): > User: yarn/mas...@dc1.fh.com is not allowed to impersonate nbhd > at > org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522) > at > org.apache.hive.service.cli.session.HiveSessionImpl.open(HiveSessionImpl.java:137) > at sun.reflect.GeneratedMethodAccessor22.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at >
[jira] [Commented] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[
https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690211#comment-16690211
]
Jesus Camacho Rodriguez commented on HIVE-20920:
[~ashutoshc], can you take a look? I will create a follow-up to avoid creating
{{IS NOT NULL}} condition if column does not contain null values as per stats.
Thanks
> Use SQL constraints to improve join reordering algorithm (II)
> -
>
> Key: HIVE-20920
> URL: https://issues.apache.org/jira/browse/HIVE-20920
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20920.01.patch, HIVE-20920.02.patch,
> HIVE-20920.03.patch, HIVE-20920.patch
>
>
> Follow-up of HIVE-20744.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[
https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690203#comment-16690203
]
Hive QA commented on HIVE-20920:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948570/HIVE-20920.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15544 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14970/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14970/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14970/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948570 - PreCommit-HIVE-Build
> Use SQL constraints to improve join reordering algorithm (II)
> -
>
> Key: HIVE-20920
> URL: https://issues.apache.org/jira/browse/HIVE-20920
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20920.01.patch, HIVE-20920.02.patch,
> HIVE-20920.03.patch, HIVE-20920.patch
>
>
> Follow-up of HIVE-20744.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[
https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690188#comment-16690188
]
Hive QA commented on HIVE-20920:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 4 new + 20 unchanged - 0 fixed
= 24 total (was 20) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
54s{color} | {color:red} ql generated 1 new + 2316 unchanged - 0 fixed = 2317
total (was 2316) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Potentially dangerous use of non-short-circuit logic in
org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount$SimpleTreeOnJoinKey.visit(RelNode,
int, RelNode) At HiveRelMdRowCount.java:logic in
org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount$SimpleTreeOnJoinKey.visit(RelNode,
int, RelNode) At HiveRelMdRowCount.java:[line 583] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14970/dev-support/hive-personality.sh
|
| git revision | master / fd5f34f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14970/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14970/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14970/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use SQL constraints to improve join reordering algorithm (II)
> -
>
> Key: HIVE-20920
> URL: https://issues.apache.org/jira/browse/HIVE-20920
> Project: Hive
> Issue Type: Improvement
> Components: CBO
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-20920.01.patch, HIVE-20920.02.patch,
> HIVE-20920.03.patch, HIVE-20920.patch
>
>
> Follow-up of HIVE-20744.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20936) Allow the Worker thread in the metastore to run outside of it
[ https://issues.apache.org/jira/browse/HIVE-20936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-20936: --- Attachment: HIVE-20936.1.patch Status: Patch Available (was: Open) > Allow the Worker thread in the metastore to run outside of it > - > > Key: HIVE-20936 > URL: https://issues.apache.org/jira/browse/HIVE-20936 > Project: Hive > Issue Type: Improvement >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > Attachments: HIVE-20936.1.patch > > > Currently the Worker thread in the metastore in bounded to the metastore, > mainly because of the TxnHandler that it has. This thread runs some map > reduce jobs which may not only be an option wherever the metastore is > running. A solution for this can be to run this thread in HS2 depending on a > flag. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20932) Vectorize Druid Storage Handler Reader
[ https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20932: -- Attachment: (was: HIVE-20932.2.patch) > Vectorize Druid Storage Handler Reader > -- > > Key: HIVE-20932 > URL: https://issues.apache.org/jira/browse/HIVE-20932 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20932.3.patch, HIVE-20932.patch > > > This patch aims at adding support for vectorize read of data from Druid to > Hive. > [~t3rmin4t0r] suggested that this will improve the performance of the top > level operators that supports vectorization. > As a first cut am just adding a wrapper around the existing Record Reader to > read up to 1024 row at a time. > Future work will be to avoid going via old reader and convert straight the > Json (smile format) to Vector primitive types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20936) Allow the Worker thread in the metastore to run outside of it
[ https://issues.apache.org/jira/browse/HIVE-20936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reassigned HIVE-20936: -- Assignee: Jaume M > Allow the Worker thread in the metastore to run outside of it > - > > Key: HIVE-20936 > URL: https://issues.apache.org/jira/browse/HIVE-20936 > Project: Hive > Issue Type: Improvement >Reporter: Jaume M >Assignee: Jaume M >Priority: Major > > Currently the Worker thread in the metastore in bounded to the metastore, > mainly because of the TxnHandler that it has. This thread runs some map > reduce jobs which may not only be an option wherever the metastore is > running. A solution for this can be to run this thread in HS2 depending on a > flag. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20935) Upload of llap package tarball fails in EC2 causing LLAP service start failure
[
https://issues.apache.org/jira/browse/HIVE-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690181#comment-16690181
]
Hive QA commented on HIVE-20935:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948563/HIVE-20935.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15544 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_6]
(batchId=190)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14969/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14969/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14969/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948563 - PreCommit-HIVE-Build
> Upload of llap package tarball fails in EC2 causing LLAP service start failure
> --
>
> Key: HIVE-20935
> URL: https://issues.apache.org/jira/browse/HIVE-20935
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.1.1
>Reporter: Gour Saha
>Assignee: Gour Saha
>Priority: Major
> Attachments: HIVE-20935.01.patch
>
>
> Even though package dir is defined as below (with a / at the end) -
> {code}
> LLAP_PACKAGE_DIR = ".yarn/package/LLAP/";
> {code}
> copyLocalFileToHdfs API fails to create the dir hierarchy of
> .yarn/package/LLAP/ first and then copy the file under it. It instead uploads
> the file under .yarn/package with name LLAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20932) Vectorize Druid Storage Handler Reader
[ https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20932: -- Attachment: HIVE-20932.3.patch > Vectorize Druid Storage Handler Reader > -- > > Key: HIVE-20932 > URL: https://issues.apache.org/jira/browse/HIVE-20932 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20932.3.patch, HIVE-20932.patch > > > This patch aims at adding support for vectorize read of data from Druid to > Hive. > [~t3rmin4t0r] suggested that this will improve the performance of the top > level operators that supports vectorization. > As a first cut am just adding a wrapper around the existing Record Reader to > read up to 1024 row at a time. > Future work will be to avoid going via old reader and convert straight the > Json (smile format) to Vector primitive types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20935) Upload of llap package tarball fails in EC2 causing LLAP service start failure
[
https://issues.apache.org/jira/browse/HIVE-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690162#comment-16690162
]
Hive QA commented on HIVE-20935:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} llap-server in master has 81 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14969/dev-support/hive-personality.sh
|
| git revision | master / fd5f34f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14969/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upload of llap package tarball fails in EC2 causing LLAP service start failure
> --
>
> Key: HIVE-20935
> URL: https://issues.apache.org/jira/browse/HIVE-20935
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.1.1
>Reporter: Gour Saha
>Assignee: Gour Saha
>Priority: Major
> Attachments: HIVE-20935.01.patch
>
>
> Even though package dir is defined as below (with a / at the end) -
> {code}
> LLAP_PACKAGE_DIR = ".yarn/package/LLAP/";
> {code}
> copyLocalFileToHdfs API fails to create the dir hierarchy of
> .yarn/package/LLAP/ first and then copy the file under it. It instead uploads
> the file under .yarn/package with name LLAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20932) Vectorize Druid Storage Handler Reader
[ https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20932: -- Attachment: HIVE-20932.2.patch > Vectorize Druid Storage Handler Reader > -- > > Key: HIVE-20932 > URL: https://issues.apache.org/jira/browse/HIVE-20932 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20932.2.patch, HIVE-20932.patch > > > This patch aims at adding support for vectorize read of data from Druid to > Hive. > [~t3rmin4t0r] suggested that this will improve the performance of the top > level operators that supports vectorization. > As a first cut am just adding a wrapper around the existing Record Reader to > read up to 1024 row at a time. > Future work will be to avoid going via old reader and convert straight the > Json (smile format) to Vector primitive types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690154#comment-16690154
]
Hive QA commented on HIVE-20931:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948557/HIVE-20931.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15539 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=196)
[druidmini_dynamic_partition.q,druidmini_test1.q,druidkafkamini_basic.q,druidmini_test_insert.q,druidkafkamini_delimited.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14968/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14968/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14968/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948557 - PreCommit-HIVE-Build
> Minor code cleaning for Druid Storage Handler
> -
>
> Key: HIVE-20931
> URL: https://issues.apache.org/jira/browse/HIVE-20931
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20931.patch, HIVE-20931.patch, HIVE-20931.patch,
> HIVE-20931.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20920) Use SQL constraints to improve join reordering algorithm (II)
[ https://issues.apache.org/jira/browse/HIVE-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-20920: --- Attachment: HIVE-20920.03.patch > Use SQL constraints to improve join reordering algorithm (II) > - > > Key: HIVE-20920 > URL: https://issues.apache.org/jira/browse/HIVE-20920 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20920.01.patch, HIVE-20920.02.patch, > HIVE-20920.03.patch, HIVE-20920.patch > > > Follow-up of HIVE-20744. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690109#comment-16690109
]
Hive QA commented on HIVE-20931:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} druid-handler in master has 4 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} druid-handler: The patch generated 0 new + 16
unchanged - 2 fixed = 16 total (was 18) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14968/dev-support/hive-personality.sh
|
| git revision | master / fd5f34f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: druid-handler U: druid-handler |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14968/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Minor code cleaning for Druid Storage Handler
> -
>
> Key: HIVE-20931
> URL: https://issues.apache.org/jira/browse/HIVE-20931
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20931.patch, HIVE-20931.patch, HIVE-20931.patch,
> HIVE-20931.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20932) Vectorize Druid Storage Handler Reader
[
https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690101#comment-16690101
]
Hive QA commented on HIVE-20932:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948559/HIVE-20932.patch
{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15544 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
(batchId=182)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=334)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14967/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14967/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14967/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948559 - PreCommit-HIVE-Build
> Vectorize Druid Storage Handler Reader
> --
>
> Key: HIVE-20932
> URL: https://issues.apache.org/jira/browse/HIVE-20932
> Project: Hive
> Issue Type: Improvement
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20932.patch
>
>
> This patch aims at adding support for vectorize read of data from Druid to
> Hive.
> [~t3rmin4t0r] suggested that this will improve the performance of the top
> level operators that supports vectorization.
> As a first cut am just adding a wrapper around the existing Record Reader to
> read up to 1024 row at a time.
> Future work will be to avoid going via old reader and convert straight the
> Json (smile format) to Vector primitive types.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20916) Fix typo in JSONCreateDatabaseMessage and add test for alter database
[
https://issues.apache.org/jira/browse/HIVE-20916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-20916:
---
Resolution: Fixed
Fix Version/s: 3.2.0
4.0.0
Status: Resolved (was: Patch Available)
Patch merged into master and branch-3. Thanks for the review [~bharos92] and
[~asherman]
> Fix typo in JSONCreateDatabaseMessage and add test for alter database
> -
>
> Key: HIVE-20916
> URL: https://issues.apache.org/jira/browse/HIVE-20916
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Fix For: 4.0.0, 3.2.0
>
> Attachments: HIVE-20916.01.patch, HIVE-20916.02.patch,
> HIVE-20916.03.patch
>
>
> {code}
> public JSONCreateDatabaseMessage(String server, String servicePrincipal,
> Database db,
> Long timestamp) {
> this.server = server;
> this.servicePrincipal = servicePrincipal;
> this.db = db.getName();
> this.timestamp = timestamp;
> try {
> this.dbJson = MessageBuilder.createDatabaseObjJson(db);
> } catch (TException ex) {
> throw new IllegalArgumentException("Could not serialize Function
> object", ex);
> }
> checkValid();
> }
> {code}
> The exception message should say Database instead of Function. Also, the
> {{TestDbNotificationListener#createDatabase}} should be modified to make sure
> that the deserialized database object from the dbJson field matches with the
> original database object
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20935) Upload of llap package tarball fails in EC2 causing LLAP service start failure
[
https://issues.apache.org/jira/browse/HIVE-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690066#comment-16690066
]
Prasanth Jayachandran commented on HIVE-20935:
--
+1
> Upload of llap package tarball fails in EC2 causing LLAP service start failure
> --
>
> Key: HIVE-20935
> URL: https://issues.apache.org/jira/browse/HIVE-20935
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.1.1
>Reporter: Gour Saha
>Assignee: Gour Saha
>Priority: Major
> Attachments: HIVE-20935.01.patch
>
>
> Even though package dir is defined as below (with a / at the end) -
> {code}
> LLAP_PACKAGE_DIR = ".yarn/package/LLAP/";
> {code}
> copyLocalFileToHdfs API fails to create the dir hierarchy of
> .yarn/package/LLAP/ first and then copy the file under it. It instead uploads
> the file under .yarn/package with name LLAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20935) Upload of llap package tarball fails in EC2 causing LLAP service start failure
[
https://issues.apache.org/jira/browse/HIVE-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690062#comment-16690062
]
Gour Saha commented on HIVE-20935:
--
Uploaded a patch to explicitly create the dir before trying to copy the file.
> Upload of llap package tarball fails in EC2 causing LLAP service start failure
> --
>
> Key: HIVE-20935
> URL: https://issues.apache.org/jira/browse/HIVE-20935
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.1.1
>Reporter: Gour Saha
>Assignee: Gour Saha
>Priority: Major
> Attachments: HIVE-20935.01.patch
>
>
> Even though package dir is defined as below (with a / at the end) -
> {code}
> LLAP_PACKAGE_DIR = ".yarn/package/LLAP/";
> {code}
> copyLocalFileToHdfs API fails to create the dir hierarchy of
> .yarn/package/LLAP/ first and then copy the file under it. It instead uploads
> the file under .yarn/package with name LLAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18935) LLAP: add jmx view of what's in cache
[ https://issues.apache.org/jira/browse/HIVE-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18935: --- Assignee: (was: Sergey Shelukhin) > LLAP: add jmx view of what's in cache > - > > Key: HIVE-18935 > URL: https://issues.apache.org/jira/browse/HIVE-18935 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Priority: Major > > Based on HIVE-16992 work -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20935) Upload of llap package tarball fails in EC2 causing LLAP service start failure
[
https://issues.apache.org/jira/browse/HIVE-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gour Saha updated HIVE-20935:
-
Status: Patch Available (was: Open)
> Upload of llap package tarball fails in EC2 causing LLAP service start failure
> --
>
> Key: HIVE-20935
> URL: https://issues.apache.org/jira/browse/HIVE-20935
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.1.1
>Reporter: Gour Saha
>Assignee: Gour Saha
>Priority: Major
> Attachments: HIVE-20935.01.patch
>
>
> Even though package dir is defined as below (with a / at the end) -
> {code}
> LLAP_PACKAGE_DIR = ".yarn/package/LLAP/";
> {code}
> copyLocalFileToHdfs API fails to create the dir hierarchy of
> .yarn/package/LLAP/ first and then copy the file under it. It instead uploads
> the file under .yarn/package with name LLAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20935) Upload of llap package tarball fails in EC2 causing LLAP service start failure
[
https://issues.apache.org/jira/browse/HIVE-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gour Saha updated HIVE-20935:
-
Attachment: HIVE-20935.01.patch
> Upload of llap package tarball fails in EC2 causing LLAP service start failure
> --
>
> Key: HIVE-20935
> URL: https://issues.apache.org/jira/browse/HIVE-20935
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.1.1
>Reporter: Gour Saha
>Assignee: Gour Saha
>Priority: Major
> Attachments: HIVE-20935.01.patch
>
>
> Even though package dir is defined as below (with a / at the end) -
> {code}
> LLAP_PACKAGE_DIR = ".yarn/package/LLAP/";
> {code}
> copyLocalFileToHdfs API fails to create the dir hierarchy of
> .yarn/package/LLAP/ first and then copy the file under it. It instead uploads
> the file under .yarn/package with name LLAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20935) Upload of llap package tarball fails in EC2 causing LLAP service start failure
[
https://issues.apache.org/jira/browse/HIVE-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gour Saha reassigned HIVE-20935:
Assignee: Gour Saha
> Upload of llap package tarball fails in EC2 causing LLAP service start failure
> --
>
> Key: HIVE-20935
> URL: https://issues.apache.org/jira/browse/HIVE-20935
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.1.1
>Reporter: Gour Saha
>Assignee: Gour Saha
>Priority: Major
>
> Even though package dir is defined as below (with a / at the end) -
> {code}
> LLAP_PACKAGE_DIR = ".yarn/package/LLAP/";
> {code}
> copyLocalFileToHdfs API fails to create the dir hierarchy of
> .yarn/package/LLAP/ first and then copy the file under it. It instead uploads
> the file under .yarn/package with name LLAP.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20932) Vectorize Druid Storage Handler Reader
[
https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690056#comment-16690056
]
Hive QA commented on HIVE-20932:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} druid-handler in master has 4 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} druid-handler: The patch generated 3 new + 77
unchanged - 2 fixed = 80 total (was 79) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
32s{color} | {color:red} druid-handler generated 1 new + 3 unchanged - 1 fixed
= 4 total (was 4) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:druid-handler |
| | Boxing/unboxing to parse a primitive
org.apache.hadoop.hive.druid.serde.DruidSerDe.doDeserialize(Writable) At
DruidSerDe.java:org.apache.hadoop.hive.druid.serde.DruidSerDe.doDeserialize(Writable)
At DruidSerDe.java:[line 452] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14967/dev-support/hive-personality.sh
|
| git revision | master / fd5f34f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14967/yetus/diff-checkstyle-druid-handler.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14967/yetus/new-findbugs-druid-handler.html
|
| modules | C: druid-handler ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14967/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorize Druid Storage Handler Reader
> --
>
> Key: HIVE-20932
> URL: https://issues.apache.org/jira/browse/HIVE-20932
> Project: Hive
> Issue Type: Improvement
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20932.patch
>
>
> This patch aims at adding support for vectorize read of data from Druid to
> Hive.
> [~t3rmin4t0r] suggested that this will improve the performance of the top
> level operators that supports vectorization.
>
[jira] [Updated] (HIVE-20932) Vectorize Druid Storage Handler Reader
[ https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20932: -- Attachment: (was: HIVE-20931.patch) > Vectorize Druid Storage Handler Reader > -- > > Key: HIVE-20932 > URL: https://issues.apache.org/jira/browse/HIVE-20932 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20932.patch > > > This patch aims at adding support for vectorize read of data from Druid to > Hive. > [~t3rmin4t0r] suggested that this will improve the performance of the top > level operators that supports vectorization. > As a first cut am just adding a wrapper around the existing Record Reader to > read up to 1024 row at a time. > Future work will be to avoid going via old reader and convert straight the > Json (smile format) to Vector primitive types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20932) Vectorize Druid Storage Handler Reader
[ https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20932: -- Attachment: HIVE-20931.patch > Vectorize Druid Storage Handler Reader > -- > > Key: HIVE-20932 > URL: https://issues.apache.org/jira/browse/HIVE-20932 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20932.patch > > > This patch aims at adding support for vectorize read of data from Druid to > Hive. > [~t3rmin4t0r] suggested that this will improve the performance of the top > level operators that supports vectorization. > As a first cut am just adding a wrapper around the existing Record Reader to > read up to 1024 row at a time. > Future work will be to avoid going via old reader and convert straight the > Json (smile format) to Vector primitive types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20932) Vectorize Druid Storage Handler Reader
[ https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20932: -- Attachment: HIVE-20932.patch > Vectorize Druid Storage Handler Reader > -- > > Key: HIVE-20932 > URL: https://issues.apache.org/jira/browse/HIVE-20932 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20932.patch > > > This patch aims at adding support for vectorize read of data from Druid to > Hive. > [~t3rmin4t0r] suggested that this will improve the performance of the top > level operators that supports vectorization. > As a first cut am just adding a wrapper around the existing Record Reader to > read up to 1024 row at a time. > Future work will be to avoid going via old reader and convert straight the > Json (smile format) to Vector primitive types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20932) Vectorize Druid Storage Handler Reader
[ https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20932: -- Status: Patch Available (was: Open) > Vectorize Druid Storage Handler Reader > -- > > Key: HIVE-20932 > URL: https://issues.apache.org/jira/browse/HIVE-20932 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20932.patch > > > This patch aims at adding support for vectorize read of data from Druid to > Hive. > [~t3rmin4t0r] suggested that this will improve the performance of the top > level operators that supports vectorization. > As a first cut am just adding a wrapper around the existing Record Reader to > read up to 1024 row at a time. > Future work will be to avoid going via old reader and convert straight the > Json (smile format) to Vector primitive types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20931: -- Attachment: HIVE-20931.patch > Minor code cleaning for Druid Storage Handler > - > > Key: HIVE-20931 > URL: https://issues.apache.org/jira/browse/HIVE-20931 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20931.patch, HIVE-20931.patch, HIVE-20931.patch, > HIVE-20931.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16690018#comment-16690018
]
Hive QA commented on HIVE-20931:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948543/HIVE-20931.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15544 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
(batchId=182)
org.apache.hive.spark.client.rpc.TestRpc.testClientTimeout (batchId=328)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14966/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14966/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14966/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948543 - PreCommit-HIVE-Build
> Minor code cleaning for Druid Storage Handler
> -
>
> Key: HIVE-20931
> URL: https://issues.apache.org/jira/browse/HIVE-20931
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20931.patch, HIVE-20931.patch, HIVE-20931.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HIVE-20923) masking_12 qtest expects wrong inputs
[
https://issues.apache.org/jira/browse/HIVE-20923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez resolved HIVE-20923.
Resolution: Not A Problem
Closing as not a problem, please reopen if you still believe this is an issue.
> masking_12 qtest expects wrong inputs
> -
>
> Key: HIVE-20923
> URL: https://issues.apache.org/jira/browse/HIVE-20923
> Project: Hive
> Issue Type: Bug
> Components: Query Planning, Test, Tests, Views
>Reporter: Karen Coppage
>Priority: Major
>
> It appears this qtest expects the wrong behavior, and should fail but doesn't.
> Below Queries 3 and 4 should not include default@src as Input
> Query 1
> {noformat}
> PREHOOK: query: create table `masking_test_subq_n1` as select cast(key as
> int) as key, value from src
> PREHOOK: type: CREATETABLE_AS_SELECT
> PREHOOK: Input: default@src
> PREHOOK: Output: database:default
> PREHOOK: Output: default@masking_test_subq_n1
> POSTHOOK: query: create table `masking_test_subq_n1` as select cast(key as
> int) as key, value from src
> POSTHOOK: type: CREATETABLE_AS_SELECT
> POSTHOOK: Input: default@src
> POSTHOOK: Output: database:default
> POSTHOOK: Output: default@masking_test_subq_n1
> POSTHOOK: Lineage: masking_test_subq_n1.key EXPRESSION
> [(src)src.FieldSchema(name:key, type:string, comment:default), ]
> POSTHOOK: Lineage: masking_test_subq_n1.value SIMPLE
> [(src)src.FieldSchema(name:value, type:string, comment:default), ]{noformat}
> Query 2
> {noformat}
> PREHOOK: query: create view `v1_n9` as select * from `masking_test_subq_n1`
> PREHOOK: type: CREATEVIEW
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Output: database:default
> PREHOOK: Output: default@v1_n9
> POSTHOOK: query: create view `v1_n9` as select * from `masking_test_subq_n1`
> POSTHOOK: type: CREATEVIEW
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Output: database:default
> POSTHOOK: Output: default@v1_n9
> POSTHOOK: Lineage: v1_n9.key SIMPLE
> [(masking_test_subq_n1)masking_test_subq_n1.FieldSchema(name:key, type:int,
> comment:null), ]
> POSTHOOK: Lineage: v1_n9.value SIMPLE
> [(masking_test_subq_n1)masking_test_subq_n1.FieldSchema(name:value,
> type:string, comment:null), ]
> {noformat}
> Query 3 (the explain output here describes a TableScan on table src)
> {noformat}
> PREHOOK: query: explain
> select * from `v1_n9`
> limit 20
> PREHOOK: type: QUERY
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Input: default@src
> PREHOOK: Input: default@v1_n9
> A masked pattern was here
> POSTHOOK: query: explain
> select * from `v1_n9`
> limit 20
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Input: default@src
> POSTHOOK: Input: default@v1_n9
> {noformat}
> Query 4
> {noformat}
> PREHOOK: query: select * from `v1_n9`
> limit 20
> PREHOOK: type: QUERY
> PREHOOK: Input: default@masking_test_subq_n1
> PREHOOK: Input: default@src
> PREHOOK: Input: default@v1_n9
> A masked pattern was here
> POSTHOOK: query: select * from `v1_n9`
> limit 20
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@masking_test_subq_n1
> POSTHOOK: Input: default@src
> POSTHOOK: Input: default@v1_n9
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20933) Disable minor compactions when using query based compactor
[ https://issues.apache.org/jira/browse/HIVE-20933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-20933: Description: HIVE-20699 introduces a new config, which can enable major compaction to be run as a HiveQL query, so that it can take advantage of the underlying execution engine and not run compaction as an MR job. This is particularly useful for cloud deployments. For now, we will disable minor compactions if this config flag is enabled. We will work on enabling minor compactions in HIVE-20934. (was: HIVE-20699 introduces a new config, which can enable major compaction to be run as a HiveQL query, so that it can take advantage of the underlying execution engine and not run compaction as an MR job. This is particularly useful for cloud deployments. For now, we will disable minor compactions if this config flag is enabled. We will work on enabling minor compactions in a separate jira.) > Disable minor compactions when using query based compactor > --- > > Key: HIVE-20933 > URL: https://issues.apache.org/jira/browse/HIVE-20933 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > > HIVE-20699 introduces a new config, which can enable major compaction to be > run as a HiveQL query, so that it can take advantage of the underlying > execution engine and not run compaction as an MR job. This is particularly > useful for cloud deployments. For now, we will disable minor compactions if > this config flag is enabled. We will work on enabling minor compactions in > HIVE-20934. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20933) Disable minor compactions when using query based compactor
[ https://issues.apache.org/jira/browse/HIVE-20933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689990#comment-16689990 ] Vaibhav Gumashta commented on HIVE-20933: - cc [~ekoifman] > Disable minor compactions when using query based compactor > --- > > Key: HIVE-20933 > URL: https://issues.apache.org/jira/browse/HIVE-20933 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > > HIVE-20699 introduces a new config, which can enable major compaction to be > run as a HiveQL query, so that it can take advantage of the underlying > execution engine and not run compaction as an MR job. This is particularly > useful for cloud deployments. For now, we will disable minor compactions if > this config flag is enabled. We will work on enabling minor compactions in a > separate jira. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20934) Query based compactor for full CRUD Acid tables
[ https://issues.apache.org/jira/browse/HIVE-20934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta reassigned HIVE-20934: --- Assignee: Vaibhav Gumashta > Query based compactor for full CRUD Acid tables > --- > > Key: HIVE-20934 > URL: https://issues.apache.org/jira/browse/HIVE-20934 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > > Follow up of HIVE-20699. This is to enable running minor compactions as a > HiveQL query -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20933) Disable minor compactions when using query based compactor
[ https://issues.apache.org/jira/browse/HIVE-20933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta reassigned HIVE-20933: --- Assignee: Vaibhav Gumashta > Disable minor compactions when using query based compactor > --- > > Key: HIVE-20933 > URL: https://issues.apache.org/jira/browse/HIVE-20933 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta >Priority: Major > > HIVE-20699 introduces a new config, which can enable major compaction to be > run as a HiveQL query, so that it can take advantage of the underlying > execution engine and not run compaction as an MR job. This is particularly > useful for cloud deployments. For now, we will disable minor compactions if > this config flag is enabled. We will work on enabling minor compactions in a > separate jira. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689968#comment-16689968
]
Hive QA commented on HIVE-20931:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} druid-handler in master has 4 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} druid-handler: The patch generated 0 new + 16
unchanged - 2 fixed = 16 total (was 18) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14966/dev-support/hive-personality.sh
|
| git revision | master / fd5f34f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: druid-handler U: druid-handler |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14966/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Minor code cleaning for Druid Storage Handler
> -
>
> Key: HIVE-20931
> URL: https://issues.apache.org/jira/browse/HIVE-20931
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20931.patch, HIVE-20931.patch, HIVE-20931.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20931: -- Attachment: HIVE-20931.patch > Minor code cleaning for Druid Storage Handler > - > > Key: HIVE-20931 > URL: https://issues.apache.org/jira/browse/HIVE-20931 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20931.patch, HIVE-20931.patch, HIVE-20931.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20794) Use Zookeeper for metastore service discovery
[
https://issues.apache.org/jira/browse/HIVE-20794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689932#comment-16689932
]
Hive QA commented on HIVE-20794:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948532/HIVE-20794.02
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15588 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_limit]
(batchId=171)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14965/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14965/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14965/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948532 - PreCommit-HIVE-Build
> Use Zookeeper for metastore service discovery
> -
>
> Key: HIVE-20794
> URL: https://issues.apache.org/jira/browse/HIVE-20794
> Project: Hive
> Issue Type: Improvement
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20794.01, HIVE-20794.02
>
>
> Right now, multiple metastore services can be specified in
> hive.metastore.uris configuration, but that list is static and can not be
> modified dynamically. Use Zookeeper for dynamic service discovery of
> metastore.
> h3. Improve ZooKeeperHiveHelper class (suggestions for name welcome)
> The Zookeeper related code (for service discovery) accesses Zookeeper
> parameters directly from HiveConf. The class is changed so that it could be
> used for both HiveServer2 and Metastore server and works with both the
> configurations. Following methods from HiveServer2 are now moved into
> ZooKeeperHiveHelper. # startZookeeperClient # addServerInstanceToZooKeeper #
> removeServerInstanceFromZooKeeper
> h3. HiveMetaStore conf changes
> # THRIFT_URIS (hive.metastore.uris) can also be used to specify ZooKeeper
> quorum. When THRIFT_SERVICE_DISCOVERY_MODE
> (hive.metastore.service.discovery.mode) is set to "zookeeper" the URIs are
> used as ZooKeeper quorum. When it's set to be empty, the URIs are used to
> locate the metastore directly.
> # Here's list of Hiveserver2's parameters and their proposed metastore conf
> counterparts. It looks odd that the Metastore related configurations do not
> have their macros start with METASTORE, but start with THRIFT. I have just
> followed naming convention used for other parameters.
> #* HIVE_SERVER2_ZOOKEEPER_NAMESPACE - THRIFT_ZOOKEEPER_NAMESPACE
> (hive.metastore.zookeeper.namespace)
> #* HIVE_ZOOKEEPER_CLIENT_PORT - THRIFT_ZOOKEEPER_CLIENT_PORT
> (hive.metastore.zookeeper.client.port)
> #* HIVE_ZOOKEEPER_CONNECTION_TIMEOUT - THRIFT_ZOOKEEPER_CONNECTION_TIMEOUT -
> (hive.metastore.zookeeper.connection.timeout)
> #* HIVE_ZOOKEEPER_CONNECTION_MAX_RETRIES -
> THRIFT_ZOOKEEPER_CONNECTION_MAX_RETRIES
> (hive.metastore.zookeeper.connection.max.retries)
> #* HIVE_ZOOKEEPER_CONNECTION_BASESLEEPTIME -
> THRIFT_ZOOKEEPER_CONNECTION_BASESLEEPTIME
> (hive.metastore.zookeeper.connection.basesleeptime)
> Following Hive ZK configurations seem to be related to managing locks and
> seem irrelevant for MS ZK.
> # HIVE_ZOOKEEPER_SESSION_TIMEOUT
> # HIVE_ZOOKEEPER_CLEAN_EXTRA_NODES
> Since there is no configuration to be published,
> HIVE_ZOOKEEPER_PUBLISH_CONFIGS does not have a THRIFT counterpart.
> h3. HiveMetaStore class changes
> # startMetaStore should also register the instance with Zookeeper, when
> configured.
> # When shutting a metastore server down it should deregister itself from
> Zookeeper, when configured.
> # These changes use the refactored code described above.
> h3. HiveMetaStoreClient class changes
> When service discovery mode is zookeeper, we fetch the metatstore URIs from
> the specified ZooKeeper and treat those as if they were specified in
> THRIFT_URIS i.e. use the existing mechanisms to choose a metastore server to
> connect to and establish a connection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20794) Use Zookeeper for metastore service discovery
[
https://issues.apache.org/jira/browse/HIVE-20794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689923#comment-16689923
]
Hive QA commented on HIVE-20794:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
15s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/util in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
3s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} The patch standalone-metastore passed checkstyle
{color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch util passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 17 unchanged - 4
fixed = 17 total (was 21) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} service: The patch generated 3 new + 35 unchanged - 0
fixed = 38 total (was 35) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
7s{color} | {color:green} The patch metastore-server passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
45s{color} | {color:red} service generated 1 new + 48 unchanged - 0 fixed = 49
total (was 48) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 4m
13s{color} | {color:green} the patch
[jira] [Commented] (HIVE-20916) Fix typo in JSONCreateDatabaseMessage and add test for alter database
[
https://issues.apache.org/jira/browse/HIVE-20916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689836#comment-16689836
]
Andrew Sherman commented on HIVE-20916:
---
+1 LGTM
> Fix typo in JSONCreateDatabaseMessage and add test for alter database
> -
>
> Key: HIVE-20916
> URL: https://issues.apache.org/jira/browse/HIVE-20916
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 4.0.0
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-20916.01.patch, HIVE-20916.02.patch,
> HIVE-20916.03.patch
>
>
> {code}
> public JSONCreateDatabaseMessage(String server, String servicePrincipal,
> Database db,
> Long timestamp) {
> this.server = server;
> this.servicePrincipal = servicePrincipal;
> this.db = db.getName();
> this.timestamp = timestamp;
> try {
> this.dbJson = MessageBuilder.createDatabaseObjJson(db);
> } catch (TException ex) {
> throw new IllegalArgumentException("Could not serialize Function
> object", ex);
> }
> checkValid();
> }
> {code}
> The exception message should say Database instead of Function. Also, the
> {{TestDbNotificationListener#createDatabase}} should be modified to make sure
> that the deserialized database object from the dbJson field matches with the
> original database object
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20794) Use Zookeeper for metastore service discovery
[ https://issues.apache.org/jira/browse/HIVE-20794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20794: -- Attachment: HIVE-20794.02 Status: Patch Available (was: In Progress) Patch fixing the ptest failures from the last ptest run and checkstyle errors. > Use Zookeeper for metastore service discovery > - > > Key: HIVE-20794 > URL: https://issues.apache.org/jira/browse/HIVE-20794 > Project: Hive > Issue Type: Improvement >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20794.01, HIVE-20794.02 > > > Right now, multiple metastore services can be specified in > hive.metastore.uris configuration, but that list is static and can not be > modified dynamically. Use Zookeeper for dynamic service discovery of > metastore. > h3. Improve ZooKeeperHiveHelper class (suggestions for name welcome) > The Zookeeper related code (for service discovery) accesses Zookeeper > parameters directly from HiveConf. The class is changed so that it could be > used for both HiveServer2 and Metastore server and works with both the > configurations. Following methods from HiveServer2 are now moved into > ZooKeeperHiveHelper. # startZookeeperClient # addServerInstanceToZooKeeper # > removeServerInstanceFromZooKeeper > h3. HiveMetaStore conf changes > # THRIFT_URIS (hive.metastore.uris) can also be used to specify ZooKeeper > quorum. When THRIFT_SERVICE_DISCOVERY_MODE > (hive.metastore.service.discovery.mode) is set to "zookeeper" the URIs are > used as ZooKeeper quorum. When it's set to be empty, the URIs are used to > locate the metastore directly. > # Here's list of Hiveserver2's parameters and their proposed metastore conf > counterparts. It looks odd that the Metastore related configurations do not > have their macros start with METASTORE, but start with THRIFT. I have just > followed naming convention used for other parameters. > #* HIVE_SERVER2_ZOOKEEPER_NAMESPACE - THRIFT_ZOOKEEPER_NAMESPACE > (hive.metastore.zookeeper.namespace) > #* HIVE_ZOOKEEPER_CLIENT_PORT - THRIFT_ZOOKEEPER_CLIENT_PORT > (hive.metastore.zookeeper.client.port) > #* HIVE_ZOOKEEPER_CONNECTION_TIMEOUT - THRIFT_ZOOKEEPER_CONNECTION_TIMEOUT - > (hive.metastore.zookeeper.connection.timeout) > #* HIVE_ZOOKEEPER_CONNECTION_MAX_RETRIES - > THRIFT_ZOOKEEPER_CONNECTION_MAX_RETRIES > (hive.metastore.zookeeper.connection.max.retries) > #* HIVE_ZOOKEEPER_CONNECTION_BASESLEEPTIME - > THRIFT_ZOOKEEPER_CONNECTION_BASESLEEPTIME > (hive.metastore.zookeeper.connection.basesleeptime) > Following Hive ZK configurations seem to be related to managing locks and > seem irrelevant for MS ZK. > # HIVE_ZOOKEEPER_SESSION_TIMEOUT > # HIVE_ZOOKEEPER_CLEAN_EXTRA_NODES > Since there is no configuration to be published, > HIVE_ZOOKEEPER_PUBLISH_CONFIGS does not have a THRIFT counterpart. > h3. HiveMetaStore class changes > # startMetaStore should also register the instance with Zookeeper, when > configured. > # When shutting a metastore server down it should deregister itself from > Zookeeper, when configured. > # These changes use the refactored code described above. > h3. HiveMetaStoreClient class changes > When service discovery mode is zookeeper, we fetch the metatstore URIs from > the specified ZooKeeper and treat those as if they were specified in > THRIFT_URIS i.e. use the existing mechanisms to choose a metastore server to > connect to and establish a connection. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20794) Use Zookeeper for metastore service discovery
[ https://issues.apache.org/jira/browse/HIVE-20794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-20794: -- Status: In Progress (was: Patch Available) > Use Zookeeper for metastore service discovery > - > > Key: HIVE-20794 > URL: https://issues.apache.org/jira/browse/HIVE-20794 > Project: Hive > Issue Type: Improvement >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20794.01 > > > Right now, multiple metastore services can be specified in > hive.metastore.uris configuration, but that list is static and can not be > modified dynamically. Use Zookeeper for dynamic service discovery of > metastore. > h3. Improve ZooKeeperHiveHelper class (suggestions for name welcome) > The Zookeeper related code (for service discovery) accesses Zookeeper > parameters directly from HiveConf. The class is changed so that it could be > used for both HiveServer2 and Metastore server and works with both the > configurations. Following methods from HiveServer2 are now moved into > ZooKeeperHiveHelper. # startZookeeperClient # addServerInstanceToZooKeeper # > removeServerInstanceFromZooKeeper > h3. HiveMetaStore conf changes > # THRIFT_URIS (hive.metastore.uris) can also be used to specify ZooKeeper > quorum. When THRIFT_SERVICE_DISCOVERY_MODE > (hive.metastore.service.discovery.mode) is set to "zookeeper" the URIs are > used as ZooKeeper quorum. When it's set to be empty, the URIs are used to > locate the metastore directly. > # Here's list of Hiveserver2's parameters and their proposed metastore conf > counterparts. It looks odd that the Metastore related configurations do not > have their macros start with METASTORE, but start with THRIFT. I have just > followed naming convention used for other parameters. > #* HIVE_SERVER2_ZOOKEEPER_NAMESPACE - THRIFT_ZOOKEEPER_NAMESPACE > (hive.metastore.zookeeper.namespace) > #* HIVE_ZOOKEEPER_CLIENT_PORT - THRIFT_ZOOKEEPER_CLIENT_PORT > (hive.metastore.zookeeper.client.port) > #* HIVE_ZOOKEEPER_CONNECTION_TIMEOUT - THRIFT_ZOOKEEPER_CONNECTION_TIMEOUT - > (hive.metastore.zookeeper.connection.timeout) > #* HIVE_ZOOKEEPER_CONNECTION_MAX_RETRIES - > THRIFT_ZOOKEEPER_CONNECTION_MAX_RETRIES > (hive.metastore.zookeeper.connection.max.retries) > #* HIVE_ZOOKEEPER_CONNECTION_BASESLEEPTIME - > THRIFT_ZOOKEEPER_CONNECTION_BASESLEEPTIME > (hive.metastore.zookeeper.connection.basesleeptime) > Following Hive ZK configurations seem to be related to managing locks and > seem irrelevant for MS ZK. > # HIVE_ZOOKEEPER_SESSION_TIMEOUT > # HIVE_ZOOKEEPER_CLEAN_EXTRA_NODES > Since there is no configuration to be published, > HIVE_ZOOKEEPER_PUBLISH_CONFIGS does not have a THRIFT counterpart. > h3. HiveMetaStore class changes > # startMetaStore should also register the instance with Zookeeper, when > configured. > # When shutting a metastore server down it should deregister itself from > Zookeeper, when configured. > # These changes use the refactored code described above. > h3. HiveMetaStoreClient class changes > When service discovery mode is zookeeper, we fetch the metatstore URIs from > the specified ZooKeeper and treat those as if they were specified in > THRIFT_URIS i.e. use the existing mechanisms to choose a metastore server to > connect to and establish a connection. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689787#comment-16689787
]
Hive QA commented on HIVE-20931:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948520/HIVE-20931.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15543 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
(batchId=182)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14964/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14964/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14964/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948520 - PreCommit-HIVE-Build
> Minor code cleaning for Druid Storage Handler
> -
>
> Key: HIVE-20931
> URL: https://issues.apache.org/jira/browse/HIVE-20931
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20931.patch, HIVE-20931.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689688#comment-16689688
]
Hive QA commented on HIVE-20931:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} druid-handler in master has 4 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} druid-handler: The patch generated 0 new + 16
unchanged - 2 fixed = 16 total (was 18) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14964/dev-support/hive-personality.sh
|
| git revision | master / c737dbd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: druid-handler U: druid-handler |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14964/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Minor code cleaning for Druid Storage Handler
> -
>
> Key: HIVE-20931
> URL: https://issues.apache.org/jira/browse/HIVE-20931
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20931.patch, HIVE-20931.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20931: -- Attachment: HIVE-20931.patch > Minor code cleaning for Druid Storage Handler > - > > Key: HIVE-20931 > URL: https://issues.apache.org/jira/browse/HIVE-20931 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20931.patch, HIVE-20931.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689609#comment-16689609
]
Hive QA commented on HIVE-20931:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948504/HIVE-20931.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15543 tests
executed
*Failed tests:*
{noformat}
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=197)
[druidmini_masking.q,druidmini_joins.q,druid_timestamptz.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14963/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14963/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14963/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948504 - PreCommit-HIVE-Build
> Minor code cleaning for Druid Storage Handler
> -
>
> Key: HIVE-20931
> URL: https://issues.apache.org/jira/browse/HIVE-20931
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20931.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20304) When hive.optimize.skewjoin and hive.auto.convert.join are both set to true, and the execution engine is mr, same stage may launch twice due to the wrong generated plan
[
https://issues.apache.org/jira/browse/HIVE-20304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yongzhi Chen updated HIVE-20304:
Resolution: Duplicate
Status: Resolved (was: Patch Available)
> When hive.optimize.skewjoin and hive.auto.convert.join are both set to true,
> and the execution engine is mr, same stage may launch twice due to the wrong
> generated plan
>
>
> Key: HIVE-20304
> URL: https://issues.apache.org/jira/browse/HIVE-20304
> Project: Hive
> Issue Type: Bug
> Components: CLI
>Affects Versions: 1.2.1, 2.3.3
>Reporter: Hui Huang
>Assignee: Hui Huang
>Priority: Major
> Fix For: 4.0.0, 1.2.1
>
> Attachments: HIVE-20304.1.patch, HIVE-20304.2.patch, HIVE-20304.patch
>
>
> `When hive.optimize.skewjoin and hive.auto.convert.join are both set to true,
> and the execution engine is set to mr, same stage of a query may launch twice
> due to the wrong generated plan. If hive.exec.parallel is also true, the same
> stage will launch at the same time and the job will failed due to the first
> completed stage clear the map.xml/reduce.xml file stored in the hdfs.
> use following sql to reproduce the issue:
> {code:java}
> CREATE TABLE `tbl1`(
> `fence` string);
> CREATE TABLE `tbl2`(
> `order_id` string,
> `phone` string,
> `search_id` string
> )
> PARTITIONED BY (
> `dt` string);
> CREATE TABLE `tbl3`(
> `order_id` string,
> `platform` string)
> PARTITIONED BY (
> `dt` string);
> CREATE TABLE `tbl4`(
> `groupname` string,
> `phone` string)
> PARTITIONED BY (
> `dt` string);
> CREATE TABLE `tbl5`(
> `search_id` string,
> `fence` string)
> PARTITIONED BY (
> `dt` string);
> SET hive.exec.parallel = TRUE;
> SET hive.auto.convert.join = TRUE;
> SET hive.optimize.skewjoin = TRUE;
> SELECT dt,
>platform,
>groupname,
>count(1) as cnt
> FROM
> (SELECT dt,
> platform,
> groupname
> FROM
> (SELECT fence
> FROM tbl1)ta
>JOIN
>(SELECT a0.dt,
>a1.platform,
>a2.groupname,
>a3.fence
> FROM
> (SELECT dt,
> order_id,
> phone,
> search_id
> FROM tbl2
> WHERE dt =20180703 )a0
> JOIN
> (SELECT order_id,
> platform,
> dt
>FROM tbl3
>WHERE dt =20180703 )a1 ON a0.order_id = a1.order_id
> INNER JOIN
> (SELECT groupname,
> phone,
> dt
>FROM tbl4
>WHERE dt =20180703 )a2 ON a0.phone = a2.phone
> LEFT JOIN
> (SELECT search_id,
> fence,
> dt
>FROM tbl5
>WHERE dt =20180703)a3 ON a0.search_id = a3.search_id)t0 ON
> ta.fence = t0.fence)t11
> GROUP BY dt,
> platform,
> groupname;
> DROP TABLE tbl1;
> DROP TABLE tbl2;
> DROP TABLE tbl3;
> DROP TABLE tbl4;
> DROP TABLE tbl5;
> {code}
> We will get some error message like this:
> Examining task ID: task_1531284442065_3637_m_00 (and more) from job
> job_1531284442065_3637
> Task with the most failures(4):
>
> Task ID:
> task_1531284442065_3637_m_00
> URL:
>
> [http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1531284442065_3637=task_1531284442065_3637_m_00]
>
> Diagnostic Messages for this Task:
> File does not exist:
> hdfs://test/tmp/hive-hadoop/hadoop/fe5efa94-abb1-420f-b6ba-ec782e7b79ad/hive_2018-08-03_17-00-17_707_592882314975289971-5/-mr-10045/757eb1f7-7a37-4a7e-abc0-4a3b8b06510c/reduce.xml
> java.io.FileNotFoundException: File does not exist:
> hdfs://test/tmp/hive-hadoop/hadoop/fe5efa94-abb1-420f-b6ba-ec782e7b79ad/hive_2018-08-03_17-00-17_707_592882314975289971-5/-mr-10045/757eb1f7-7a37-4a7e-abc0-4a3b8b06510c/reduce.xml
> Looking into the plan by executing explain, I found that the Stage-4 and
> Stage-5 can reached from multi root tasks.
> {code:java}
> Explain
> STAGE DEPENDENCIES:
> Stage-21 is a root stage , consists of Stage-34, Stage-5
> Stage-34 has a backup stage: Stage-5
> Stage-20 depends on stages: Stage-34
> Stage-17 depends on stages: Stage-5, Stage-18, Stage-20 , consists of
> Stage-32, Stage-33, Stage-1
> Stage-32 has a backup stage: Stage-1
> Stage-15 depends on stages: Stage-32
> Stage-10 depends on stages: Stage-1, Stage-15, Stage-16 , consists of
> Stage-31, Stage-2
[jira] [Commented] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[
https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689546#comment-16689546
]
Hive QA commented on HIVE-20931:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} druid-handler in master has 4 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} druid-handler: The patch generated 0 new + 16
unchanged - 2 fixed = 16 total (was 18) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14963/dev-support/hive-personality.sh
|
| git revision | master / c737dbd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: druid-handler U: druid-handler |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14963/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Minor code cleaning for Druid Storage Handler
> -
>
> Key: HIVE-20931
> URL: https://issues.apache.org/jira/browse/HIVE-20931
> Project: Hive
> Issue Type: Improvement
> Components: Druid integration
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-20931.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20932) Vectorize Druid Storage Handler Reader
[ https://issues.apache.org/jira/browse/HIVE-20932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra reassigned HIVE-20932: - > Vectorize Druid Storage Handler Reader > -- > > Key: HIVE-20932 > URL: https://issues.apache.org/jira/browse/HIVE-20932 > Project: Hive > Issue Type: Improvement >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > > This patch aims at adding support for vectorize read of data from Druid to > Hive. > [~t3rmin4t0r] suggested that this will improve the performance of the top > level operators that supports vectorization. > As a first cut am just adding a wrapper around the existing Record Reader to > read up to 1024 row at a time. > Future work will be to avoid going via old reader and convert straight the > Json (smile format) to Vector primitive types. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20931: -- Attachment: HIVE-20931.patch > Minor code cleaning for Druid Storage Handler > - > > Key: HIVE-20931 > URL: https://issues.apache.org/jira/browse/HIVE-20931 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20931.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra reassigned HIVE-20931: - > Minor code cleaning for Druid Storage Handler > - > > Key: HIVE-20931 > URL: https://issues.apache.org/jira/browse/HIVE-20931 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20931.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20931) Minor code cleaning for Druid Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-20931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-20931: -- Status: Patch Available (was: Open) > Minor code cleaning for Druid Storage Handler > - > > Key: HIVE-20931 > URL: https://issues.apache.org/jira/browse/HIVE-20931 > Project: Hive > Issue Type: Improvement > Components: Druid integration >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-20931.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689496#comment-16689496
]
Miklos Gergely commented on HIVE-20919:
---
[~ashutoshc]
{color:#FF}ERROR:{color} -1 due to no test(s) being added or modified.
this is intentional, this patch is a refactoring patch and the refactored code
part is covered by unit and integration tests. So by definition no tests were
modified / added, but they all still running fine.
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch,
> HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch,
> HIVE-20919.06.patch, HIVE-20919.07.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689485#comment-16689485
]
Hive QA commented on HIVE-20919:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948484/HIVE-20919.07.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15543 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14962/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14962/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14962/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948484 - PreCommit-HIVE-Build
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch,
> HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch,
> HIVE-20919.06.patch, HIVE-20919.07.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689427#comment-16689427
]
Hive QA commented on HIVE-20919:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 6 new + 77 unchanged - 152
fixed = 83 total (was 229) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} ql generated 0 new + 2315 unchanged - 1 fixed = 2315
total (was 2316) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14962/dev-support/hive-personality.sh
|
| git revision | master / c737dbd |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14962/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14962/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch,
> HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch,
> HIVE-20919.06.patch, HIVE-20919.07.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20930) VectorCoalesce in FILTER mode doesn't take effect
[
https://issues.apache.org/jira/browse/HIVE-20930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689399#comment-16689399
]
Hive QA commented on HIVE-20930:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948474/HIVE-20930.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15528 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=154)
[intersect_all.q,unionDistinct_1.q,table_nonprintable.q,orc_llap_counters1.q,mm_cttas.q,whroot_external1.q,global_limit.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,results_cache_diff_fs.q,cttl.q,parallel_colstats.q,load_hdfs_file_with_space_in_the_name.q]
org.apache.hadoop.hive.metastore.TestPartitionManagement.testPartitionDiscoveryTransactionalTable
(batchId=222)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14961/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14961/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14961/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948474 - PreCommit-HIVE-Build
> VectorCoalesce in FILTER mode doesn't take effect
> -
>
> Key: HIVE-20930
> URL: https://issues.apache.org/jira/browse/HIVE-20930
> Project: Hive
> Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-20930.1.patch
>
>
> HIVE-20277 fixed vectorized case expressions for FILTER, but VectorCoalesce
> is still not fixed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor reassigned HIVE-20360:
---
Assignee: Laszlo Bodor
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch, HIVE-20360.07.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohan reassigned HIVE-20360:
Assignee: (was: Rohan)
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch, HIVE-20360.07.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rohan reassigned HIVE-20360:
Assignee: Rohan (was: Laszlo Bodor)
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Rohan
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch, HIVE-20360.07.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689349#comment-16689349
]
Laszlo Bodor commented on HIVE-20360:
-
:) thanks [~kgyrtkirk] !
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch, HIVE-20360.07.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20930) VectorCoalesce in FILTER mode doesn't take effect
[
https://issues.apache.org/jira/browse/HIVE-20930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689346#comment-16689346
]
Hive QA commented on HIVE-20930:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14961/dev-support/hive-personality.sh
|
| git revision | master / dd5121c7 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14961/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> VectorCoalesce in FILTER mode doesn't take effect
> -
>
> Key: HIVE-20930
> URL: https://issues.apache.org/jira/browse/HIVE-20930
> Project: Hive
> Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Attachments: HIVE-20930.1.patch
>
>
> HIVE-20277 fixed vectorized case expressions for FILTER, but VectorCoalesce
> is still not fixed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-20360:
Resolution: Fixed
Status: Resolved (was: Patch Available)
oh...a green run!
pushed to master. Thank you [~abstractdog] for taking care of this!
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch, HIVE-20360.07.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689318#comment-16689318
]
Hive QA commented on HIVE-20360:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948470/HIVE-20360.07.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15543 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14960/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14960/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14960/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948470 - PreCommit-HIVE-Build
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch, HIVE-20360.07.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-14557) Nullpointer When both SkewJoin and Mapjoin Enabled
[
https://issues.apache.org/jira/browse/HIVE-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-14557:
--
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks [~ychena] for the review.
> Nullpointer When both SkewJoin and Mapjoin Enabled
> ---
>
> Key: HIVE-14557
> URL: https://issues.apache.org/jira/browse/HIVE-14557
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 1.1.0, 2.1.0
>Reporter: Nemon Lou
>Assignee: Rui Li
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-14557.2.patch, HIVE-14557.3.patch,
> HIVE-14557.3.patch, HIVE-14557.patch
>
>
> The following sql failed with return code 2 on mr.
> {noformat}
> create table a(id int,id1 int);
> create table b(id int,id1 int);
> create table c(id int,id1 int);
> set hive.optimize.skewjoin=true;
> select a.id,b.id,c.id1 from a,b,c where a.id=b.id and a.id1=c.id1;
> {noformat}
> Error log as follows:
> {noformat}
> 2016-08-17 21:13:42,081 INFO [main]
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper:
> Id =0
>
> Id =21
>
> Id =28
>
> Id =16
>
> <\Children>
> Id = 28 null<\Parent>
> <\FS>
> <\Children>
> Id = 21 nullId = 33
> Id =33
> null
> <\Children>
> <\Parent>
> <\HASHTABLEDUMMY><\Parent>
> <\MAPJOIN>
> <\Children>
> Id = 0 null<\Parent>
> <\TS>
> <\Children>
> <\Parent>
> <\MAP>
> 2016-08-17 21:13:42,084 INFO [main]
> org.apache.hadoop.hive.ql.exec.TableScanOperator: Initializing operator TS[21]
> 2016-08-17 21:13:42,084 INFO [main]
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper: Initializing dummy operator
> 2016-08-17 21:13:42,086 INFO [main]
> org.apache.hadoop.hive.ql.exec.MapOperator: DESERIALIZE_ERRORS:0,
> RECORDS_IN:0,
> 2016-08-17 21:13:42,087 ERROR [main]
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper: Hit error while closing
> operators - failing tree
> 2016-08-17 21:13:42,088 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.closeOp(MapJoinOperator.java:474)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:682)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:696)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:696)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20360) QTest: ignore driver/qtest exclusions if -Dqfile param is set
[
https://issues.apache.org/jira/browse/HIVE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689287#comment-16689287
]
Hive QA commented on HIVE-20360:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/util in master has 48 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} itests/util: The patch generated 2 new + 68 unchanged
- 5 fixed = 70 total (was 73) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 18s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14960/dev-support/hive-personality.sh
|
| git revision | master / 156ed46 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14960/yetus/diff-checkstyle-itests_util.txt
|
| modules | C: itests/util U: itests/util |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14960/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> QTest: ignore driver/qtest exclusions if -Dqfile param is set
> -
>
> Key: HIVE-20360
> URL: https://issues.apache.org/jira/browse/HIVE-20360
> Project: Hive
> Issue Type: Improvement
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-20360.01.patch, HIVE-20360.02.patch,
> HIVE-20360.03.patch, HIVE-20360.04.patch, HIVE-20360.05.patch,
> HIVE-20360.06.patch, HIVE-20360.07.patch
>
>
> Sometimes I need to run qtests with another driver for testing purposes. In
> this case I have to edit testconfiguration.properties which seems a bit
> hacky, even if it's temporary.
> In this case, no tests will run (however there's a log message):
> {code:java}
> mvn test -Pitests -pl itests/qtest -pl itests/util -Dtest=TestCliDriver
> -Dqfile=bucketizedhiveinputformat.q
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19945) Beeline - run against a different sql engine
[
https://issues.apache.org/jira/browse/HIVE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689286#comment-16689286
]
Zoltan Haindrich commented on HIVE-19945:
-
Interesting - so far I've never noticed that it started as a fork; however I
only worked on it a occasionly...
I'm not sure how much work it would be to move toward using sqlline as a
library in beeline - but it would make sense to me...I think it's a good idea :)
fyi: [~pvary]
> Beeline - run against a different sql engine
>
>
> Key: HIVE-19945
> URL: https://issues.apache.org/jira/browse/HIVE-19945
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
>
> Original idea by [~kgyrtkirk]
> "I think beeline also support to load different sql drivers (not sure about
> this...but I think I saw some pointers to this) Anyway...it would be great to
> be able to execute a test against a different sql engine; like psql."
> something like:
> {code}
> mvn test -Dtest=TestQ -Dqengine=ExternalPSQL
> -Djdbc.uri=psql://localhost:5432/somedb
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19945) Beeline - run against a different sql engine
[
https://issues.apache.org/jira/browse/HIVE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689257#comment-16689257
]
Sergey Nuyanzin edited comment on HIVE-19945 at 11/16/18 10:34 AM:
---
[~abstractdog], [~kgyrtkirk] I found this ticket while searching for something
similar here.
As you probably know beeline is a fork of sqlline and there is another fork
https://github.com/julianhyde/sqlline .
Recently there were added a number of improvements like syntax highlighting,
emacs/vi mode, proper multiline handling, upgrade to jline3 and lots of others.
And of course it could be run against anything that supports jdbc
Now I also work with Hive and I started to make sqlline being able to connect
to Hive as well
What do you think about moving beeline being on top of sqlline with all these
features + beeline specific customizations?
For example in 1.5.0 sqlline version there was introduced allowance of
customization https://github.com/julianhyde/sqlline/pull/118 and pluggable
commands https://github.com/julianhyde/sqlline/pull/119
this allowed Apache Drill to get off their own fork of sqlline DRILL-3853.
was (Author: sergey nuyanzin):
[~abstractdog], [~kgyrtkirk] I found this ticket while searching for something
similar here.
As you probably know beeline is a fork of sqlline and there is another fork
https://github.com/julianhyde/sqlline .
Recently there were added a number of improvements like syntax highlighting,
emacs/vi mode, proper multiline handling, upgrade to jline3 and lots of others.
And of course it could be run against anything that supports jdbc
Now I also work with Hive and I started to make sqlline being able to connect
to Hive as well
What do you think about moving beeline being on top of sqlline with all these
features + beeline specific customizations?
For example in 1.5.0 sqlline version the was introduced allowance of
customization https://github.com/julianhyde/sqlline/pull/118 and pluggable
commands https://github.com/julianhyde/sqlline/pull/119
this allowed Apache Drill to get off their own fork of sqlline DRILL-3853.
> Beeline - run against a different sql engine
>
>
> Key: HIVE-19945
> URL: https://issues.apache.org/jira/browse/HIVE-19945
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
>
> Original idea by [~kgyrtkirk]
> "I think beeline also support to load different sql drivers (not sure about
> this...but I think I saw some pointers to this) Anyway...it would be great to
> be able to execute a test against a different sql engine; like psql."
> something like:
> {code}
> mvn test -Dtest=TestQ -Dqengine=ExternalPSQL
> -Djdbc.uri=psql://localhost:5432/somedb
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19945) Beeline - run against a different sql engine
[
https://issues.apache.org/jira/browse/HIVE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689257#comment-16689257
]
Sergey Nuyanzin edited comment on HIVE-19945 at 11/16/18 10:33 AM:
---
[~abstractdog], [~kgyrtkirk] I found this ticket while searching for something
similar here.
As you probably know beeline is a fork of sqlline and there is another fork
https://github.com/julianhyde/sqlline .
Recently there were added a number of improvements like syntax highlighting,
emacs/vi mode, proper multiline handling, upgrade to jline3 and lots of others.
And of course it could be run against anything that supports jdbc
Now I also work with Hive and I started to make sqlline being able to connect
to Hive as well
What do you think about moving beeline being on top of sqlline with all these
features + beeline specific customizations?
For example in 1.5.0 sqlline version the was introduced allowance of
customization https://github.com/julianhyde/sqlline/pull/118 and pluggable
commands https://github.com/julianhyde/sqlline/pull/119
this allowed Apache Drill to get off their own fork of sqlline DRILL-3853.
was (Author: sergey nuyanzin):
[~abstractdog], [~kgyrtkirk] I found this ticket while searching for something
similar here.
As you probably know beeline is a fork of sqlline and there is another fork
https://github.com/julianhyde/sqlline .
Recently there were added a number of improvements like syntax highlighting,
emacs/vi mode, proper multiline handling, upgrade to jline3 and lots of others.
And of course it could be run against anything that supports jdbc
Now I also work with Hive and I started to make sqlline being able to connect
to Hive as well
What do you think about moving beeline being on top of sqlline with all these
features + beeline specific customizations?
> Beeline - run against a different sql engine
>
>
> Key: HIVE-19945
> URL: https://issues.apache.org/jira/browse/HIVE-19945
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
>
> Original idea by [~kgyrtkirk]
> "I think beeline also support to load different sql drivers (not sure about
> this...but I think I saw some pointers to this) Anyway...it would be great to
> be able to execute a test against a different sql engine; like psql."
> something like:
> {code}
> mvn test -Dtest=TestQ -Dqengine=ExternalPSQL
> -Djdbc.uri=psql://localhost:5432/somedb
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19945) Beeline - run against a different sql engine
[
https://issues.apache.org/jira/browse/HIVE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689257#comment-16689257
]
Sergey Nuyanzin edited comment on HIVE-19945 at 11/16/18 10:16 AM:
---
[~abstractdog], [~kgyrtkirk] I found this ticket while searching for something
similar here.
As you probably know beeline is a fork of sqlline and there is another fork
https://github.com/julianhyde/sqlline .
Recently there were added a number of improvements like syntax highlighting,
emacs/vi mode, proper multiline handling, upgrade to jline3 and lots of others.
And of course it could be run against anything that supports jdbc
Now I also work with Hive and I started to make sqlline being able to connect
to Hive as well
What do you think about moving beeline being on top of sqlline with all these
features + beeline specific customizations?
was (Author: sergey nuyanzin):
[~abstractdog], [~kgyrtkirk] I found this ticket while searching for something
similar here.
As you probably know beeline is a fork of sqlline and there is another fork
https://github.com/julianhyde/sqlline .
Recently there were added a number of improvements like syntax highlighting,
emacs/vi mode, proper multiline handling, upgrade to jline3 and lots of others.
Now I also work with Hive and I started to make sqlline being able to connect
to Hive as well
What do you think about moving beeline being on top of sqlline with all these
features + beeline specific customizations?
> Beeline - run against a different sql engine
>
>
> Key: HIVE-19945
> URL: https://issues.apache.org/jira/browse/HIVE-19945
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
>
> Original idea by [~kgyrtkirk]
> "I think beeline also support to load different sql drivers (not sure about
> this...but I think I saw some pointers to this) Anyway...it would be great to
> be able to execute a test against a different sql engine; like psql."
> something like:
> {code}
> mvn test -Dtest=TestQ -Dqengine=ExternalPSQL
> -Djdbc.uri=psql://localhost:5432/somedb
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19945) Beeline - run against a different sql engine
[
https://issues.apache.org/jira/browse/HIVE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689257#comment-16689257
]
Sergey Nuyanzin edited comment on HIVE-19945 at 11/16/18 10:15 AM:
---
[~abstractdog], [~kgyrtkirk] I found this ticket while searching for something
similar here.
As you probably know beeline is a fork of sqlline and there is another fork
https://github.com/julianhyde/sqlline .
Recently there were added a number of improvements like syntax highlighting,
emacs/vi mode, proper multiline handling, upgrade to jline3 and lots of others.
Now I also work with Hive and I started to make sqlline being able to connect
to Hive as well
What do you think about moving beeline being on top of sqlline with all these
features + beeline specific customizations?
was (Author: sergey nuyanzin):
[~abstractdog], [~kgyrtkirk] I found this ticket while searching for something
similar here.
As you probably know beeline is a fork of sqlline and there is another fork
https://github.com/julianhyde/sqlline .
Recently there were added a number of improvement like syntax highlighting,
emacs/vi mode, proper multiline handling, upgrade to jline3 and lots of others.
Now I also work with Hive and I started to make sqlline being able to connect
to Hive as well
What do you think about moving beeline being on top of sqlline with all these
feature + beeline specific customizations?
> Beeline - run against a different sql engine
>
>
> Key: HIVE-19945
> URL: https://issues.apache.org/jira/browse/HIVE-19945
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
>
> Original idea by [~kgyrtkirk]
> "I think beeline also support to load different sql drivers (not sure about
> this...but I think I saw some pointers to this) Anyway...it would be great to
> be able to execute a test against a different sql engine; like psql."
> something like:
> {code}
> mvn test -Dtest=TestQ -Dqengine=ExternalPSQL
> -Djdbc.uri=psql://localhost:5432/somedb
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19945) Beeline - run against a different sql engine
[
https://issues.apache.org/jira/browse/HIVE-19945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689257#comment-16689257
]
Sergey Nuyanzin commented on HIVE-19945:
[~abstractdog], [~kgyrtkirk] I found this ticket while searching for something
similar here.
As you probably know beeline is a fork of sqlline and there is another fork
https://github.com/julianhyde/sqlline .
Recently there were added a number of improvement like syntax highlighting,
emacs/vi mode, proper multiline handling, upgrade to jline3 and lots of others.
Now I also work with Hive and I started to make sqlline being able to connect
to Hive as well
What do you think about moving beeline being on top of sqlline with all these
feature + beeline specific customizations?
> Beeline - run against a different sql engine
>
>
> Key: HIVE-19945
> URL: https://issues.apache.org/jira/browse/HIVE-19945
> Project: Hive
> Issue Type: Improvement
> Components: Beeline
>Affects Versions: 3.0.0
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
>
> Original idea by [~kgyrtkirk]
> "I think beeline also support to load different sql drivers (not sure about
> this...but I think I saw some pointers to this) Anyway...it would be great to
> be able to execute a test against a different sql engine; like psql."
> something like:
> {code}
> mvn test -Dtest=TestQ -Dqengine=ExternalPSQL
> -Djdbc.uri=psql://localhost:5432/somedb
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20929) The method HiveMetaStoreClient.getValidWriteIds(String fullTableName) failed to execute.
[
https://issues.apache.org/jira/browse/HIVE-20929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689247#comment-16689247
]
Igor Kryvenko commented on HIVE-20929:
--
Seems like your client and metastore are different by version.
{{validTxnList}} now is an optional parameter, after
[HIVE-20293|https://jira.apache.org/jira/browse/HIVE-20293]
> The method HiveMetaStoreClient.getValidWriteIds(String fullTableName) failed
> to execute.
>
>
> Key: HIVE-20929
> URL: https://issues.apache.org/jira/browse/HIVE-20929
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Affects Versions: 3.1.0
> Environment: hadoop3.1.0
> hive3.1.0
>Reporter: weizheng
>Assignee: weizheng
>Priority: Major
>
> When I use the getValidWriteIds(String fullTableName) method of
> HiveMetaStoreClient to get writeIds, I get the errors:
> Exception in thread "main" org.apache.thrift.protocol.TProtocolException:
> Required field 'validTxnList' is unset!
> Struct:GetValidWriteIdsRequest(fullTableNames:[default.w2], validTxnList:null)
> at
> org.apache.hadoop.hive.metastore.api.GetValidWriteIdsRequest.validate(GetValidWriteIdsRequest.java:396)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_valid_write_ids_args.validate(ThriftHiveMetastore.java)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_valid_write_ids_args$get_valid_write_ids_argsStandardScheme.write(ThriftHiveMetastore.java)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_valid_write_ids_args$get_valid_write_ids_argsStandardScheme.write(ThriftHiveMetastore.java)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_valid_write_ids_args.write(ThriftHiveMetastore.java)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:71)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:62)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_get_valid_write_ids(ThriftHiveMetastore.java:5443)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_valid_write_ids(ThriftHiveMetastore.java:5435)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getValidWriteIds(HiveMetaStoreClient.java:2581)
> at com.inspur.mutation.test.TableOperate.getRowsMap(TableOperate.java:521)
> at com.inspur.procuder.GenerateData.main(GenerateData.java:65)
>
> So I can only use the following methods instead：
> ValidWriteIdList writeIds;
> tablesList.add(AcidUtils.getFullTableName(table.getDbName(),
> table.getTableName()));
> List writeIdList =
> metaStoreClient.getValidWriteIds(tablesList,txns.toString());
> writeIds = TxnUtils.createValidReaderWriteIdList(writeIdList.get(0));
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Patch Available (was: Open) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch, > HIVE-20919.06.patch, HIVE-20919.07.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Attachment: HIVE-20919.07.patch > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch, > HIVE-20919.06.patch, HIVE-20919.07.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[ https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-20919: -- Status: Open (was: Patch Available) > Break up UpdateDeleteSemanticAnalyzer > - > > Key: HIVE-20919 > URL: https://issues.apache.org/jira/browse/HIVE-20919 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.2.0 >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 3.2.0 > > Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch, > HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch, > HIVE-20919.06.patch, HIVE-20919.07.patch > > > UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge > queries by rewriting them to a different form. This is a clear violation of > [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and > therefore needs to be refactored. An abstract ancestor needs to take the > common part, and each of the specific tasks should be handled by a separate > class. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689201#comment-16689201
]
Hive QA commented on HIVE-20919:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12948460/HIVE-20919.06.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 15543 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
(batchId=182)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=232)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14959/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14959/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14959/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12948460 - PreCommit-HIVE-Build
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch,
> HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch,
> HIVE-20919.06.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20919) Break up UpdateDeleteSemanticAnalyzer
[
https://issues.apache.org/jira/browse/HIVE-20919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16689168#comment-16689168
]
Hive QA commented on HIVE-20919:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2316 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 6 new + 77 unchanged - 152
fixed = 83 total (was 229) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} ql generated 0 new + 2315 unchanged - 1 fixed = 2315
total (was 2316) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-14959/dev-support/hive-personality.sh
|
| git revision | master / 156ed46 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14959/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-14959/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up UpdateDeleteSemanticAnalyzer
> -
>
> Key: HIVE-20919
> URL: https://issues.apache.org/jira/browse/HIVE-20919
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.2.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-20919.01.patch, HIVE-20919.02.patch,
> HIVE-20919.03.patch, HIVE-20919.04.patch, HIVE-20919.05.patch,
> HIVE-20919.06.patch
>
>
> UpdateDeleteSemanticAnalyzer handles update, delete, acid export and merge
> queries by rewriting them to a different form. This is a clear violation of
> [SRP|https://en.wikipedia.org/wiki/Single_responsibility_principle], and
> therefore needs to be refactored. An abstract ancestor needs to take the
> common part, and each of the specific tasks should be handled by a separate
> class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20930) VectorCoalesce in FILTER mode doesn't take effect
[ https://issues.apache.org/jira/browse/HIVE-20930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-20930: -- Status: Patch Available (was: Open) > VectorCoalesce in FILTER mode doesn't take effect > - > > Key: HIVE-20930 > URL: https://issues.apache.org/jira/browse/HIVE-20930 > Project: Hive > Issue Type: Bug >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-20930.1.patch > > > HIVE-20277 fixed vectorized case expressions for FILTER, but VectorCoalesce > is still not fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20930) VectorCoalesce in FILTER mode doesn't take effect
[ https://issues.apache.org/jira/browse/HIVE-20930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi updated HIVE-20930: -- Attachment: HIVE-20930.1.patch > VectorCoalesce in FILTER mode doesn't take effect > - > > Key: HIVE-20930 > URL: https://issues.apache.org/jira/browse/HIVE-20930 > Project: Hive > Issue Type: Bug >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > Attachments: HIVE-20930.1.patch > > > HIVE-20277 fixed vectorized case expressions for FILTER, but VectorCoalesce > is still not fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-20930) VectorCoalesce in FILTER mode doesn't take effect
[ https://issues.apache.org/jira/browse/HIVE-20930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Teddy Choi reassigned HIVE-20930: - > VectorCoalesce in FILTER mode doesn't take effect > - > > Key: HIVE-20930 > URL: https://issues.apache.org/jira/browse/HIVE-20930 > Project: Hive > Issue Type: Bug >Reporter: Teddy Choi >Assignee: Teddy Choi >Priority: Major > > HIVE-20277 fixed vectorized case expressions for FILTER, but VectorCoalesce > is still not fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
