[jira] [Commented] (HIVE-19851) upgrade jQuery version
[
https://issues.apache.org/jira/browse/HIVE-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509211#comment-16509211
]
Hive QA commented on HIVE-19851:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11724/dev-support/hive-personality.sh
|
| git revision | master / 8902563 |
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11724/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> upgrade jQuery version
> --
>
> Key: HIVE-19851
> URL: https://issues.apache.org/jira/browse/HIVE-19851
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19851.1.patch, HIVE-19851.2.patch
>
>
> jQuery version seems to be very old. Update to latest stable version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19846) Removed Deprecated Calls From FileUtils-getJarFilesByPath
[
https://issues.apache.org/jira/browse/HIVE-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509209#comment-16509209
]
Hive QA commented on HIVE-19846:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927323/HIVE-19846.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14521 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.client.TestRuntimeStats.testCleanup[Remote]
(batchId=212)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testQueueingWithThreads
(batchId=277)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11723/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11723/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11723/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927323 - PreCommit-HIVE-Build
> Removed Deprecated Calls From FileUtils-getJarFilesByPath
> -
>
> Key: HIVE-19846
> URL: https://issues.apache.org/jira/browse/HIVE-19846
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-19846.1.patch, HIVE-19846.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19418) add background stats updater similar to compactor
[ https://issues.apache.org/jira/browse/HIVE-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509188#comment-16509188 ] Alisha Prabhu commented on HIVE-19418: -- Hi [~sershe], [~kgyrtkirk] , I was able to reproduce the failure for TestStatsUpdaterThread on our local environment(x86, ppc64le). The test case has passed after increasing the timeout. > add background stats updater similar to compactor > - > > Key: HIVE-19418 > URL: https://issues.apache.org/jira/browse/HIVE-19418 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19418.01.patch, HIVE-19418.02.patch, > HIVE-19418.03.patch, HIVE-19418.04.patch, HIVE-19418.05.patch, > HIVE-19418.06.patch, HIVE-19418.06.patch, HIVE-19418.07.patch, > HIVE-19418.07.patch, HIVE-19418.patch > > > There's a JIRA HIVE-19416 to add snapshot version to stats for MM/ACID tables > to make them usable in a transaction without breaking ACID (for metadata-only > optimization). However, stats for ACID tables can still become unusable if > e.g. two parallel inserts run - neither sees the data written by the other, > so after both finish, the snapshots on either set of stats won't match the > current snapshot and the stats will be unusable. > Additionally, for ACID and non-ACID tables alike, a lot of the stats, with > some exceptions like numRows, cannot be aggregated (i.e. you cannot combine > ndvs from two inserts), and for ACID even less can be aggregated (you cannot > derive min/max if some rows are deleted but you don't scan the rest of the > dataset). > Therefore we will add background logic to metastore (similar to, and > partially inside, the ACID compactor) to update stats. > It will have 3 modes of operation. > 1) Off. > 2) Update only the stats that exist but are out of date (generating stats can > be expensive, so if the user is only analyzing a subset of tables it should > be able to only update that subset). We can simply look at existing stats and > only analyze for the relevant partitions and columns. > 3) On: 2 + create stats for all tables and columns missing stats. > There will also be a table parameter to skip stats update. > In phase 1, the process will operate outside of compactor, and run analyze > command on the table. The analyze command will automatically save the stats > with ACID snapshot information if needed, based on HIVE-19416, so we don't > need to do any special state management and this will work for all table > types. However it's also more expensive. > In phase 2, we can explore adding stats collection during MM compaction that > uses a temp table. If we don't have open writers during major compaction (so > we overwrite all of the data), the temp table stats can simply be copied over > to the main table with correct snapshot information, saving us a table scan. > In phase 3, we can add custom stats collection logic to full ACID compactor > that is not query based, the same way as we'd do for (2). Alternatively we > can wait for ACID compactor to become query based and just reuse (2). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19859) Inspect lock components for DBHiveLock while verifying whether transaction list is valid
[ https://issues.apache.org/jira/browse/HIVE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509186#comment-16509186 ] Ashutosh Chauhan commented on HIVE-19859: - +1 > Inspect lock components for DBHiveLock while verifying whether transaction > list is valid > > > Key: HIVE-19859 > URL: https://issues.apache.org/jira/browse/HIVE-19859 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19859.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19846) Removed Deprecated Calls From FileUtils-getJarFilesByPath
[
https://issues.apache.org/jira/browse/HIVE-19846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509177#comment-16509177
]
Hive QA commented on HIVE-19846:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} common: The patch generated 0 new + 27 unchanged - 1
fixed = 27 total (was 28) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 8m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11723/dev-support/hive-personality.sh
|
| git revision | master / 8902563 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11723/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Removed Deprecated Calls From FileUtils-getJarFilesByPath
> -
>
> Key: HIVE-19846
> URL: https://issues.apache.org/jira/browse/HIVE-19846
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: BELUGA BEHR
>Priority: Minor
> Attachments: HIVE-19846.1.patch, HIVE-19846.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19739) Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.
[ https://issues.apache.org/jira/browse/HIVE-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19739: Status: Patch Available (was: Open) Attached 04.patch with fixes for Findbugs and Checkstyle issues. > Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded > data/metadata. > - > > Key: HIVE-19739 > URL: https://issues.apache.org/jira/browse/HIVE-19739 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 4.0.0 > > Attachments: HIVE-19739.01.patch, HIVE-19739.02.patch, > HIVE-19739.03.patch, HIVE-19739.04.patch > > > Currently. bootstrap REPL LOAD have added checkpoint identifiers in > DB/table/partition object properties once the data/metadata related to the > object is successfully loaded. > If the Db exist and is not empty, then currently we are throwing exception. > But need to support it for the retry scenario after a failure. > If there is a retry of bootstrap load using the same dump, then instead of > throwing error, we should check if any of the tables/partitions are > completely loaded using the checkpoint identifiers. If yes, then skip it or > else drop/create them again. > If the bootstrap load is performed using different dump, then it should throw > exception. > Allow bootstrap on empty Db only if ckpt property is not set. Also, if > bootstrap load is completed on the target Db, then shouldn't allow bootstrap > retry at all. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19739) Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.
[ https://issues.apache.org/jira/browse/HIVE-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19739: Attachment: HIVE-19739.04.patch > Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded > data/metadata. > - > > Key: HIVE-19739 > URL: https://issues.apache.org/jira/browse/HIVE-19739 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 4.0.0 > > Attachments: HIVE-19739.01.patch, HIVE-19739.02.patch, > HIVE-19739.03.patch, HIVE-19739.04.patch > > > Currently. bootstrap REPL LOAD have added checkpoint identifiers in > DB/table/partition object properties once the data/metadata related to the > object is successfully loaded. > If the Db exist and is not empty, then currently we are throwing exception. > But need to support it for the retry scenario after a failure. > If there is a retry of bootstrap load using the same dump, then instead of > throwing error, we should check if any of the tables/partitions are > completely loaded using the checkpoint identifiers. If yes, then skip it or > else drop/create them again. > If the bootstrap load is performed using different dump, then it should throw > exception. > Allow bootstrap on empty Db only if ckpt property is not set. Also, if > bootstrap load is completed on the target Db, then shouldn't allow bootstrap > retry at all. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19739) Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.
[ https://issues.apache.org/jira/browse/HIVE-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19739: Status: Open (was: Patch Available) > Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded > data/metadata. > - > > Key: HIVE-19739 > URL: https://issues.apache.org/jira/browse/HIVE-19739 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 4.0.0 > > Attachments: HIVE-19739.01.patch, HIVE-19739.02.patch, > HIVE-19739.03.patch > > > Currently. bootstrap REPL LOAD have added checkpoint identifiers in > DB/table/partition object properties once the data/metadata related to the > object is successfully loaded. > If the Db exist and is not empty, then currently we are throwing exception. > But need to support it for the retry scenario after a failure. > If there is a retry of bootstrap load using the same dump, then instead of > throwing error, we should check if any of the tables/partitions are > completely loaded using the checkpoint identifiers. If yes, then skip it or > else drop/create them again. > If the bootstrap load is performed using different dump, then it should throw > exception. > Allow bootstrap on empty Db only if ckpt property is not set. Also, if > bootstrap load is completed on the target Db, then shouldn't allow bootstrap > retry at all. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19237) Only use an operatorId once in a plan
[
https://issues.apache.org/jira/browse/HIVE-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509155#comment-16509155
]
Hive QA commented on HIVE-19237:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927300/HIVE-19237.01-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 14455 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=254)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dp_counter_mm]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dp_counter_non_mm]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_llap]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_input_counters]
(batchId=173)
org.apache.hadoop.hive.metastore.client.TestFunctions.testCreateFunctionNullDatabaseName[Embedded]
(batchId=210)
org.apache.hadoop.hive.metastore.client.TestFunctions.testCreateFunctionNullDatabaseName[Remote]
(batchId=210)
org.apache.hive.hcatalog.api.TestHCatClient.testBasicDDLCommands (batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testCreateTableLike (batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testDatabaseLocation (batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testDropPartitionsWithPartialSpec
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testDropTableException (batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testEmptyTableInstantiation
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testGetMessageBusTopicName
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testGetPartitionsWithPartialSpec
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testObjectNotFoundException
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testOtherFailure (batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionRegistrationWithCustomSchema
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSchema (batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionSpecRegistrationWithCustomSchema
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testPartitionsHCatClientImpl
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testRenameTable (batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testReplicationTaskIter
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testTableSchemaPropagation
(batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testTransportFailure (batchId=194)
org.apache.hive.hcatalog.api.TestHCatClient.testUpdateTableSchema (batchId=194)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=306)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11722/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11722/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11722/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 27 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927300 - PreCommit-HIVE-Build
> Only use an operatorId once in a plan
> -
>
> Key: HIVE-19237
> URL: https://issues.apache.org/jira/browse/HIVE-19237
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19237.01-branch-3.patch, HIVE-19237.01.patch,
> HIVE-19237.02.patch, HIVE-19237.03.patch, HIVE-19237.04.patch,
> HIVE-19237.05.patch, HIVE-19237.05.patch, HIVE-19237.06.patch,
> HIVE-19237.07.patch, HIVE-19237.08.patch, HIVE-19237.08.patch,
> HIVE-19237.09.patch, HIVE-19237.10.patch, HIVE-19237.10.patch,
> HIVE-19237.11.patch, HIVE-19237.11.patch, HIVE-19237.11.patch,
> HIVE-19237.12.patch
>
>
> Column stats autogather plan part is added from a plan compiled by the driver
> itself; however that driver starts to use operatorIds from 1 ; so it's
> possible that 2 SEL_1 operators end up in the same plan...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19815) Repl dump should not propagate the checkpoint and repl source properties
[ https://issues.apache.org/jira/browse/HIVE-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-19815: Resolution: Fixed Status: Resolved (was: Patch Available) > Repl dump should not propagate the checkpoint and repl source properties > > > Key: HIVE-19815 > URL: https://issues.apache.org/jira/browse/HIVE-19815 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, repl >Affects Versions: 3.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19815.01-branch-3.patch, HIVE-19815.01.patch, > HIVE-19815.02.patch > > > For replication scenarios of A-> B -> C the repl dump on B should not include > the checkpoint property when dumping out table information. > Alter tables/partitions during incremental should not propagate this as well. > Also should not propagate the the db level parameters set by replication > internally. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19815) Repl dump should not propagate the checkpoint and repl source properties
[ https://issues.apache.org/jira/browse/HIVE-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509143#comment-16509143 ] Sankar Hariappan commented on HIVE-19815: - Test failures are irrelevant to this patch. Patch committed to branch-3. > Repl dump should not propagate the checkpoint and repl source properties > > > Key: HIVE-19815 > URL: https://issues.apache.org/jira/browse/HIVE-19815 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, repl >Affects Versions: 3.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19815.01-branch-3.patch, HIVE-19815.01.patch, > HIVE-19815.02.patch > > > For replication scenarios of A-> B -> C the repl dump on B should not include > the checkpoint property when dumping out table information. > Alter tables/partitions during incremental should not propagate this as well. > Also should not propagate the the db level parameters set by replication > internally. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19604) Incorrect Handling of Boolean in DruidSerde
[
https://issues.apache.org/jira/browse/HIVE-19604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509142#comment-16509142
]
Ashutosh Chauhan commented on HIVE-19604:
-
{{(Number) value).doubleValue() != 0)}}
Since double is not precise this test is not reliable. Since we know its an int
value I think {{(Number) value).intValue() != 0)}} is better.
> Incorrect Handling of Boolean in DruidSerde
> ---
>
> Key: HIVE-19604
> URL: https://issues.apache.org/jira/browse/HIVE-19604
> Project: Hive
> Issue Type: Bug
> Components: Druid integration
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19604.01.patch, HIVE-19604.patch
>
>
> Results of boolean expressions from Druid are expressed in the form of
> numeric 1 or 0.
> When reading the results in DruidSerde both 1 and 0 are translated to String
> and then we call Boolean.valueOf(stringForm), this leads to the boolean being
> read always as false.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19418) add background stats updater similar to compactor
[ https://issues.apache.org/jira/browse/HIVE-19418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509120#comment-16509120 ] Vineet Garg commented on HIVE-19418: [~sershe] This patch in branch-3 is causing following failures (I have confirmed it using git bisect) * org.apache.hadoop.hive.metastore.client.TestFunctions.testCreateFunctionNullDatabaseName[Embedded] * org.apache.hadoop.hive.metastore.client.TestFunctions.testCreateFunctionNullDatabaseName[Remote] Can you please fix these tests or revert your patch from branch-3? I don't understand why this was pushed to branch-3 without a test run. Ref: https://builds.apache.org/job/PreCommit-HIVE-Build/11694/testReport/ > add background stats updater similar to compactor > - > > Key: HIVE-19418 > URL: https://issues.apache.org/jira/browse/HIVE-19418 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19418.01.patch, HIVE-19418.02.patch, > HIVE-19418.03.patch, HIVE-19418.04.patch, HIVE-19418.05.patch, > HIVE-19418.06.patch, HIVE-19418.06.patch, HIVE-19418.07.patch, > HIVE-19418.07.patch, HIVE-19418.patch > > > There's a JIRA HIVE-19416 to add snapshot version to stats for MM/ACID tables > to make them usable in a transaction without breaking ACID (for metadata-only > optimization). However, stats for ACID tables can still become unusable if > e.g. two parallel inserts run - neither sees the data written by the other, > so after both finish, the snapshots on either set of stats won't match the > current snapshot and the stats will be unusable. > Additionally, for ACID and non-ACID tables alike, a lot of the stats, with > some exceptions like numRows, cannot be aggregated (i.e. you cannot combine > ndvs from two inserts), and for ACID even less can be aggregated (you cannot > derive min/max if some rows are deleted but you don't scan the rest of the > dataset). > Therefore we will add background logic to metastore (similar to, and > partially inside, the ACID compactor) to update stats. > It will have 3 modes of operation. > 1) Off. > 2) Update only the stats that exist but are out of date (generating stats can > be expensive, so if the user is only analyzing a subset of tables it should > be able to only update that subset). We can simply look at existing stats and > only analyze for the relevant partitions and columns. > 3) On: 2 + create stats for all tables and columns missing stats. > There will also be a table parameter to skip stats update. > In phase 1, the process will operate outside of compactor, and run analyze > command on the table. The analyze command will automatically save the stats > with ACID snapshot information if needed, based on HIVE-19416, so we don't > need to do any special state management and this will work for all table > types. However it's also more expensive. > In phase 2, we can explore adding stats collection during MM compaction that > uses a temp table. If we don't have open writers during major compaction (so > we overwrite all of the data), the temp table stats can simply be copied over > to the main table with correct snapshot information, saving us a table scan. > In phase 3, we can add custom stats collection logic to full ACID compactor > that is not query based, the same way as we'd do for (2). Alternatively we > can wait for ACID compactor to become query based and just reuse (2). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19237) Only use an operatorId once in a plan
[
https://issues.apache.org/jira/browse/HIVE-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509118#comment-16509118
]
Hive QA commented on HIVE-19237:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-11722/patches/PreCommit-HIVE-Build-11722.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11722/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Only use an operatorId once in a plan
> -
>
> Key: HIVE-19237
> URL: https://issues.apache.org/jira/browse/HIVE-19237
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19237.01-branch-3.patch, HIVE-19237.01.patch,
> HIVE-19237.02.patch, HIVE-19237.03.patch, HIVE-19237.04.patch,
> HIVE-19237.05.patch, HIVE-19237.05.patch, HIVE-19237.06.patch,
> HIVE-19237.07.patch, HIVE-19237.08.patch, HIVE-19237.08.patch,
> HIVE-19237.09.patch, HIVE-19237.10.patch, HIVE-19237.10.patch,
> HIVE-19237.11.patch, HIVE-19237.11.patch, HIVE-19237.11.patch,
> HIVE-19237.12.patch
>
>
> Column stats autogather plan part is added from a plan compiled by the driver
> itself; however that driver starts to use operatorIds from 1 ; so it's
> possible that 2 SEL_1 operators end up in the same plan...
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[
https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509111#comment-16509111
]
Hive QA commented on HIVE-17852:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 63m
28s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
42s{color} | {color:red} branch/itests/hive-unit cannot run setBugDatabaseInfo
from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
24s{color} | {color:red} branch/metastore cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 5m
17s{color} | {color:red} branch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
4s{color} | {color:red} branch/standalone-metastore cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 12m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
6s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 43m
9s{color} | {color:red} root: The patch generated 1365 new + 247794 unchanged -
1458 fixed = 249159 total (was 249252) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
35s{color} | {color:red} itests/hive-unit: The patch generated 150 new + 11911
unchanged - 157 fixed = 12061 total (was 12068) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 22m
26s{color} | {color:red} ql: The patch generated 742 new + 127638 unchanged -
830 fixed = 128380 total (was 128468) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 3m
1s{color} | {color:red} standalone-metastore: The patch generated 473 new +
19439 unchanged - 471 fixed = 19912 total (was 19910) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
31s{color} | {color:red} patch/itests/hive-unit cannot run setBugDatabaseInfo
from findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
19s{color} | {color:red} patch/metastore cannot run setBugDatabaseInfo from
findbugs {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
27s{color} | {color:red} patch/ql cannot run setBugDatabaseInfo from findbugs
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
28s{color} | {color:red} patch/standalone-metastore cannot run
setBugDatabaseInfo from findbugs {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 5m
1s{color} | {color:red} root generated 2 new + 366 unchanged - 2 fixed = 368
total (was 368) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
46s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} |
[jira] [Commented] (HIVE-19739) Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.
[
https://issues.apache.org/jira/browse/HIVE-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509114#comment-16509114
]
Hive QA commented on HIVE-19739:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927297/HIVE-19739.03.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14524 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcDriver2.testSelectExecAsync2 (batchId=244)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11721/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11721/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11721/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927297 - PreCommit-HIVE-Build
> Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded
> data/metadata.
> -
>
> Key: HIVE-19739
> URL: https://issues.apache.org/jira/browse/HIVE-19739
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, repl
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: DR, pull-request-available, replication
> Fix For: 4.0.0
>
> Attachments: HIVE-19739.01.patch, HIVE-19739.02.patch,
> HIVE-19739.03.patch
>
>
> Currently. bootstrap REPL LOAD have added checkpoint identifiers in
> DB/table/partition object properties once the data/metadata related to the
> object is successfully loaded.
> If the Db exist and is not empty, then currently we are throwing exception.
> But need to support it for the retry scenario after a failure.
> If there is a retry of bootstrap load using the same dump, then instead of
> throwing error, we should check if any of the tables/partitions are
> completely loaded using the checkpoint identifiers. If yes, then skip it or
> else drop/create them again.
> If the bootstrap load is performed using different dump, then it should throw
> exception.
> Allow bootstrap on empty Db only if ckpt property is not set. Also, if
> bootstrap load is completed on the target Db, then shouldn't allow bootstrap
> retry at all.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19739) Bootstrap REPL LOAD to use checkpoints to validate and skip the loaded data/metadata.
[
https://issues.apache.org/jira/browse/HIVE-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509100#comment-16509100
]
Hive QA commented on HIVE-19739:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2279 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
11s{color} | {color:blue} standalone-metastore in master has 216 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
35s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} itests/hive-unit: The patch generated 65 new + 65
unchanged - 1 fixed = 130 total (was 66) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 25 new + 93 unchanged - 2
fixed = 118 total (was 95) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} standalone-metastore: The patch generated 14 new + 391
unchanged - 4 fixed = 405 total (was 395) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
18s{color} | {color:red} ql generated 1 new + 2278 unchanged - 1 fixed = 2279
total (was 2279) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 20s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Switch statement found in
org.apache.hadoop.hive.ql.exec.repl.bootstrap.load.LoadDatabase.tasks() where
default case is missing At LoadDatabase.java:where default case is missing At
LoadDatabase.java:[lines 65-70] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11721/dev-support/hive-personality.sh
|
| git revision | master / 8902563 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11721/yetus/patch-mvninstall-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11721/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11721/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11721/yetus/diff-checkstyle-standalone-metastore.txt
|
| findbugs |
[jira] [Updated] (HIVE-19532) Modify Hive Driver/Executor to support transactional-stats-using COUNT aggregation queries
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-19532: -- Status: Patch Available (was: Open) > Modify Hive Driver/Executor to support transactional-stats-using COUNT > aggregation queries > --- > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.prepatch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19532) Modify Hive Driver/Executor to support transactional-stats-using COUNT aggregation queries
[ https://issues.apache.org/jira/browse/HIVE-19532?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-19532: -- Attachment: HIVE-19532.01.patch > Modify Hive Driver/Executor to support transactional-stats-using COUNT > aggregation queries > --- > > Key: HIVE-19532 > URL: https://issues.apache.org/jira/browse/HIVE-19532 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19532.01.patch, HIVE-19532.01.prepatch, > HIVE-19532.02.prepatch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19866) improve LLAP cache purge
[ https://issues.apache.org/jira/browse/HIVE-19866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19866: Summary: improve LLAP cache purge (was: improve cache purge) > improve LLAP cache purge > > > Key: HIVE-19866 > URL: https://issues.apache.org/jira/browse/HIVE-19866 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19866.patch > > > 1) Memory needs to be accounted for. > 2) LRFU eviction doesn't need to maintain state between individual removals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19866) improve cache purge
[ https://issues.apache.org/jira/browse/HIVE-19866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19866: Attachment: HIVE-19866.patch > improve cache purge > --- > > Key: HIVE-19866 > URL: https://issues.apache.org/jira/browse/HIVE-19866 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19866.patch > > > 1) Memory needs to be accounted for. > 2) LRFU eviction doesn't need to maintain state between individual removals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19866) improve cache purge
[ https://issues.apache.org/jira/browse/HIVE-19866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19866: Status: Patch Available (was: Open) [~prasanth_j] [~t3rmin4t0r] can you take a look? > improve cache purge > --- > > Key: HIVE-19866 > URL: https://issues.apache.org/jira/browse/HIVE-19866 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19866.patch > > > 1) Memory needs to be accounted for. > 2) LRFU eviction doesn't need to maintain state between individual removals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19866) improve cache purge
[ https://issues.apache.org/jira/browse/HIVE-19866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19866: --- > improve cache purge > --- > > Key: HIVE-19866 > URL: https://issues.apache.org/jira/browse/HIVE-19866 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > 1) Memory needs to be accounted for. > 2) LRFU eviction doesn't need to maintain state between individual removals. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19865) Full ACID table stats has wrong rawDataSize
[ https://issues.apache.org/jira/browse/HIVE-19865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom reassigned HIVE-19865: - Assignee: Steve Yeom > Full ACID table stats has wrong rawDataSize > --- > > Key: HIVE-19865 > URL: https://issues.apache.org/jira/browse/HIVE-19865 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 4.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 4.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets
[
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509073#comment-16509073
]
Hive QA commented on HIVE-19653:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927286/HIVE-19653.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11720/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11720/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11720/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12927286/HIVE-19653.1.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927286 - PreCommit-HIVE-Build
> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 4.0.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19653.1.patch, HIVE-19653.patch
>
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((), (a), (b), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will
> output:
> {code:java}
> NULL NULL123456
> NULL 123456
> NULL123456
> 123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all
> grouping sets contains the processing expr before pushdown. otherwise the
> expr value after GBY is changed and the result is wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets
[
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509071#comment-16509071
]
Hive QA commented on HIVE-19653:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927286/HIVE-19653.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 14523 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_grouping_sets_pushdown1]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_grouping_sets4]
(batchId=161)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[groupby_grouping_sets_pushdown1]
(batchId=145)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testQueueingWithThreads
(batchId=277)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11719/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11719/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11719/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927286 - PreCommit-HIVE-Build
> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 4.0.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19653.1.patch, HIVE-19653.patch
>
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((), (a), (b), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will
> output:
> {code:java}
> NULL NULL123456
> NULL 123456
> NULL123456
> 123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all
> grouping sets contains the processing expr before pushdown. otherwise the
> expr value after GBY is changed and the result is wrong.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets
[
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509045#comment-16509045
]
Hive QA commented on HIVE-19653:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 5m
4s{color} | {color:blue} ql in master has 2279 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
17s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11719/dev-support/hive-personality.sh
|
| git revision | master / 8902563 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11719/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Affects Versions: 4.0.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19653.1.patch, HIVE-19653.patch
>
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((), (a), (b), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will
> output:
> {code:java}
> NULL NULL123456
> NULL 123456
> NULL123456
> 123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all
> grouping sets contains the processing expr before pushdown. otherwise the
> expr value after GBY is changed and the result is wrong.
--
This message was sent by
[jira] [Commented] (HIVE-17852) remove support for list bucketing "stored as directories" in 3.0
[
https://issues.apache.org/jira/browse/HIVE-17852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509010#comment-16509010
]
Hive QA commented on HIVE-17852:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927283/HIVE-17852.08.patch
{color:green}SUCCESS:{color} +1 due to 34 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 14518 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=167)
org.apache.hadoop.hive.metastore.tools.TestSchemaToolForMetastore.testValidateLocations
(batchId=214)
org.apache.hadoop.hive.metastore.tools.TestSchemaToolForMetastore.testValidateNullValues
(batchId=214)
org.apache.hive.beeline.TestSchemaTool.testValidateLocations (batchId=238)
org.apache.hive.beeline.TestSchemaTool.testValidateNullValues (batchId=238)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveDatabase (batchId=239)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveTable (batchId=239)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveTableToNonExistentDb
(batchId=239)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveTableWithExistingTableOfSameNameAlreadyInTargetDatabase
(batchId=239)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveTableWithinCatalog
(batchId=239)
org.apache.hive.service.server.TestInformationSchemaWithPrivilege.test
(batchId=241)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11718/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11718/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11718/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927283 - PreCommit-HIVE-Build
> remove support for list bucketing "stored as directories" in 3.0
>
>
> Key: HIVE-17852
> URL: https://issues.apache.org/jira/browse/HIVE-17852
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-17852.01.patch, HIVE-17852.02.patch,
> HIVE-17852.03.patch, HIVE-17852.04.patch, HIVE-17852.05.patch,
> HIVE-17852.06.patch, HIVE-17852.07.patch, HIVE-17852.08.patch
>
>
> From the email thread:
> 1) LB, when stored as directories, adds a lot of low-level complexity to Hive
> tables that has to be accounted for in many places in the code where the
> files are written or modified - from FSOP to ACID/replication/export.
> 2) While working on some FSOP code I noticed that some of that logic is
> broken - e.g. the duplicate file removal from tasks, a pretty fundamental
> correctness feature in Hive, may be broken. LB also doesn’t appear to be
> compatible with e.g. regular bucketing.
> 3) The feature hasn’t seen development activity in a while; it also doesn’t
> appear to be used a lot.
> Keeping with the theme of cleaning up “legacy” code for 3.0, I was proposing
> we remove it.
> (2) also suggested that, if needed, it might be easier to implement similar
> functionality by adding some flexibility to partitions (which LB directories
> look like anyway); that would also keep the logic on a higher level of
> abstraction (split generation, partition pruning) as opposed to many
> low-level places like FSOP, etc.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19861) Fix temp table path generation for acid table export
[ https://issues.apache.org/jira/browse/HIVE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509007#comment-16509007 ] Eugene Koifman commented on HIVE-19861: --- +1 > Fix temp table path generation for acid table export > > > Key: HIVE-19861 > URL: https://issues.apache.org/jira/browse/HIVE-19861 > Project: Hive > Issue Type: Bug > Components: Import/Export, Transactions >Affects Versions: 3.0.0 >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19861.1.patch > > > Temp tables that are analyzed by the SemanticAnalyzer get their default > location set to a location in the session directory. Export of Acid tables > also creates temp tables, but this is done via a plan transformation, and the > temp table creation never goes through the SemanticAnalyzer, meaning the > location is not set. There is some other logic in DDLTask (which I am > changing in HIV-19837) which ends up automatically setting this path to the > default table location in the warehouse directory. This should be fixed so > that the path defaults to a location in the session directory, like with > normal temp tables. > cc [~ekoifman] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19838) simplify & fix ColumnizedDeleteEventRegistry load loop
[ https://issues.apache.org/jira/browse/HIVE-19838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16509003#comment-16509003 ] Eugene Koifman commented on HIVE-19838: --- +1 I left a couple of nits on RB Ignore my previous comment about distinctOwids. It's a poorly named variable - it's really counting the number of distinct (writeid, bucketproperty) pairs and the search on CompressedOwid matches this. Note to self: For unbucketed tables, if multiple bucket files are all loaded, each files has it's own reader in the heap, which means regardless of how delete events are spread among files, the heap sorts all of them by (writeid, bucketprop. rowid) so ColumnizedDeleteEventRegistry.isDeleted() looks ok. > simplify & fix ColumnizedDeleteEventRegistry load loop > -- > > Key: HIVE-19838 > URL: https://issues.apache.org/jira/browse/HIVE-19838 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19838.01.patch, HIVE-19838.patch > > > Apparently sometimes the delete count in ACID stats doesn't match what merger > actually returns. > It could be due to some deltas having duplicate deletes from parallel queries > (I guess?) that are being squashed by the merger or some other reasons beyond > my mortal comprehension. > The loop assumes the merger will return the exact number of records, so it > fails with array index exception. Also, it could actually be done in a single > loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19378) "hive.lock.numretries" Is Misleading
[
https://issues.apache.org/jira/browse/HIVE-19378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508985#comment-16508985
]
Alice Fan commented on HIVE-19378:
--
Hi [~belugabehr],
I was proposing to change the local variable name from 'numRetriesForLock' to
'numTotalTriesForLock' and same for unLock method, but this will be just code
level of change.
I think many of 'retries' in hive actually stand for number of total tries, for
example MetastoreConf's THRIFT_CONNECTION_RETRIES. I am not sure if it is
adding benefit with changing the name of 'hive.lock.numretries' to
'hive.lock.numTotalTries' in configuration, considering that will require many
changes in documentation. I am open to have more discussion here :)
> "hive.lock.numretries" Is Misleading
>
>
> Key: HIVE-19378
> URL: https://issues.apache.org/jira/browse/HIVE-19378
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Alice Fan
>Priority: Minor
> Attachments: HIVE-19378.1.patch
>
>
> Configuration 'hive.lock.numretries' is confusing. It's not actually a
> 'retry' count, it's the total number of attempt to try:
>
> {code:java|title=ZooKeeperHiveLockManager.java}
> do {
> lastException = null;
> tryNum++;
> try {
> if (tryNum > 1) {
> Thread.sleep(sleepTime);
> prepareRetry();
> }
> ret = lockPrimitive(key, mode, keepAlive, parentCreated,
> conflictingLocks);
> ...
> } while (tryNum < numRetriesForLock);
> {code}
> So, from this code you can see that on the first loop, {{tryNum}} is set to
> 1, in which case, if the configuration num*retries* is set to 1, there will
> be one attempt total. With a *retry* value of 1, I would assume one initial
> attempt and one additional retry. Please change to:
> {code}
> while (tryNum <= numRetriesForLock);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19203) Thread-Safety Issue in HiveMetaStore
[
https://issues.apache.org/jira/browse/HIVE-19203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508978#comment-16508978
]
Aihua Xu commented on HIVE-19203:
-
+1.
> Thread-Safety Issue in HiveMetaStore
>
>
> Key: HIVE-19203
> URL: https://issues.apache.org/jira/browse/HIVE-19203
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: Alice Fan
>Priority: Minor
> Attachments: HIVE-19203.1.patch
>
>
> [https://github.com/apache/hive/blob/550d1e1196b7c801c572092db974a459aac6c249/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L345-L351]
> {code:java}
> private static int nextSerialNum = 0;
> private static ThreadLocal threadLocalId = new
> ThreadLocal() {
> @Override
> protected Integer initialValue() {
> return nextSerialNum++;
> }
> };{code}
>
> {{nextSerialNum}} needs to be an atomic value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19864) Address TestTriggersWorkloadManager flakiness
[ https://issues.apache.org/jira/browse/HIVE-19864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-19864: > Address TestTriggersWorkloadManager flakiness > - > > Key: HIVE-19864 > URL: https://issues.apache.org/jira/browse/HIVE-19864 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > TestTriggersWorkloadManager seems flaky and all test cases gets timed out at > times. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19228) Remove commons-httpclient 3.x usage
[ https://issues.apache.org/jira/browse/HIVE-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508966#comment-16508966 ] Prasanth Jayachandran commented on HIVE-19228: -- cherrypicked to branch-3 as well. > Remove commons-httpclient 3.x usage > --- > > Key: HIVE-19228 > URL: https://issues.apache.org/jira/browse/HIVE-19228 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19228.1.patch, HIVE-19228.2.patch, > HIVE-19228.3.patch, HIVE-19228.4.patch, HIVE-19228.4.patch, > HIVE-19228.4.patch, HIVE-19228.5.patch, HIVE-19228.6.patch > > > Commons-httpclient is not supported well anymore. Remove dependency and move > to Apache HTTP client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19228) Remove commons-httpclient 3.x usage
[ https://issues.apache.org/jira/browse/HIVE-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19228: - Fix Version/s: 3.1.0 > Remove commons-httpclient 3.x usage > --- > > Key: HIVE-19228 > URL: https://issues.apache.org/jira/browse/HIVE-19228 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19228.1.patch, HIVE-19228.2.patch, > HIVE-19228.3.patch, HIVE-19228.4.patch, HIVE-19228.4.patch, > HIVE-19228.4.patch, HIVE-19228.5.patch, HIVE-19228.6.patch > > > Commons-httpclient is not supported well anymore. Remove dependency and move > to Apache HTTP client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19815) Repl dump should not propagate the checkpoint and repl source properties
[
https://issues.apache.org/jira/browse/HIVE-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508963#comment-16508963
]
Hive QA commented on HIVE-19815:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927279/HIVE-19815.01-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 14457 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=254)
org.apache.hadoop.hive.metastore.client.TestFunctions.testCreateFunctionNullDatabaseName[Embedded]
(batchId=210)
org.apache.hadoop.hive.metastore.client.TestFunctions.testCreateFunctionNullDatabaseName[Remote]
(batchId=210)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testQueueingWithThreads
(batchId=275)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=306)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11717/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11717/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11717/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927279 - PreCommit-HIVE-Build
> Repl dump should not propagate the checkpoint and repl source properties
>
>
> Key: HIVE-19815
> URL: https://issues.apache.org/jira/browse/HIVE-19815
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, repl
>Affects Versions: 3.1.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19815.01-branch-3.patch, HIVE-19815.01.patch,
> HIVE-19815.02.patch
>
>
> For replication scenarios of A-> B -> C the repl dump on B should not include
> the checkpoint property when dumping out table information.
> Alter tables/partitions during incremental should not propagate this as well.
> Also should not propagate the the db level parameters set by replication
> internally.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19838) simplify & fix ColumnizedDeleteEventRegistry load loop
[ https://issues.apache.org/jira/browse/HIVE-19838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508926#comment-16508926 ] Sergey Shelukhin commented on HIVE-19838: - Hmm... if you look at the below code in the current master, it does count each bp change as being distinct OWID. So this patch keeps that behavior, that doesn't change. > simplify & fix ColumnizedDeleteEventRegistry load loop > -- > > Key: HIVE-19838 > URL: https://issues.apache.org/jira/browse/HIVE-19838 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19838.01.patch, HIVE-19838.patch > > > Apparently sometimes the delete count in ACID stats doesn't match what merger > actually returns. > It could be due to some deltas having duplicate deletes from parallel queries > (I guess?) that are being squashed by the merger or some other reasons beyond > my mortal comprehension. > The loop assumes the merger will return the exact number of records, so it > fails with array index exception. Also, it could actually be done in a single > loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19838) simplify & fix ColumnizedDeleteEventRegistry load loop
[
https://issues.apache.org/jira/browse/HIVE-19838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508925#comment-16508925
]
Eugene Koifman commented on HIVE-19838:
---
I think one of the ways {{totalDeleteEventCount}} in
{{ColumnizedDeleteEventRegistry}} may be off, is that {{DeleteReaderValue}}
takes a ValidWriteIdList which means that {{next()}} may skip some event
because it belongs to a transaction that was not yet committed when the current
reader locked in the snapshot.
In practice, this would require compaction (at least a minor one) which
includes a txn that is open to the reader's txn, to complete before the
VectorizedOrc reader starts reading - which is possible but not very likely.
Another issue, which I think is eliminated by the current patch is,
{noformat}
if (lastSeenOwid != deleteRecordKey.originalWriteId ||
lastSeenBucketProperty != deleteRecordKey.bucketProperty) {
++distinctOwids;
lastSeenOwid = deleteRecordKey.originalWriteId;
lastSeenBucketProperty = deleteRecordKey.bucketProperty;
}
{noformat}
{{distinctOwids}} is incremented when bucketProperty changes, which seems
invalid even for bucketed tables.
> simplify & fix ColumnizedDeleteEventRegistry load loop
> --
>
> Key: HIVE-19838
> URL: https://issues.apache.org/jira/browse/HIVE-19838
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19838.01.patch, HIVE-19838.patch
>
>
> Apparently sometimes the delete count in ACID stats doesn't match what merger
> actually returns.
> It could be due to some deltas having duplicate deletes from parallel queries
> (I guess?) that are being squashed by the merger or some other reasons beyond
> my mortal comprehension.
> The loop assumes the merger will return the exact number of records, so it
> fails with array index exception. Also, it could actually be done in a single
> loop.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19862) Postgres init script has a glitch around UNIQUE_DATABASE
[
https://issues.apache.org/jira/browse/HIVE-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508923#comment-16508923
]
Alan Gates commented on HIVE-19862:
---
+1. Assuming it passes the ITestPostgres in standalone-metastore I don't think
there's any reason to wait for a nightly run as this change won't be tested by
the nightly run.
> Postgres init script has a glitch around UNIQUE_DATABASE
>
>
> Key: HIVE-19862
> URL: https://issues.apache.org/jira/browse/HIVE-19862
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19862.1.patch, HIVE-19862.2.patch
>
>
> {code}
> ALTER TABLE ONLY "DBS" ADD CONSTRAINT "UNIQUE_DATABASE" UNIQUE ("NAME");
> {code}
> Should also include "CTLG_NAME".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19861) Fix temp table path generation for acid table export
[ https://issues.apache.org/jira/browse/HIVE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19861: -- Status: Patch Available (was: Open) RB at https://reviews.apache.org/r/67540/ [~ekoifman], can you take a look? > Fix temp table path generation for acid table export > > > Key: HIVE-19861 > URL: https://issues.apache.org/jira/browse/HIVE-19861 > Project: Hive > Issue Type: Bug > Components: Import/Export, Transactions >Affects Versions: 3.0.0 >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19861.1.patch > > > Temp tables that are analyzed by the SemanticAnalyzer get their default > location set to a location in the session directory. Export of Acid tables > also creates temp tables, but this is done via a plan transformation, and the > temp table creation never goes through the SemanticAnalyzer, meaning the > location is not set. There is some other logic in DDLTask (which I am > changing in HIV-19837) which ends up automatically setting this path to the > default table location in the warehouse directory. This should be fixed so > that the path defaults to a location in the session directory, like with > normal temp tables. > cc [~ekoifman] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19862) Postgres init script has a glitch around UNIQUE_DATABASE
[
https://issues.apache.org/jira/browse/HIVE-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-19862:
--
Attachment: HIVE-19862.2.patch
> Postgres init script has a glitch around UNIQUE_DATABASE
>
>
> Key: HIVE-19862
> URL: https://issues.apache.org/jira/browse/HIVE-19862
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19862.1.patch, HIVE-19862.2.patch
>
>
> {code}
> ALTER TABLE ONLY "DBS" ADD CONSTRAINT "UNIQUE_DATABASE" UNIQUE ("NAME");
> {code}
> Should also include "CTLG_NAME".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19862) Postgres init script has a glitch around UNIQUE_DATABASE
[
https://issues.apache.org/jira/browse/HIVE-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-19862:
--
Target Version/s: 3.1.0, 3.0.1, 4.0.0 (was: 3.1.0, 4.0.0)
> Postgres init script has a glitch around UNIQUE_DATABASE
>
>
> Key: HIVE-19862
> URL: https://issues.apache.org/jira/browse/HIVE-19862
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19862.1.patch, HIVE-19862.2.patch
>
>
> {code}
> ALTER TABLE ONLY "DBS" ADD CONSTRAINT "UNIQUE_DATABASE" UNIQUE ("NAME");
> {code}
> Should also include "CTLG_NAME".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19862) Postgres init script has a glitch around UNIQUE_DATABASE
[
https://issues.apache.org/jira/browse/HIVE-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508916#comment-16508916
]
Daniel Dai commented on HIVE-19862:
---
Yes, hive-schema-3.0.0.postgres.sql has the same issue. Thanks Alan!
> Postgres init script has a glitch around UNIQUE_DATABASE
>
>
> Key: HIVE-19862
> URL: https://issues.apache.org/jira/browse/HIVE-19862
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19862.1.patch, HIVE-19862.2.patch
>
>
> {code}
> ALTER TABLE ONLY "DBS" ADD CONSTRAINT "UNIQUE_DATABASE" UNIQUE ("NAME");
> {code}
> Should also include "CTLG_NAME".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19861) Fix temp table path generation for acid table export
[ https://issues.apache.org/jira/browse/HIVE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19861: -- Attachment: HIVE-19861.1.patch > Fix temp table path generation for acid table export > > > Key: HIVE-19861 > URL: https://issues.apache.org/jira/browse/HIVE-19861 > Project: Hive > Issue Type: Bug > Components: Import/Export, Transactions >Affects Versions: 3.0.0 >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > Attachments: HIVE-19861.1.patch > > > Temp tables that are analyzed by the SemanticAnalyzer get their default > location set to a location in the session directory. Export of Acid tables > also creates temp tables, but this is done via a plan transformation, and the > temp table creation never goes through the SemanticAnalyzer, meaning the > location is not set. There is some other logic in DDLTask (which I am > changing in HIV-19837) which ends up automatically setting this path to the > default table location in the warehouse directory. This should be fixed so > that the path defaults to a location in the session directory, like with > normal temp tables. > cc [~ekoifman] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19862) Postgres init script has a glitch around UNIQUE_DATABASE
[
https://issues.apache.org/jira/browse/HIVE-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508905#comment-16508905
]
Alan Gates commented on HIVE-19862:
---
This fix should be applied to hive-schema-3.0.0 as well so that is lands in
Hive 3.0.1
> Postgres init script has a glitch around UNIQUE_DATABASE
>
>
> Key: HIVE-19862
> URL: https://issues.apache.org/jira/browse/HIVE-19862
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19862.1.patch
>
>
> {code}
> ALTER TABLE ONLY "DBS" ADD CONSTRAINT "UNIQUE_DATABASE" UNIQUE ("NAME");
> {code}
> Should also include "CTLG_NAME".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19770) Support for CBO for queries with multiple same columns in select
[
https://issues.apache.org/jira/browse/HIVE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508890#comment-16508890
]
Vineet Garg commented on HIVE-19770:
Also found an existing bug (HIVE-19863) which is fixed by this patch.
> Support for CBO for queries with multiple same columns in select
>
>
> Key: HIVE-19770
> URL: https://issues.apache.org/jira/browse/HIVE-19770
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19770.1.patch, HIVE-19770.2.patch
>
>
> Currently queries such as {code:sql} select a,a from t1 where b > 10 {code}
> are not supported for CBO.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19770) Support for CBO for queries with multiple same columns in select
[
https://issues.apache.org/jira/browse/HIVE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1650#comment-1650
]
Vineet Garg commented on HIVE-19770:
bq. One query has different (worse) plan (it now has cbo + vectorization) and
new plan for some reason contains extra reducer. The root cause for this has
yet to be determined but this should not introduce correctness issues.
This query consist of group by + bunch of PTF operations. Earlier in non-cbo
PTF was done on reduce side because mapper partition key was same as PTF key
but now PTF is done in separate vertex/reducer because partition key is changed
to contain all group by keys. Semantically this is correct but not optimized.
I'll open a JIRA to track this.
> Support for CBO for queries with multiple same columns in select
>
>
> Key: HIVE-19770
> URL: https://issues.apache.org/jira/browse/HIVE-19770
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19770.1.patch, HIVE-19770.2.patch
>
>
> Currently queries such as {code:sql} select a,a from t1 where b > 10 {code}
> are not supported for CBO.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19770) Support for CBO for queries with multiple same columns in select
[
https://issues.apache.org/jira/browse/HIVE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19770:
---
Attachment: HIVE-19770.2.patch
> Support for CBO for queries with multiple same columns in select
>
>
> Key: HIVE-19770
> URL: https://issues.apache.org/jira/browse/HIVE-19770
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19770.1.patch, HIVE-19770.2.patch
>
>
> Currently queries such as {code:sql} select a,a from t1 where b > 10 {code}
> are not supported for CBO.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19770) Support for CBO for queries with multiple same columns in select
[
https://issues.apache.org/jira/browse/HIVE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19770:
---
Status: Patch Available (was: Open)
> Support for CBO for queries with multiple same columns in select
>
>
> Key: HIVE-19770
> URL: https://issues.apache.org/jira/browse/HIVE-19770
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19770.1.patch, HIVE-19770.2.patch
>
>
> Currently queries such as {code:sql} select a,a from t1 where b > 10 {code}
> are not supported for CBO.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19770) Support for CBO for queries with multiple same columns in select
[
https://issues.apache.org/jira/browse/HIVE-19770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19770:
---
Status: Open (was: Patch Available)
> Support for CBO for queries with multiple same columns in select
>
>
> Key: HIVE-19770
> URL: https://issues.apache.org/jira/browse/HIVE-19770
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-19770.1.patch, HIVE-19770.2.patch
>
>
> Currently queries such as {code:sql} select a,a from t1 where b > 10 {code}
> are not supported for CBO.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19838) simplify & fix ColumnizedDeleteEventRegistry load loop
[ https://issues.apache.org/jira/browse/HIVE-19838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508877#comment-16508877 ] Sergey Shelukhin commented on HIVE-19838: - No, it's a different test. > simplify & fix ColumnizedDeleteEventRegistry load loop > -- > > Key: HIVE-19838 > URL: https://issues.apache.org/jira/browse/HIVE-19838 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19838.01.patch, HIVE-19838.patch > > > Apparently sometimes the delete count in ACID stats doesn't match what merger > actually returns. > It could be due to some deltas having duplicate deletes from parallel queries > (I guess?) that are being squashed by the merger or some other reasons beyond > my mortal comprehension. > The loop assumes the merger will return the exact number of records, so it > fails with array index exception. Also, it could actually be done in a single > loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19863) UNION query produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-19863:
--
> UNION query produce wrong results
> -
>
> Key: HIVE-19863
> URL: https://issues.apache.org/jira/browse/HIVE-19863
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> *Reproducer*
> {code:sql}
> SET hive.vectorized.execution.enabled=false;
> set hive.map.aggr=false;
> set hive.strict.checks.bucketing=false;
> set hive.explain.user=true;
> CREATE TABLE src1 (key STRING COMMENT 'default', value STRING COMMENT
> 'default') STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv3.txt" INTO TABLE src1;
> ANALYZE TABLE src1 COMPUTE STATISTICS;
> ANALYZE TABLE src1 COMPUTE STATISTICS FOR COLUMNS key,value;
> CREATE TABLE src (key STRING COMMENT 'default', value STRING COMMENT
> 'default') STORED AS TEXTFILE;
> LOAD DATA LOCAL INPATH "${hiveconf:test.data.dir}/kv1.txt" INTO TABLE src;
> ANALYZE TABLE src COMPUTE STATISTICS;
> ANALYZE TABLE src COMPUTE STATISTICS FOR COLUMNS key,value;
> SELECT x.key, z.value, y.value
> FROM src1 x JOIN src y ON (x.key = y.key)
> JOIN (select * from src1 union select * from src)z ON (x.value = z.value)
> union
> SELECT x.key, z.value, y.value
> FROM src1 x JOIN src y ON (x.key = y.key)
> JOIN (select * from src1 union select * from src)z ON (x.value = z.value);
> {code}
> *Expected Result*
> {code:sql}
> 128 val_128
> 146 val_146 val_146
> 150 val_150 val_150
> 213 val_213 val_213
> 224 val_224
> 238 val_238 val_238
> 255 val_255 val_255
> 273 val_273 val_273
> 278 val_278 val_278
> 311 val_311 val_311
> 369 val_369
> 401 val_401 val_401
> 406 val_406 val_406
> 66val_66 val_66
> 98val_98 val_98
> {code}
> *Actual Result*
> {code:sql}
> 128
> 146 val_146
> 150 val_150
> 213 val_213
> 224
> 238 val_238
> 255 val_255
> 273 val_273
> 278 val_278
> 311 val_311
> 369
> 401 val_401
> 406 val_406
> 66val_66
> 98val_98
> {code}
> One whole column is missing from the result
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19838) simplify & fix ColumnizedDeleteEventRegistry load loop
[ https://issues.apache.org/jira/browse/HIVE-19838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508850#comment-16508850 ] Prasanth Jayachandran commented on HIVE-19838: -- Is this related to HIVE-19597 or something new? > simplify & fix ColumnizedDeleteEventRegistry load loop > -- > > Key: HIVE-19838 > URL: https://issues.apache.org/jira/browse/HIVE-19838 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19838.01.patch, HIVE-19838.patch > > > Apparently sometimes the delete count in ACID stats doesn't match what merger > actually returns. > It could be due to some deltas having duplicate deletes from parallel queries > (I guess?) that are being squashed by the merger or some other reasons beyond > my mortal comprehension. > The loop assumes the merger will return the exact number of records, so it > fails with array index exception. Also, it could actually be done in a single > loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19743) hive is not pushing predicate down to HBaseStorageHandler if hive key mapped with hbase is stored as varchar
[
https://issues.apache.org/jira/browse/HIVE-19743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh resolved HIVE-19743.
---
Resolution: Invalid
> hive is not pushing predicate down to HBaseStorageHandler if hive key mapped
> with hbase is stored as varchar
>
>
> Key: HIVE-19743
> URL: https://issues.apache.org/jira/browse/HIVE-19743
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler, Hive
>Affects Versions: 2.1.0
> Environment: java8,centos7
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> Steps to Reproduce:
> {code}
> //hbase table
> create 'mytable', 'cf'
> put 'mytable', 'ABCDEF|GHIJK|ijj123kl-mn4o-4pq5-678r-st90123u0v4',
> 'cf:message', 'hello world'
> put 'mytable', 'ABCDEF1|GHIJK1|ijj123kl-mn4o-4pq5-678r-st90123u0v41',
> 'cf:foo', 0x0
> // hive table with key stored as varchar
> show create table hbase_table_4;
> +---+--+
> | createtab_stmt |
> +---+--+
> | CREATE EXTERNAL TABLE `hbase_table_4`( |
> | `hbase_key` varchar(80) COMMENT 'from deserializer', |
> | `value` string COMMENT 'from deserializer', |
> | `value1` string COMMENT 'from deserializer') |
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.hbase.HBaseSerDe' |
> | STORED BY |
> | 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' |
> | WITH SERDEPROPERTIES ( |
> | 'hbase.columns.mapping'=':key,cf:foo,cf:message', |
> | 'serialization.format'='1') |
> | TBLPROPERTIES ( |
> | 'COLUMN_STATS_ACCURATE'='\{\"BASIC_STATS\":\"true\"}', |
> | 'hbase.table.name'='mytable', |
> | 'numFiles'='0', |
> | 'numRows'='0', |
> | 'rawDataSize'='0', |
> | 'totalSize'='0', |
> | 'transient_lastDdlTime'='1527708430') |
> +---+--+
>
> // hive table key stored as string
> CREATE EXTERNAL TABLE `hbase_table_5`( |
> | `hbase_key` string COMMENT 'from deserializer', |
> | `value` string COMMENT 'from deserializer', |
> | `value1` string COMMENT 'from deserializer') |
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.hbase.HBaseSerDe' |
> | STORED BY |
> | 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' |
> | WITH SERDEPROPERTIES ( |
> | 'hbase.columns.mapping'=':key,cf:foo,cf:message', |
> | 'serialization.format'='1') |
> | TBLPROPERTIES ( |
> | 'COLUMN_STATS_ACCURATE'='\{\"BASIC_STATS\":\"true\"}', |
> | 'hbase.table.name'='mytable', |
> | 'numFiles'='0', |
> | 'numRows'='0', |
> | 'rawDataSize'='0', |
> | 'totalSize'='0', |
> | 'transient_lastDdlTime'='1527708520') |
>
> Explain Plan
> explain select * from hbase_table_4 where
> hbase_key='ABCDEF|GHIJK|ijj123kl-mn4o-4pq5-678r-st90123u0v4'
> Stage-0
> |
> | Fetch Operator
> |
> | limit:-1
> |
> | Select Operator [SEL_2]
> |
> | Output:["_col0","_col1","_col2"]
> |
> | Filter Operator [FIL_4]
> |
> | predicate:(UDFToString(hbase_key) =
> 'ABCDEF|GHIJK|ijj123kl-mn4o-4pq5-678r-st90123u0v4') |
> | TableScan [TS_0]
> |
> | Output:["hbase_key","value","value1"]
>
> explain on table with key stored as string
> explain
[jira] [Assigned] (HIVE-19743) hive is not pushing predicate down to HBaseStorageHandler if hive key mapped with hbase is stored as varchar
[
https://issues.apache.org/jira/browse/HIVE-19743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh reassigned HIVE-19743:
-
Assignee: Rajkumar Singh
> hive is not pushing predicate down to HBaseStorageHandler if hive key mapped
> with hbase is stored as varchar
>
>
> Key: HIVE-19743
> URL: https://issues.apache.org/jira/browse/HIVE-19743
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler, Hive
>Affects Versions: 2.1.0
> Environment: java8,centos7
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
>
> Steps to Reproduce:
> {code}
> //hbase table
> create 'mytable', 'cf'
> put 'mytable', 'ABCDEF|GHIJK|ijj123kl-mn4o-4pq5-678r-st90123u0v4',
> 'cf:message', 'hello world'
> put 'mytable', 'ABCDEF1|GHIJK1|ijj123kl-mn4o-4pq5-678r-st90123u0v41',
> 'cf:foo', 0x0
> // hive table with key stored as varchar
> show create table hbase_table_4;
> +---+--+
> | createtab_stmt |
> +---+--+
> | CREATE EXTERNAL TABLE `hbase_table_4`( |
> | `hbase_key` varchar(80) COMMENT 'from deserializer', |
> | `value` string COMMENT 'from deserializer', |
> | `value1` string COMMENT 'from deserializer') |
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.hbase.HBaseSerDe' |
> | STORED BY |
> | 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' |
> | WITH SERDEPROPERTIES ( |
> | 'hbase.columns.mapping'=':key,cf:foo,cf:message', |
> | 'serialization.format'='1') |
> | TBLPROPERTIES ( |
> | 'COLUMN_STATS_ACCURATE'='\{\"BASIC_STATS\":\"true\"}', |
> | 'hbase.table.name'='mytable', |
> | 'numFiles'='0', |
> | 'numRows'='0', |
> | 'rawDataSize'='0', |
> | 'totalSize'='0', |
> | 'transient_lastDdlTime'='1527708430') |
> +---+--+
>
> // hive table key stored as string
> CREATE EXTERNAL TABLE `hbase_table_5`( |
> | `hbase_key` string COMMENT 'from deserializer', |
> | `value` string COMMENT 'from deserializer', |
> | `value1` string COMMENT 'from deserializer') |
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.hbase.HBaseSerDe' |
> | STORED BY |
> | 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' |
> | WITH SERDEPROPERTIES ( |
> | 'hbase.columns.mapping'=':key,cf:foo,cf:message', |
> | 'serialization.format'='1') |
> | TBLPROPERTIES ( |
> | 'COLUMN_STATS_ACCURATE'='\{\"BASIC_STATS\":\"true\"}', |
> | 'hbase.table.name'='mytable', |
> | 'numFiles'='0', |
> | 'numRows'='0', |
> | 'rawDataSize'='0', |
> | 'totalSize'='0', |
> | 'transient_lastDdlTime'='1527708520') |
>
> Explain Plan
> explain select * from hbase_table_4 where
> hbase_key='ABCDEF|GHIJK|ijj123kl-mn4o-4pq5-678r-st90123u0v4'
> Stage-0
> |
> | Fetch Operator
> |
> | limit:-1
> |
> | Select Operator [SEL_2]
> |
> | Output:["_col0","_col1","_col2"]
> |
> | Filter Operator [FIL_4]
> |
> | predicate:(UDFToString(hbase_key) =
> 'ABCDEF|GHIJK|ijj123kl-mn4o-4pq5-678r-st90123u0v4') |
> | TableScan [TS_0]
> |
> | Output:["hbase_key","value","value1"]
>
> explain on table with key stored as string
>
[jira] [Commented] (HIVE-19743) hive is not pushing predicate down to HBaseStorageHandler if hive key mapped with hbase is stored as varchar
[
https://issues.apache.org/jira/browse/HIVE-19743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508837#comment-16508837
]
Rajkumar Singh commented on HIVE-19743:
---
varchar() != string - that make sense, closing this jira, Thanks for your help.
> hive is not pushing predicate down to HBaseStorageHandler if hive key mapped
> with hbase is stored as varchar
>
>
> Key: HIVE-19743
> URL: https://issues.apache.org/jira/browse/HIVE-19743
> Project: Hive
> Issue Type: Bug
> Components: HBase Handler, Hive
>Affects Versions: 2.1.0
> Environment: java8,centos7
>Reporter: Rajkumar Singh
>Priority: Major
>
> Steps to Reproduce:
> {code}
> //hbase table
> create 'mytable', 'cf'
> put 'mytable', 'ABCDEF|GHIJK|ijj123kl-mn4o-4pq5-678r-st90123u0v4',
> 'cf:message', 'hello world'
> put 'mytable', 'ABCDEF1|GHIJK1|ijj123kl-mn4o-4pq5-678r-st90123u0v41',
> 'cf:foo', 0x0
> // hive table with key stored as varchar
> show create table hbase_table_4;
> +---+--+
> | createtab_stmt |
> +---+--+
> | CREATE EXTERNAL TABLE `hbase_table_4`( |
> | `hbase_key` varchar(80) COMMENT 'from deserializer', |
> | `value` string COMMENT 'from deserializer', |
> | `value1` string COMMENT 'from deserializer') |
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.hbase.HBaseSerDe' |
> | STORED BY |
> | 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' |
> | WITH SERDEPROPERTIES ( |
> | 'hbase.columns.mapping'=':key,cf:foo,cf:message', |
> | 'serialization.format'='1') |
> | TBLPROPERTIES ( |
> | 'COLUMN_STATS_ACCURATE'='\{\"BASIC_STATS\":\"true\"}', |
> | 'hbase.table.name'='mytable', |
> | 'numFiles'='0', |
> | 'numRows'='0', |
> | 'rawDataSize'='0', |
> | 'totalSize'='0', |
> | 'transient_lastDdlTime'='1527708430') |
> +---+--+
>
> // hive table key stored as string
> CREATE EXTERNAL TABLE `hbase_table_5`( |
> | `hbase_key` string COMMENT 'from deserializer', |
> | `value` string COMMENT 'from deserializer', |
> | `value1` string COMMENT 'from deserializer') |
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.hbase.HBaseSerDe' |
> | STORED BY |
> | 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' |
> | WITH SERDEPROPERTIES ( |
> | 'hbase.columns.mapping'=':key,cf:foo,cf:message', |
> | 'serialization.format'='1') |
> | TBLPROPERTIES ( |
> | 'COLUMN_STATS_ACCURATE'='\{\"BASIC_STATS\":\"true\"}', |
> | 'hbase.table.name'='mytable', |
> | 'numFiles'='0', |
> | 'numRows'='0', |
> | 'rawDataSize'='0', |
> | 'totalSize'='0', |
> | 'transient_lastDdlTime'='1527708520') |
>
> Explain Plan
> explain select * from hbase_table_4 where
> hbase_key='ABCDEF|GHIJK|ijj123kl-mn4o-4pq5-678r-st90123u0v4'
> Stage-0
> |
> | Fetch Operator
> |
> | limit:-1
> |
> | Select Operator [SEL_2]
> |
> | Output:["_col0","_col1","_col2"]
> |
> | Filter Operator [FIL_4]
> |
> | predicate:(UDFToString(hbase_key) =
> 'ABCDEF|GHIJK|ijj123kl-mn4o-4pq5-678r-st90123u0v4') |
> | TableScan [TS_0]
> |
> | Output:["hbase_key","value","value1"]
>
>
[jira] [Updated] (HIVE-19862) Postgres init script has a glitch around UNIQUE_DATABASE
[
https://issues.apache.org/jira/browse/HIVE-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-19862:
--
Attachment: HIVE-19862.1.patch
> Postgres init script has a glitch around UNIQUE_DATABASE
>
>
> Key: HIVE-19862
> URL: https://issues.apache.org/jira/browse/HIVE-19862
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19862.1.patch
>
>
> {code}
> ALTER TABLE ONLY "DBS" ADD CONSTRAINT "UNIQUE_DATABASE" UNIQUE ("NAME");
> {code}
> Should also include "CTLG_NAME".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19862) Postgres init script has a glitch around UNIQUE_DATABASE
[
https://issues.apache.org/jira/browse/HIVE-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai updated HIVE-19862:
--
Status: Patch Available (was: Open)
> Postgres init script has a glitch around UNIQUE_DATABASE
>
>
> Key: HIVE-19862
> URL: https://issues.apache.org/jira/browse/HIVE-19862
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19862.1.patch
>
>
> {code}
> ALTER TABLE ONLY "DBS" ADD CONSTRAINT "UNIQUE_DATABASE" UNIQUE ("NAME");
> {code}
> Should also include "CTLG_NAME".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19862) Postgres init script has a glitch around UNIQUE_DATABASE
[
https://issues.apache.org/jira/browse/HIVE-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Dai reassigned HIVE-19862:
-
> Postgres init script has a glitch around UNIQUE_DATABASE
>
>
> Key: HIVE-19862
> URL: https://issues.apache.org/jira/browse/HIVE-19862
> Project: Hive
> Issue Type: Bug
> Components: Standalone Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
>
> {code}
> ALTER TABLE ONLY "DBS" ADD CONSTRAINT "UNIQUE_DATABASE" UNIQUE ("NAME");
> {code}
> Should also include "CTLG_NAME".
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19838) simplify & fix ColumnizedDeleteEventRegistry load loop
[ https://issues.apache.org/jira/browse/HIVE-19838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508829#comment-16508829 ] Sergey Shelukhin commented on HIVE-19838: - Looks unrelated, passes for me locally. cc [~prasanth_j] looks like these tests are flaky again, although this time they usually fail all together. > simplify & fix ColumnizedDeleteEventRegistry load loop > -- > > Key: HIVE-19838 > URL: https://issues.apache.org/jira/browse/HIVE-19838 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19838.01.patch, HIVE-19838.patch > > > Apparently sometimes the delete count in ACID stats doesn't match what merger > actually returns. > It could be due to some deltas having duplicate deletes from parallel queries > (I guess?) that are being squashed by the merger or some other reasons beyond > my mortal comprehension. > The loop assumes the merger will return the exact number of records, so it > fails with array index exception. Also, it could actually be done in a single > loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19815) Repl dump should not propagate the checkpoint and repl source properties
[
https://issues.apache.org/jira/browse/HIVE-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508827#comment-16508827
]
Hive QA commented on HIVE-19815:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 9s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-11717/patches/PreCommit-HIVE-Build-11717.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11717/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Repl dump should not propagate the checkpoint and repl source properties
>
>
> Key: HIVE-19815
> URL: https://issues.apache.org/jira/browse/HIVE-19815
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, repl
>Affects Versions: 3.1.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19815.01-branch-3.patch, HIVE-19815.01.patch,
> HIVE-19815.02.patch
>
>
> For replication scenarios of A-> B -> C the repl dump on B should not include
> the checkpoint property when dumping out table information.
> Alter tables/partitions during incremental should not propagate this as well.
> Also should not propagate the the db level parameters set by replication
> internally.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19826) OrcRawRecordMerger doesn't work for more than one file in non vectorized case
[ https://issues.apache.org/jira/browse/HIVE-19826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19826: Resolution: Fixed Fix Version/s: 4.0.0 3.1.0 Status: Resolved (was: Patch Available) Thanks for committing it though, I will resolve the jira for now > OrcRawRecordMerger doesn't work for more than one file in non vectorized case > - > > Key: HIVE-19826 > URL: https://issues.apache.org/jira/browse/HIVE-19826 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19826.patch > > > Key object in the map is reused and reset, leading to bizarre merges and > wrong results. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19853) Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
[
https://issues.apache.org/jira/browse/HIVE-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508822#comment-16508822
]
Hive QA commented on HIVE-19853:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927276/HIVE-19853.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14521 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11716/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11716/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11716/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927276 - PreCommit-HIVE-Build
> Arrow serializer needs to create a TimeStampMicroTZVector instead of
> TimeStampMicroVector
> -
>
> Key: HIVE-19853
> URL: https://issues.apache.org/jira/browse/HIVE-19853
> Project: Hive
> Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19853.1.patch
>
>
> HIVE-19723 changed nanosecond to microsecond in Arrow serialization. However,
> it needs to be microsecond with time zone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19838) simplify & fix ColumnizedDeleteEventRegistry load loop
[ https://issues.apache.org/jira/browse/HIVE-19838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19838: Attachment: HIVE-19838.01.patch > simplify & fix ColumnizedDeleteEventRegistry load loop > -- > > Key: HIVE-19838 > URL: https://issues.apache.org/jira/browse/HIVE-19838 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19838.01.patch, HIVE-19838.patch > > > Apparently sometimes the delete count in ACID stats doesn't match what merger > actually returns. > It could be due to some deltas having duplicate deletes from parallel queries > (I guess?) that are being squashed by the merger or some other reasons beyond > my mortal comprehension. > The loop assumes the merger will return the exact number of records, so it > fails with array index exception. Also, it could actually be done in a single > loop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19861) Fix temp table path generation for acid table export
[ https://issues.apache.org/jira/browse/HIVE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-19861: -- Affects Version/s: 3.0.0 > Fix temp table path generation for acid table export > > > Key: HIVE-19861 > URL: https://issues.apache.org/jira/browse/HIVE-19861 > Project: Hive > Issue Type: Bug > Components: Import/Export, Transactions >Affects Versions: 3.0.0 >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > > Temp tables that are analyzed by the SemanticAnalyzer get their default > location set to a location in the session directory. Export of Acid tables > also creates temp tables, but this is done via a plan transformation, and the > temp table creation never goes through the SemanticAnalyzer, meaning the > location is not set. There is some other logic in DDLTask (which I am > changing in HIV-19837) which ends up automatically setting this path to the > default table location in the warehouse directory. This should be fixed so > that the path defaults to a location in the session directory, like with > normal temp tables. > cc [~ekoifman] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19861) Fix temp table path generation for acid table export
[ https://issues.apache.org/jira/browse/HIVE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere reassigned HIVE-19861: - > Fix temp table path generation for acid table export > > > Key: HIVE-19861 > URL: https://issues.apache.org/jira/browse/HIVE-19861 > Project: Hive > Issue Type: Bug > Components: Import/Export, Transactions >Reporter: Jason Dere >Assignee: Jason Dere >Priority: Major > > Temp tables that are analyzed by the SemanticAnalyzer get their default > location set to a location in the session directory. Export of Acid tables > also creates temp tables, but this is done via a plan transformation, and the > temp table creation never goes through the SemanticAnalyzer, meaning the > location is not set. There is some other logic in DDLTask (which I am > changing in HIV-19837) which ends up automatically setting this path to the > default table location in the warehouse directory. This should be fixed so > that the path defaults to a location in the session directory, like with > normal temp tables. > cc [~ekoifman] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19378) "hive.lock.numretries" Is Misleading
[
https://issues.apache.org/jira/browse/HIVE-19378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508776#comment-16508776
]
BELUGA BEHR commented on HIVE-19378:
[~afan] At the end of the day, if the variables in the code change, that's OK,
but what I am concerned more about are the external user-facing configuration
'hive.lock.numretries' and 'hive.unlock.numretries'
Are you proposing that we change the configuration names as well to included
the word 'total'?
> "hive.lock.numretries" Is Misleading
>
>
> Key: HIVE-19378
> URL: https://issues.apache.org/jira/browse/HIVE-19378
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Alice Fan
>Priority: Minor
> Attachments: HIVE-19378.1.patch
>
>
> Configuration 'hive.lock.numretries' is confusing. It's not actually a
> 'retry' count, it's the total number of attempt to try:
>
> {code:java|title=ZooKeeperHiveLockManager.java}
> do {
> lastException = null;
> tryNum++;
> try {
> if (tryNum > 1) {
> Thread.sleep(sleepTime);
> prepareRetry();
> }
> ret = lockPrimitive(key, mode, keepAlive, parentCreated,
> conflictingLocks);
> ...
> } while (tryNum < numRetriesForLock);
> {code}
> So, from this code you can see that on the first loop, {{tryNum}} is set to
> 1, in which case, if the configuration num*retries* is set to 1, there will
> be one attempt total. With a *retry* value of 1, I would assume one initial
> attempt and one additional retry. Please change to:
> {code}
> while (tryNum <= numRetriesForLock);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19604) Incorrect Handling of Boolean in DruidSerde
[ https://issues.apache.org/jira/browse/HIVE-19604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508760#comment-16508760 ] Nishant Bangarwa commented on HIVE-19604: - [~ashutoshc] updated patch added minitest, We still need instanceof check as we store booleans as dimensions in druid which are of type string. In the query we get string if we directly query those dimension columns, In case we use expressions whoose result is boolean we will get a numeric value. > Incorrect Handling of Boolean in DruidSerde > --- > > Key: HIVE-19604 > URL: https://issues.apache.org/jira/browse/HIVE-19604 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19604.01.patch, HIVE-19604.patch > > > Results of boolean expressions from Druid are expressed in the form of > numeric 1 or 0. > When reading the results in DruidSerde both 1 and 0 are translated to String > and then we call Boolean.valueOf(stringForm), this leads to the boolean being > read always as false. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19853) Arrow serializer needs to create a TimeStampMicroTZVector instead of TimeStampMicroVector
[
https://issues.apache.org/jira/browse/HIVE-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508755#comment-16508755
]
Hive QA commented on HIVE-19853:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
21s{color} | {color:blue} ql in master has 2279 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} ql: The patch generated 43 new + 520 unchanged - 62
fixed = 563 total (was 582) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11716/dev-support/hive-personality.sh
|
| git revision | master / 8902563 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11716/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11716/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Arrow serializer needs to create a TimeStampMicroTZVector instead of
> TimeStampMicroVector
> -
>
> Key: HIVE-19853
> URL: https://issues.apache.org/jira/browse/HIVE-19853
> Project: Hive
> Issue Type: Bug
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19853.1.patch
>
>
> HIVE-19723 changed nanosecond to microsecond in Arrow serialization. However,
> it needs to be microsecond with time zone.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19604) Incorrect Handling of Boolean in DruidSerde
[ https://issues.apache.org/jira/browse/HIVE-19604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19604: Attachment: HIVE-19604.01.patch > Incorrect Handling of Boolean in DruidSerde > --- > > Key: HIVE-19604 > URL: https://issues.apache.org/jira/browse/HIVE-19604 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19604.01.patch, HIVE-19604.patch > > > Results of boolean expressions from Druid are expressed in the form of > numeric 1 or 0. > When reading the results in DruidSerde both 1 and 0 are translated to String > and then we call Boolean.valueOf(stringForm), this leads to the boolean being > read always as false. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19604) Incorrect Handling of Boolean in DruidSerde
[ https://issues.apache.org/jira/browse/HIVE-19604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19604: Status: Patch Available (was: Open) > Incorrect Handling of Boolean in DruidSerde > --- > > Key: HIVE-19604 > URL: https://issues.apache.org/jira/browse/HIVE-19604 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19604.01.patch, HIVE-19604.patch > > > Results of boolean expressions from Druid are expressed in the form of > numeric 1 or 0. > When reading the results in DruidSerde both 1 and 0 are translated to String > and then we call Boolean.valueOf(stringForm), this leads to the boolean being > read always as false. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19604) Incorrect Handling of Boolean in DruidSerde
[ https://issues.apache.org/jira/browse/HIVE-19604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nishant Bangarwa updated HIVE-19604: Status: Open (was: Patch Available) > Incorrect Handling of Boolean in DruidSerde > --- > > Key: HIVE-19604 > URL: https://issues.apache.org/jira/browse/HIVE-19604 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19604.patch > > > Results of boolean expressions from Druid are expressed in the form of > numeric 1 or 0. > When reading the results in DruidSerde both 1 and 0 are translated to String > and then we call Boolean.valueOf(stringForm), this leads to the boolean being > read always as false. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19860) HiveServer2 ObjectInspectorFactory memory leak with cachedUnionStructObjectInspector
[ https://issues.apache.org/jira/browse/HIVE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-19860: -- Issue Type: Bug (was: Improvement) > HiveServer2 ObjectInspectorFactory memory leak with > cachedUnionStructObjectInspector > > > Key: HIVE-19860 > URL: https://issues.apache.org/jira/browse/HIVE-19860 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 2.1.0 > Environment: hiveserver2 Interactive with LLAP. >Reporter: Rajkumar Singh >Priority: Major > Attachments: Screen Shot 2018-06-11 at 2.01.00 PM.png > > > hiveserver2 is start seeing the memory pressure once the > cachedUnionStructObjectInspector start going > [https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java#L345] > I did not see any eviction policy for cachedUnionStructObjectInspector, so we > should implement some size or time-based eviction policy. > !Screen Shot 2018-06-11 at 2.01.00 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19789) reenable orc_llap test
[ https://issues.apache.org/jira/browse/HIVE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508738#comment-16508738 ] Vineet Garg commented on HIVE-19789: Just to remind we haven't relaxed our "green run" policy therefore "test failures are unrelated" is not good enough reason to commit. You still require green run for master. > reenable orc_llap test > -- > > Key: HIVE-19789 > URL: https://issues.apache.org/jira/browse/HIVE-19789 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Matt McCline >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19789.01.patch, HIVE-19789.02.patch > > > Test has been disabled, looks like by mistake (or due to some issue with the > patch there that was never addressed), in HIVE-11394. > It needs to be reenabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19860) HiveServer2 ObjectInspectorFactory memory leak with cachedUnionStructObjectInspector
[ https://issues.apache.org/jira/browse/HIVE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-19860: -- Attachment: Screen Shot 2018-06-11 at 2.01.00 PM.png > HiveServer2 ObjectInspectorFactory memory leak with > cachedUnionStructObjectInspector > > > Key: HIVE-19860 > URL: https://issues.apache.org/jira/browse/HIVE-19860 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 2.1.0 > Environment: hiveserver2 Interactive with LLAP. >Reporter: Rajkumar Singh >Priority: Major > Attachments: Screen Shot 2018-06-11 at 2.01.00 PM.png > > > hiveserver2 is start seeing the memory pressure once the > cachedUnionStructObjectInspector start going > [https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java#L345] > I did not see any eviction policy for cachedUnionStructObjectInspector, so we > should implement some size or time-based eviction policy. > > !Screen Shot 2018-06-11 at 1.52.50 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19860) HiveServer2 ObjectInspectorFactory memory leak with cachedUnionStructObjectInspector
[ https://issues.apache.org/jira/browse/HIVE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-19860: -- Attachment: (was: Screen Shot 2018-06-11 at 1.52.50 PM.png) > HiveServer2 ObjectInspectorFactory memory leak with > cachedUnionStructObjectInspector > > > Key: HIVE-19860 > URL: https://issues.apache.org/jira/browse/HIVE-19860 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 2.1.0 > Environment: hiveserver2 Interactive with LLAP. >Reporter: Rajkumar Singh >Priority: Major > Attachments: Screen Shot 2018-06-11 at 2.01.00 PM.png > > > hiveserver2 is start seeing the memory pressure once the > cachedUnionStructObjectInspector start going > [https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java#L345] > I did not see any eviction policy for cachedUnionStructObjectInspector, so we > should implement some size or time-based eviction policy. > !Screen Shot 2018-06-11 at 2.01.00 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19860) HiveServer2 ObjectInspectorFactory memory leak with cachedUnionStructObjectInspector
[ https://issues.apache.org/jira/browse/HIVE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-19860: -- Description: hiveserver2 is start seeing the memory pressure once the cachedUnionStructObjectInspector start going [https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java#L345] I did not see any eviction policy for cachedUnionStructObjectInspector, so we should implement some size or time-based eviction policy. !Screen Shot 2018-06-11 at 2.01.00 PM.png! was: hiveserver2 is start seeing the memory pressure once the cachedUnionStructObjectInspector start going [https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java#L345] I did not see any eviction policy for cachedUnionStructObjectInspector, so we should implement some size or time-based eviction policy. !Screen Shot 2018-06-11 at 1.52.50 PM.png! > HiveServer2 ObjectInspectorFactory memory leak with > cachedUnionStructObjectInspector > > > Key: HIVE-19860 > URL: https://issues.apache.org/jira/browse/HIVE-19860 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 2.1.0 > Environment: hiveserver2 Interactive with LLAP. >Reporter: Rajkumar Singh >Priority: Major > Attachments: Screen Shot 2018-06-11 at 2.01.00 PM.png > > > hiveserver2 is start seeing the memory pressure once the > cachedUnionStructObjectInspector start going > [https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java#L345] > I did not see any eviction policy for cachedUnionStructObjectInspector, so we > should implement some size or time-based eviction policy. > !Screen Shot 2018-06-11 at 2.01.00 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19860) HiveServer2 ObjectInspectorFactory memory leak with cachedUnionStructObjectInspector
[ https://issues.apache.org/jira/browse/HIVE-19860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-19860: -- Attachment: Screen Shot 2018-06-11 at 1.52.50 PM.png > HiveServer2 ObjectInspectorFactory memory leak with > cachedUnionStructObjectInspector > > > Key: HIVE-19860 > URL: https://issues.apache.org/jira/browse/HIVE-19860 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 2.1.0 > Environment: hiveserver2 Interactive with LLAP. >Reporter: Rajkumar Singh >Priority: Major > Attachments: Screen Shot 2018-06-11 at 1.52.50 PM.png > > > hiveserver2 is start seeing the memory pressure once the > cachedUnionStructObjectInspector start going > [https://github.com/apache/hive/blob/master/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/ObjectInspectorFactory.java#L345] > I did not see any eviction policy for cachedUnionStructObjectInspector, so we > should implement some size or time-based eviction policy. > > !Screen Shot 2018-06-11 at 1.52.50 PM.png! -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19378) "hive.lock.numretries" Is Misleading
[
https://issues.apache.org/jira/browse/HIVE-19378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508732#comment-16508732
]
Alice Fan commented on HIVE-19378:
--
Hi [~belugabehr],
I think the logic should stay as "<" instead of "<=" as once it goes into the
while loop it will do tryNum++ at very beginning. Discussed with [~aihuaxu], he
mentioned usually "numRetries" is equivalent to numTotalTries.
Let me know what you think. I can change the variable name from numRetries to
numTotalTries, if you think that will make more sense. Let me know. Thank you!
> "hive.lock.numretries" Is Misleading
>
>
> Key: HIVE-19378
> URL: https://issues.apache.org/jira/browse/HIVE-19378
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.0.0, 2.4.0
>Reporter: BELUGA BEHR
>Assignee: Alice Fan
>Priority: Minor
> Attachments: HIVE-19378.1.patch
>
>
> Configuration 'hive.lock.numretries' is confusing. It's not actually a
> 'retry' count, it's the total number of attempt to try:
>
> {code:java|title=ZooKeeperHiveLockManager.java}
> do {
> lastException = null;
> tryNum++;
> try {
> if (tryNum > 1) {
> Thread.sleep(sleepTime);
> prepareRetry();
> }
> ret = lockPrimitive(key, mode, keepAlive, parentCreated,
> conflictingLocks);
> ...
> } while (tryNum < numRetriesForLock);
> {code}
> So, from this code you can see that on the first loop, {{tryNum}} is set to
> 1, in which case, if the configuration num*retries* is set to 1, there will
> be one attempt total. With a *retry* value of 1, I would assume one initial
> attempt and one additional retry. Please change to:
> {code}
> while (tryNum <= numRetriesForLock);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19824) Improve online datasize estimations for MapJoins
[
https://issues.apache.org/jira/browse/HIVE-19824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508730#comment-16508730
]
Zoltan Haindrich commented on HIVE-19824:
-
[~sershe] I'm not sure; but I feel that you might have wanted to note this to
the other ticket (HIVE-19823); this is only about correcting the used memory
estimation in {{ConvertMapJoin}} - to prevent the conversion of joins which
will eventually lead to an OOM...
> Improve online datasize estimations for MapJoins
>
>
> Key: HIVE-19824
> URL: https://issues.apache.org/jira/browse/HIVE-19824
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19824.01.patch, HIVE-19824.01wip01.patch,
> HIVE-19824.01wip01.patch
>
>
> Statistics.datasize() only accounts for "real" data size; but for example
> handling 1M rows might introduce some datastructure overhead...if the "real"
> data is small - even this overhead might become the real memory usage
> for 6.5M rows of (int,int) the estimation is 52MB
> in reality this eats up ~260MB from which 210MB is used to service the
> hashmap functionality to that many rows.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19789) reenable orc_llap test
[ https://issues.apache.org/jira/browse/HIVE-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508729#comment-16508729 ] Prasanth Jayachandran commented on HIVE-19789: -- This patch got committed without a green run causing 3 of my patches to be resubmitted or rebased (HIVE-19629, HIVE-19851 and HIVE-19852)... These small enough changes can be pain for others (1 of my patch is >2MB that needed rebase and other 2 are small enough changes). > reenable orc_llap test > -- > > Key: HIVE-19789 > URL: https://issues.apache.org/jira/browse/HIVE-19789 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Matt McCline >Priority: Major > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-19789.01.patch, HIVE-19789.02.patch > > > Test has been disabled, looks like by mistake (or due to some issue with the > patch there that was never addressed), in HIVE-11394. > It needs to be reenabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[
https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508723#comment-16508723
]
Jesus Camacho Rodriguez commented on HIVE-19855:
[~sershe], the error that I see that the test hits (at least locally) is the
following:
{noformat}
java.lang.UnsupportedOperationException: null
at
org.apache.hadoop.hive.ql.lockmgr.DbLockManager$DbHiveLock.getHiveLockMode(DbLockManager.java:330)
~[classes/:?]
at
org.apache.hadoop.hive.ql.Driver.isValidTxnListState(Driver.java:812)
~[classes/:?]
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1940)
~[classes/:?]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1683) ~[classes/:?]
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1672) ~[classes/:?]
at
org.apache.hadoop.hive.ql.DriverUtils.runOnDriver(DriverUtils.java:40)
~[classes/:?]
at
org.apache.hadoop.hive.ql.stats.StatsUpdaterThread.runOneWorkerIteration(StatsUpdaterThread.java:578)
[classes/:?]
at
org.apache.hadoop.hive.ql.stats.StatsUpdaterThread$WorkerRunnable.run(StatsUpdaterThread.java:618)
[classes/:?]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_101]
...
{noformat}
HIVE-19859 fixes that issue.
> TestStatsUpdaterThread.testQueueingWithThreads fails often
> --
>
> Key: HIVE-19855
> URL: https://issues.apache.org/jira/browse/HIVE-19855
> Project: Hive
> Issue Type: Test
>Reporter: Peter Vary
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19855.patch
>
>
> Taking a look at here, it seems, that
> TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second
> run:
> [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/]
>
> We should fix this
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19826) OrcRawRecordMerger doesn't work for more than one file in non vectorized case
[ https://issues.apache.org/jira/browse/HIVE-19826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508718#comment-16508718 ] Sergey Shelukhin commented on HIVE-19826: - I think you just modify errata.txt in the root dir and commit as addendum commit, I assume no review or QA is needed > OrcRawRecordMerger doesn't work for more than one file in non vectorized case > - > > Key: HIVE-19826 > URL: https://issues.apache.org/jira/browse/HIVE-19826 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19826.patch > > > Key object in the map is reused and reset, leading to bizarre merges and > wrong results. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508714#comment-16508714 ] Sergey Shelukhin commented on HIVE-19855: - Hmm.. is HIVE-19859 fixing an issue for this test? > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508700#comment-16508700 ] Jesus Camacho Rodriguez edited comment on HIVE-19855 at 6/11/18 8:49 PM: - [~pvary], uploaded patch that disables the test. Can you +1? The test will be reenabled by HIVE-19859 (patch already available), I could reproduce it locally 1 out of 2 runs. Cc [~sershe] was (Author: jcamachorodriguez): [~pvary], uploaded patch that disables the test. The test will be reenabled by HIVE-19859 (patch already available), I could reproduce it locally 1 out of 2 runs. Cc [~sershe] > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19584) Dictionary encoding for string types
[
https://issues.apache.org/jira/browse/HIVE-19584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508710#comment-16508710
]
Hive QA commented on HIVE-19584:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12927274/HIVE-19584.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14508 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=152)
[parquet_complex_types_vectorization.q,import_exported_table.q,except_distinct.q,orc_merge10.q,mapreduce1.q,explainuser_2.q,mm_all.q,orc_merge4.q,dynamic_semijoin_user_level.q,temp_table_external.q,orc_struct_type_vectorization.q,llap_udf.q,schemeAuthority.q,external_table_with_space_in_location_path.q,remote_script.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11715/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11715/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11715/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12927274 - PreCommit-HIVE-Build
> Dictionary encoding for string types
>
>
> Key: HIVE-19584
> URL: https://issues.apache.org/jira/browse/HIVE-19584
> Project: Hive
> Issue Type: Sub-task
>Reporter: Teddy Choi
>Assignee: Teddy Choi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-19584.1.patch, HIVE-19584.2.patch,
> HIVE-19584.3.patch, HIVE-19584.4.patch, HIVE-19584.5.patch
>
>
> Apache Arrow supports dictionary encoding for some data types. So implement
> dictionary encoding for string types in Arrow SerDe.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19605) TAB_COL_STATS table has no index on db/table name
[ https://issues.apache.org/jira/browse/HIVE-19605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-19605: --- Fix Version/s: 2.4.0 > TAB_COL_STATS table has no index on db/table name > - > > Key: HIVE-19605 > URL: https://issues.apache.org/jira/browse/HIVE-19605 > Project: Hive > Issue Type: Bug > Components: Metastore >Reporter: Todd Lipcon >Assignee: Vihang Karajgaonkar >Priority: Major > Fix For: 2.4.0, 3.1.0, 4.0.0 > > Attachments: HIVE-19605.01.patch, HIVE-19605.02.patch, > HIVE-19605.03-branch-2.patch > > > The TAB_COL_STATS table is missing an index on (CAT_NAME, DB_NAME, > TABLE_NAME). The getTableColumnStatistics call queries based on this tuple. > This makes those queries take a significant amount of time in large > metastores since they do a full table scan. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19203) Thread-Safety Issue in HiveMetaStore
[
https://issues.apache.org/jira/browse/HIVE-19203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508707#comment-16508707
]
Alice Fan commented on HIVE-19203:
--
[~belugabehr] and [~aihuaxu],
Variable nextSerialNum is changed to AtomicInteger. Please review and provide
feedback. Thanks!
> Thread-Safety Issue in HiveMetaStore
>
>
> Key: HIVE-19203
> URL: https://issues.apache.org/jira/browse/HIVE-19203
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: Alice Fan
>Priority: Minor
> Attachments: HIVE-19203.1.patch
>
>
> [https://github.com/apache/hive/blob/550d1e1196b7c801c572092db974a459aac6c249/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L345-L351]
> {code:java}
> private static int nextSerialNum = 0;
> private static ThreadLocal threadLocalId = new
> ThreadLocal() {
> @Override
> protected Integer initialValue() {
> return nextSerialNum++;
> }
> };{code}
>
> {{nextSerialNum}} needs to be an atomic value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508700#comment-16508700 ] Jesus Camacho Rodriguez commented on HIVE-19855: [~pvary], uploaded patch that disables the test. The test will be reenabled by HIVE-19859 (patch already available), I could reproduce it locally 1 out of 2 runs. > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508700#comment-16508700 ] Jesus Camacho Rodriguez edited comment on HIVE-19855 at 6/11/18 8:35 PM: - [~pvary], uploaded patch that disables the test. The test will be reenabled by HIVE-19859 (patch already available), I could reproduce it locally 1 out of 2 runs. Cc [~sershe] was (Author: jcamachorodriguez): [~pvary], uploaded patch that disables the test. The test will be reenabled by HIVE-19859 (patch already available), I could reproduce it locally 1 out of 2 runs. > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19859) Inspect lock components for DBHiveLock while verifying whether transaction list is valid
[ https://issues.apache.org/jira/browse/HIVE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19859: --- Attachment: (was: HIVE-19859.patch) > Inspect lock components for DBHiveLock while verifying whether transaction > list is valid > > > Key: HIVE-19859 > URL: https://issues.apache.org/jira/browse/HIVE-19859 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19859.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19859) Inspect lock components for DBHiveLock while verifying whether transaction list is valid
[ https://issues.apache.org/jira/browse/HIVE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19859: --- Attachment: HIVE-19859.patch > Inspect lock components for DBHiveLock while verifying whether transaction > list is valid > > > Key: HIVE-19859 > URL: https://issues.apache.org/jira/browse/HIVE-19859 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19859.patch, HIVE-19859.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19855: --- Attachment: HIVE-19855.patch > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19855: --- Attachment: (was: HIVE-19855.patch) > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19859) Inspect lock components for DBHiveLock while verifying whether transaction list is valid
[ https://issues.apache.org/jira/browse/HIVE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19859: --- Attachment: HIVE-19859.patch > Inspect lock components for DBHiveLock while verifying whether transaction > list is valid > > > Key: HIVE-19859 > URL: https://issues.apache.org/jira/browse/HIVE-19859 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19859.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-19855 started by Jesus Camacho Rodriguez. -- > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19855: --- Status: Open (was: Patch Available) > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19859) Inspect lock components for DBHiveLock while verifying whether transaction list is valid
[ https://issues.apache.org/jira/browse/HIVE-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19859: --- Assignee: Jesus Camacho Rodriguez Status: Patch Available (was: Open) > Inspect lock components for DBHiveLock while verifying whether transaction > list is valid > > > Key: HIVE-19859 > URL: https://issues.apache.org/jira/browse/HIVE-19859 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-19855: -- Assignee: Jesus Camacho Rodriguez > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19855) TestStatsUpdaterThread.testQueueingWithThreads fails often
[ https://issues.apache.org/jira/browse/HIVE-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19855: --- Attachment: HIVE-19855.patch > TestStatsUpdaterThread.testQueueingWithThreads fails often > -- > > Key: HIVE-19855 > URL: https://issues.apache.org/jira/browse/HIVE-19855 > Project: Hive > Issue Type: Test >Reporter: Peter Vary >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19855.patch > > > Taking a look at here, it seems, that > TestStatsUpdaterThread.testQueueingWithThreads fails on almost every second > run: > [https://builds.apache.org/job/PreCommit-HIVE-Build/11698/testReport/junit/org.apache.hadoop.hive.ql.stats/TestStatsUpdaterThread/testQueueingWithThreads/history/] > > We should fix this -- This message was sent by Atlassian JIRA (v7.6.3#76005)
