[jira] [Commented] (HIVE-18578) Some class has missed the ASF header
[
https://issues.apache.org/jira/browse/HIVE-18578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344643#comment-16344643
]
Hive QA commented on HIVE-18578:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} itests/qtest-druid: The patch generated 0 new + 5
unchanged - 1 fixed = 5 total (was 6) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 9m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 0edf6b3 |
| Default Java | 1.8.0_111 |
| modules | C: itests/qtest-druid U: itests/qtest-druid |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8921/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Some class has missed the ASF header
>
>
> Key: HIVE-18578
> URL: https://issues.apache.org/jira/browse/HIVE-18578
> Project: Hive
> Issue Type: Bug
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Trivial
> Fix For: 3.0.0
>
> Attachments: HIVE-18578.1.patch
>
>
> Some class has missed the ASF header
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18579) Changes from HIVE-18495 introduced import paths from shaded jars
[
https://issues.apache.org/jira/browse/HIVE-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-18579:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master.
> Changes from HIVE-18495 introduced import paths from shaded jars
>
>
> Key: HIVE-18579
> URL: https://issues.apache.org/jira/browse/HIVE-18579
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Ashutosh Chauhan
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18579.patch
>
>
> When compiling the latest code after HIVE-18495 seeing the following issue:
> {noformat}
> -
> [ERROR] COMPILATION ERROR :
> -
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java:[29,64]
> package org.apache.hadoop.hbase.shaded.com.google.common.collect does not
> exist
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[43,68]
> package org.apache.hadoop.hbase.shaded.com.google.common.annotations does
> not exist
> ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[100,4]
> cannot find symbol
> symbol: class VisibleForTesting
> location: class org.apache.hive.testutils.MiniZooKeeperCluster
> [INFO] 3 errors
> {noformat}
> Seems like org.apache.hadoop.hbase.shaded.com.google.* is being used, I am
> guessing the idea was to use com.google.*. Not sure why we didn't see this
> failing in Apache Hive build system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-18574:
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Gopal!
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18577) SemanticAnalyzer.validate has some pointless metastore calls
[
https://issues.apache.org/jira/browse/HIVE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344624#comment-16344624
]
Hive QA commented on HIVE-18577:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908252/HIVE-18577.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 12779 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=150)
[intersect_all.q,unionDistinct_1.q,orc_ppd_schema_evol_3a.q,table_nonprintable.q,tez_union_dynamic_partition.q,tez_union_dynamic_partition_2.q,temp_table_external.q,global_limit.q,llap_udf.q,schemeAuthority.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,parallel_colstats.q]
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8920/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8920/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8920/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12908252 - PreCommit-HIVE-Build
> SemanticAnalyzer.validate has some pointless metastore calls
>
>
> Key: HIVE-18577
> URL: https://issues.apache.org/jira/browse/HIVE-18577
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18577.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344623#comment-16344623
]
Ashutosh Chauhan commented on HIVE-18574:
-
On hive repo:
{code}
$ git grep "import io.netty" | wc -l
74
$ git grep "import org.jboss.netty" | wc -l
49
{code}
is what I am talking about. Hive itself uses both netty3 and netty4 which is
unnecessary.
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18579) Changes from HIVE-18495 introduced import paths from shaded jars
[
https://issues.apache.org/jira/browse/HIVE-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich reassigned HIVE-18579:
---
Assignee: Ashutosh Chauhan (was: Zoltan Haindrich)
+1
I'm not sure how this could have became undetected...it seems that we've ~3
shaded guava libraries loaded..plus the regular :)
this should be checked somehow; to prevent future problems arising from this
source...
> Changes from HIVE-18495 introduced import paths from shaded jars
>
>
> Key: HIVE-18579
> URL: https://issues.apache.org/jira/browse/HIVE-18579
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Ashutosh Chauhan
>Priority: Critical
> Attachments: HIVE-18579.patch
>
>
> When compiling the latest code after HIVE-18495 seeing the following issue:
> {noformat}
> -
> [ERROR] COMPILATION ERROR :
> -
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java:[29,64]
> package org.apache.hadoop.hbase.shaded.com.google.common.collect does not
> exist
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[43,68]
> package org.apache.hadoop.hbase.shaded.com.google.common.annotations does
> not exist
> ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[100,4]
> cannot find symbol
> symbol: class VisibleForTesting
> location: class org.apache.hive.testutils.MiniZooKeeperCluster
> [INFO] 3 errors
> {noformat}
> Seems like org.apache.hadoop.hbase.shaded.com.google.* is being used, I am
> guessing the idea was to use com.google.*. Not sure why we didn't see this
> failing in Apache Hive build system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently
[
https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344606#comment-16344606
]
anishek commented on HIVE-16886:
nope i dont have one either.
> HMS log notifications may have duplicated event IDs if multiple HMS are
> running concurrently
>
>
> Key: HIVE-16886
> URL: https://issues.apache.org/jira/browse/HIVE-16886
> Project: Hive
> Issue Type: Bug
> Components: Hive, Metastore
>Affects Versions: 3.0.0, 2.3.2, 2.3.3
>Reporter: Sergio Peña
>Assignee: anishek
>Priority: Major
> Labels: TODOC3.0
> Fix For: 3.0.0
>
> Attachments: HIVE-16886.1.patch, HIVE-16886.2.patch,
> HIVE-16886.3.patch, HIVE-16886.4.patch, HIVE-16886.5.patch,
> HIVE-16886.6.patch, HIVE-16886.7.patch, HIVE-16886.8.patch,
> datastore-identity-holes.diff
>
>
> When running multiple Hive Metastore servers and DB notifications are
> enabled, I could see that notifications can be persisted with a duplicated
> event ID.
> This does not happen when running multiple threads in a single HMS node due
> to the locking acquired on the DbNotificationsLog class, but multiple HMS
> could cause conflicts.
> The issue is in the ObjectStore#addNotificationEvent() method. The event ID
> fetched from the datastore is used for the new notification, incremented in
> the server itself, then persisted or updated back to the datastore. If 2
> servers read the same ID, then these 2 servers write a new notification with
> the same ID.
> The event ID is not unique nor a primary key.
> Here's a test case using the TestObjectStore class that confirms this issue:
> {noformat}
> @Test
> public void testConcurrentAddNotifications() throws ExecutionException,
> InterruptedException {
> final int NUM_THREADS = 2;
> CountDownLatch countIn = new CountDownLatch(NUM_THREADS);
> CountDownLatch countOut = new CountDownLatch(1);
> HiveConf conf = new HiveConf();
> conf.setVar(HiveConf.ConfVars.METASTORE_EXPRESSION_PROXY_CLASS,
> MockPartitionExpressionProxy.class.getName());
> ExecutorService executorService =
> Executors.newFixedThreadPool(NUM_THREADS);
> FutureTask tasks[] = new FutureTask[NUM_THREADS];
> for (int i=0; i final int n = i;
> tasks[i] = new FutureTask(new Callable() {
> @Override
> public Void call() throws Exception {
> ObjectStore store = new ObjectStore();
> store.setConf(conf);
> NotificationEvent dbEvent =
> new NotificationEvent(0, 0,
> EventMessage.EventType.CREATE_DATABASE.toString(), "CREATE DATABASE DB" + n);
> System.out.println("ADDING NOTIFICATION");
> countIn.countDown();
> countOut.await();
> store.addNotificationEvent(dbEvent);
> System.out.println("FINISH NOTIFICATION");
> return null;
> }
> });
> executorService.execute(tasks[i]);
> }
> countIn.await();
> countOut.countDown();
> for (int i = 0; i < NUM_THREADS; ++i) {
> tasks[i].get();
> }
> NotificationEventResponse eventResponse =
> objectStore.getNextNotification(new NotificationEventRequest());
> Assert.assertEquals(2, eventResponse.getEventsSize());
> Assert.assertEquals(1, eventResponse.getEvents().get(0).getEventId());
> // This fails because the next notification has an event ID = 1
> Assert.assertEquals(2, eventResponse.getEvents().get(1).getEventId());
> }
> {noformat}
> The last assertion fails expecting an event ID 1 instead of 2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18546) Remove unnecessary code introduced in HIVE-14498
[ https://issues.apache.org/jira/browse/HIVE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-18546: --- Attachment: HIVE-18546.03.patch > Remove unnecessary code introduced in HIVE-14498 > > > Key: HIVE-18546 > URL: https://issues.apache.org/jira/browse/HIVE-18546 > Project: Hive > Issue Type: Bug > Components: Materialized views >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-18546.02.patch, HIVE-18546.03.patch > > > HIVE-14498 introduced some code to check the invalidation of materialized > views that can be simplified, relying instead on existing transaction ids. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18048) Vectorization: Support Struct type with vectorization
[ https://issues.apache.org/jira/browse/HIVE-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344596#comment-16344596 ] Colin Ma commented on HIVE-18048: - [~Ferd], thanks for review. The patch is updated as your comments, new tests for ORC and filter operator are added. > Vectorization: Support Struct type with vectorization > - > > Key: HIVE-18048 > URL: https://issues.apache.org/jira/browse/HIVE-18048 > Project: Hive > Issue Type: Improvement >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Major > Attachments: HIVE-18048.001.patch, HIVE-18048.002.patch, > HIVE-18048.003.patch, HIVE-18048.004.patch, HIVE-18048.005.patch, > HIVE-18048.006.patch > > > Struct type is not supported in MapWork with vectorization, it should be > supported to improve the performance. > New UDF will be added to access the field of Struct. > Note: > * Filter operator won't be supported. > * Nested complex type won't be tested in this ticket. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18048) Vectorization: Support Struct type with vectorization
[ https://issues.apache.org/jira/browse/HIVE-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HIVE-18048: Attachment: HIVE-18048.006.patch > Vectorization: Support Struct type with vectorization > - > > Key: HIVE-18048 > URL: https://issues.apache.org/jira/browse/HIVE-18048 > Project: Hive > Issue Type: Improvement >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Major > Attachments: HIVE-18048.001.patch, HIVE-18048.002.patch, > HIVE-18048.003.patch, HIVE-18048.004.patch, HIVE-18048.005.patch, > HIVE-18048.006.patch > > > Struct type is not supported in MapWork with vectorization, it should be > supported to improve the performance. > New UDF will be added to access the field of Struct. > Note: > * Filter operator won't be supported. > * Nested complex type won't be tested in this ticket. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18577) SemanticAnalyzer.validate has some pointless metastore calls
[
https://issues.apache.org/jira/browse/HIVE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344577#comment-16344577
]
Hive QA commented on HIVE-18577:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 1 new + 780 unchanged - 1
fixed = 781 total (was 781) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / be8a451 |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8920/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8920/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> SemanticAnalyzer.validate has some pointless metastore calls
>
>
> Key: HIVE-18577
> URL: https://issues.apache.org/jira/browse/HIVE-18577
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18577.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18579) Changes from HIVE-18495 introduced import paths from shaded jars
[
https://issues.apache.org/jira/browse/HIVE-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-18579:
Status: Patch Available (was: Open)
[~kgyrtkirk] Can you please take a look?
> Changes from HIVE-18495 introduced import paths from shaded jars
>
>
> Key: HIVE-18579
> URL: https://issues.apache.org/jira/browse/HIVE-18579
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Zoltan Haindrich
>Priority: Critical
> Attachments: HIVE-18579.patch
>
>
> When compiling the latest code after HIVE-18495 seeing the following issue:
> {noformat}
> -
> [ERROR] COMPILATION ERROR :
> -
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java:[29,64]
> package org.apache.hadoop.hbase.shaded.com.google.common.collect does not
> exist
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[43,68]
> package org.apache.hadoop.hbase.shaded.com.google.common.annotations does
> not exist
> ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[100,4]
> cannot find symbol
> symbol: class VisibleForTesting
> location: class org.apache.hive.testutils.MiniZooKeeperCluster
> [INFO] 3 errors
> {noformat}
> Seems like org.apache.hadoop.hbase.shaded.com.google.* is being used, I am
> guessing the idea was to use com.google.*. Not sure why we didn't see this
> failing in Apache Hive build system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18579) Changes from HIVE-18495 introduced import paths from shaded jars
[
https://issues.apache.org/jira/browse/HIVE-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-18579:
Attachment: HIVE-18579.patch
> Changes from HIVE-18495 introduced import paths from shaded jars
>
>
> Key: HIVE-18579
> URL: https://issues.apache.org/jira/browse/HIVE-18579
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Zoltan Haindrich
>Priority: Critical
> Attachments: HIVE-18579.patch
>
>
> When compiling the latest code after HIVE-18495 seeing the following issue:
> {noformat}
> -
> [ERROR] COMPILATION ERROR :
> -
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java:[29,64]
> package org.apache.hadoop.hbase.shaded.com.google.common.collect does not
> exist
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[43,68]
> package org.apache.hadoop.hbase.shaded.com.google.common.annotations does
> not exist
> ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[100,4]
> cannot find symbol
> symbol: class VisibleForTesting
> location: class org.apache.hive.testutils.MiniZooKeeperCluster
> [INFO] 3 errors
> {noformat}
> Seems like org.apache.hadoop.hbase.shaded.com.google.* is being used, I am
> guessing the idea was to use com.google.*. Not sure why we didn't see this
> failing in Apache Hive build system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18569) Hive Druid indexing not dealing with decimals in correct way.
[ https://issues.apache.org/jira/browse/HIVE-18569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344572#comment-16344572 ] Ashutosh Chauhan commented on HIVE-18569: - TestMiniDruidCliDriver.testCliDriver[druidmini_mv] may be related. +1 > Hive Druid indexing not dealing with decimals in correct way. > - > > Key: HIVE-18569 > URL: https://issues.apache.org/jira/browse/HIVE-18569 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-18569.patch > > > Currently, a decimal column is indexed as double in druid. > This should not happen and either the user has to add an explicit cast or we > can add a flag to enable approximation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18577) SemanticAnalyzer.validate has some pointless metastore calls
[ https://issues.apache.org/jira/browse/HIVE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344566#comment-16344566 ] Ashutosh Chauhan commented on HIVE-18577: - HIVE-11145 removed Protect mode because of same reason that this is expensive metadata call, but stopped short of removing corresponding logic which it should had. +1 > SemanticAnalyzer.validate has some pointless metastore calls > > > Key: HIVE-18577 > URL: https://issues.apache.org/jira/browse/HIVE-18577 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18577.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17552) Enable bucket map join by default
[
https://issues.apache.org/jira/browse/HIVE-17552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344560#comment-16344560
]
Hive QA commented on HIVE-17552:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908250/HIVE-17552.7.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 12794 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_onesideskew]
(batchId=72)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_5]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_9]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConcurrentStatements (batchId=238)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8919/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8919/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8919/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12908250 - PreCommit-HIVE-Build
> Enable bucket map join by default
> -
>
> Key: HIVE-17552
> URL: https://issues.apache.org/jira/browse/HIVE-17552
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-17552.1.patch, HIVE-17552.2.patch,
> HIVE-17552.3.patch, HIVE-17552.4.patch, HIVE-17552.5.patch,
> HIVE-17552.6.patch, HIVE-17552.7.patch
>
>
> Currently bucket map join is disabled by default, however, it is potentially
> most optimal join we have. Need to enable it by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently
[
https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344546#comment-16344546
]
Alexander Kolbasov commented on HIVE-16886:
---
[~anishek] I don't have access to MS SQL server - do you have something? The
test requires tables created for HMS (at least two notification tables, the
rest are irrelevant).
> HMS log notifications may have duplicated event IDs if multiple HMS are
> running concurrently
>
>
> Key: HIVE-16886
> URL: https://issues.apache.org/jira/browse/HIVE-16886
> Project: Hive
> Issue Type: Bug
> Components: Hive, Metastore
>Affects Versions: 3.0.0, 2.3.2, 2.3.3
>Reporter: Sergio Peña
>Assignee: anishek
>Priority: Major
> Labels: TODOC3.0
> Fix For: 3.0.0
>
> Attachments: HIVE-16886.1.patch, HIVE-16886.2.patch,
> HIVE-16886.3.patch, HIVE-16886.4.patch, HIVE-16886.5.patch,
> HIVE-16886.6.patch, HIVE-16886.7.patch, HIVE-16886.8.patch,
> datastore-identity-holes.diff
>
>
> When running multiple Hive Metastore servers and DB notifications are
> enabled, I could see that notifications can be persisted with a duplicated
> event ID.
> This does not happen when running multiple threads in a single HMS node due
> to the locking acquired on the DbNotificationsLog class, but multiple HMS
> could cause conflicts.
> The issue is in the ObjectStore#addNotificationEvent() method. The event ID
> fetched from the datastore is used for the new notification, incremented in
> the server itself, then persisted or updated back to the datastore. If 2
> servers read the same ID, then these 2 servers write a new notification with
> the same ID.
> The event ID is not unique nor a primary key.
> Here's a test case using the TestObjectStore class that confirms this issue:
> {noformat}
> @Test
> public void testConcurrentAddNotifications() throws ExecutionException,
> InterruptedException {
> final int NUM_THREADS = 2;
> CountDownLatch countIn = new CountDownLatch(NUM_THREADS);
> CountDownLatch countOut = new CountDownLatch(1);
> HiveConf conf = new HiveConf();
> conf.setVar(HiveConf.ConfVars.METASTORE_EXPRESSION_PROXY_CLASS,
> MockPartitionExpressionProxy.class.getName());
> ExecutorService executorService =
> Executors.newFixedThreadPool(NUM_THREADS);
> FutureTask tasks[] = new FutureTask[NUM_THREADS];
> for (int i=0; i final int n = i;
> tasks[i] = new FutureTask(new Callable() {
> @Override
> public Void call() throws Exception {
> ObjectStore store = new ObjectStore();
> store.setConf(conf);
> NotificationEvent dbEvent =
> new NotificationEvent(0, 0,
> EventMessage.EventType.CREATE_DATABASE.toString(), "CREATE DATABASE DB" + n);
> System.out.println("ADDING NOTIFICATION");
> countIn.countDown();
> countOut.await();
> store.addNotificationEvent(dbEvent);
> System.out.println("FINISH NOTIFICATION");
> return null;
> }
> });
> executorService.execute(tasks[i]);
> }
> countIn.await();
> countOut.countDown();
> for (int i = 0; i < NUM_THREADS; ++i) {
> tasks[i].get();
> }
> NotificationEventResponse eventResponse =
> objectStore.getNextNotification(new NotificationEventRequest());
> Assert.assertEquals(2, eventResponse.getEventsSize());
> Assert.assertEquals(1, eventResponse.getEvents().get(0).getEventId());
> // This fails because the next notification has an event ID = 1
> Assert.assertEquals(2, eventResponse.getEvents().get(1).getEventId());
> }
> {noformat}
> The last assertion fails expecting an event ID 1 instead of 2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18550) Keep the hbase table name property as hbase.table.name
[ https://issues.apache.org/jira/browse/HIVE-18550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344543#comment-16344543 ] Aihua Xu commented on HIVE-18550: - [~ychena] Thanks for reviewing. This HiveHFileOutputFormat is for hbase bulk loading. To read from the hbase, it can be handled by HiveHBaseTableInputFormat which takes care of the read already. > Keep the hbase table name property as hbase.table.name > -- > > Key: HIVE-18550 > URL: https://issues.apache.org/jira/browse/HIVE-18550 > Project: Hive > Issue Type: Sub-task > Components: HBase Handler >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > Attachments: HIVE-18550.1.patch > > > With hbase 2.0 support, I made some changes to the hbase table name property > change in HIVE-18366 and HIVE-18202. By checking the logic, seems the change > is not necessary since hbase.table.name is internal to hive hbase handler. We > just need to map hbase.table.name to > hbase.mapreduce.hfileoutputformat.table.name for HiveHFileOutputFormat. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18048) Vectorization: Support Struct type with vectorization
[ https://issues.apache.org/jira/browse/HIVE-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HIVE-18048: Attachment: (was: HIVE-18048.006.patch) > Vectorization: Support Struct type with vectorization > - > > Key: HIVE-18048 > URL: https://issues.apache.org/jira/browse/HIVE-18048 > Project: Hive > Issue Type: Improvement >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Major > Attachments: HIVE-18048.001.patch, HIVE-18048.002.patch, > HIVE-18048.003.patch, HIVE-18048.004.patch, HIVE-18048.005.patch > > > Struct type is not supported in MapWork with vectorization, it should be > supported to improve the performance. > New UDF will be added to access the field of Struct. > Note: > * Filter operator won't be supported. > * Nested complex type won't be tested in this ticket. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18048) Vectorization: Support Struct type with vectorization
[ https://issues.apache.org/jira/browse/HIVE-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HIVE-18048: Attachment: HIVE-18048.006.patch > Vectorization: Support Struct type with vectorization > - > > Key: HIVE-18048 > URL: https://issues.apache.org/jira/browse/HIVE-18048 > Project: Hive > Issue Type: Improvement >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Major > Attachments: HIVE-18048.001.patch, HIVE-18048.002.patch, > HIVE-18048.003.patch, HIVE-18048.004.patch, HIVE-18048.005.patch, > HIVE-18048.006.patch > > > Struct type is not supported in MapWork with vectorization, it should be > supported to improve the performance. > New UDF will be added to access the field of Struct. > Note: > * Filter operator won't be supported. > * Nested complex type won't be tested in this ticket. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18577) SemanticAnalyzer.validate has some pointless metastore calls
[ https://issues.apache.org/jira/browse/HIVE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344539#comment-16344539 ] Ashutosh Chauhan commented on HIVE-18577: - +1 > SemanticAnalyzer.validate has some pointless metastore calls > > > Key: HIVE-18577 > URL: https://issues.apache.org/jira/browse/HIVE-18577 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18577.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-16886) HMS log notifications may have duplicated event IDs if multiple HMS are running concurrently
[
https://issues.apache.org/jira/browse/HIVE-16886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344530#comment-16344530
]
anishek commented on HIVE-16886:
[~akolb] seems to work based on code as you suggest, if you could test this as
well against a mssqlserver that will be great, I am ok with reimplementing this
patch.
[~thejas] anything else we need to take care of ?
> HMS log notifications may have duplicated event IDs if multiple HMS are
> running concurrently
>
>
> Key: HIVE-16886
> URL: https://issues.apache.org/jira/browse/HIVE-16886
> Project: Hive
> Issue Type: Bug
> Components: Hive, Metastore
>Affects Versions: 3.0.0, 2.3.2, 2.3.3
>Reporter: Sergio Peña
>Assignee: anishek
>Priority: Major
> Labels: TODOC3.0
> Fix For: 3.0.0
>
> Attachments: HIVE-16886.1.patch, HIVE-16886.2.patch,
> HIVE-16886.3.patch, HIVE-16886.4.patch, HIVE-16886.5.patch,
> HIVE-16886.6.patch, HIVE-16886.7.patch, HIVE-16886.8.patch,
> datastore-identity-holes.diff
>
>
> When running multiple Hive Metastore servers and DB notifications are
> enabled, I could see that notifications can be persisted with a duplicated
> event ID.
> This does not happen when running multiple threads in a single HMS node due
> to the locking acquired on the DbNotificationsLog class, but multiple HMS
> could cause conflicts.
> The issue is in the ObjectStore#addNotificationEvent() method. The event ID
> fetched from the datastore is used for the new notification, incremented in
> the server itself, then persisted or updated back to the datastore. If 2
> servers read the same ID, then these 2 servers write a new notification with
> the same ID.
> The event ID is not unique nor a primary key.
> Here's a test case using the TestObjectStore class that confirms this issue:
> {noformat}
> @Test
> public void testConcurrentAddNotifications() throws ExecutionException,
> InterruptedException {
> final int NUM_THREADS = 2;
> CountDownLatch countIn = new CountDownLatch(NUM_THREADS);
> CountDownLatch countOut = new CountDownLatch(1);
> HiveConf conf = new HiveConf();
> conf.setVar(HiveConf.ConfVars.METASTORE_EXPRESSION_PROXY_CLASS,
> MockPartitionExpressionProxy.class.getName());
> ExecutorService executorService =
> Executors.newFixedThreadPool(NUM_THREADS);
> FutureTask tasks[] = new FutureTask[NUM_THREADS];
> for (int i=0; i final int n = i;
> tasks[i] = new FutureTask(new Callable() {
> @Override
> public Void call() throws Exception {
> ObjectStore store = new ObjectStore();
> store.setConf(conf);
> NotificationEvent dbEvent =
> new NotificationEvent(0, 0,
> EventMessage.EventType.CREATE_DATABASE.toString(), "CREATE DATABASE DB" + n);
> System.out.println("ADDING NOTIFICATION");
> countIn.countDown();
> countOut.await();
> store.addNotificationEvent(dbEvent);
> System.out.println("FINISH NOTIFICATION");
> return null;
> }
> });
> executorService.execute(tasks[i]);
> }
> countIn.await();
> countOut.countDown();
> for (int i = 0; i < NUM_THREADS; ++i) {
> tasks[i].get();
> }
> NotificationEventResponse eventResponse =
> objectStore.getNextNotification(new NotificationEventRequest());
> Assert.assertEquals(2, eventResponse.getEventsSize());
> Assert.assertEquals(1, eventResponse.getEvents().get(0).getEventId());
> // This fails because the next notification has an event ID = 1
> Assert.assertEquals(2, eventResponse.getEvents().get(1).getEventId());
> }
> {noformat}
> The last assertion fails expecting an event ID 1 instead of 2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17552) Enable bucket map join by default
[
https://issues.apache.org/jira/browse/HIVE-17552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344517#comment-16344517
]
Hive QA commented on HIVE-17552:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 9m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / be8a451 |
| Default Java | 1.8.0_111 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8919/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Enable bucket map join by default
> -
>
> Key: HIVE-17552
> URL: https://issues.apache.org/jira/browse/HIVE-17552
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Attachments: HIVE-17552.1.patch, HIVE-17552.2.patch,
> HIVE-17552.3.patch, HIVE-17552.4.patch, HIVE-17552.5.patch,
> HIVE-17552.6.patch, HIVE-17552.7.patch
>
>
> Currently bucket map join is disabled by default, however, it is potentially
> most optimal join we have. Need to enable it by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18048) Vectorization: Support Struct type with vectorization
[ https://issues.apache.org/jira/browse/HIVE-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HIVE-18048: Description: Struct type is not supported in MapWork with vectorization, it should be supported to improve the performance. New UDF will be added to access the field of Struct. Note: * Filter operator won't be supported. * Nested complex type won't be tested in this ticket. was: Struct type is not supported in MapWork with vectorization, it should be supported to improve the performance. New UDF will be added to access the field of Struct. Note: * Filter operator won't be supported. > Vectorization: Support Struct type with vectorization > - > > Key: HIVE-18048 > URL: https://issues.apache.org/jira/browse/HIVE-18048 > Project: Hive > Issue Type: Improvement >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Major > Attachments: HIVE-18048.001.patch, HIVE-18048.002.patch, > HIVE-18048.003.patch, HIVE-18048.004.patch, HIVE-18048.005.patch > > > Struct type is not supported in MapWork with vectorization, it should be > supported to improve the performance. > New UDF will be added to access the field of Struct. > Note: > * Filter operator won't be supported. > * Nested complex type won't be tested in this ticket. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[
https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344510#comment-16344510
]
Hive QA commented on HIVE-18552:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908249/HIVE-18552.1.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 30 failed/errored test(s), 12795 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_bmj_schema_evolution]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=163)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_view_failure6]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_view_failure6_2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_double_bigint]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_double_bigint_2]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_string_bigint]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[compare_string_bigint_2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input4]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input4_2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input_part0_neg]
(batchId=94)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[input_part0_neg_2]
(batchId=95)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8918/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8918/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8918/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 30 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12908249 - PreCommit-HIVE-Build
> Split hive.strict.checks.large.query into two configs
> -
>
> Key: HIVE-18552
> URL: https://issues.apache.org/jira/browse/HIVE-18552
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18552.1.patch
>
>
> {{hive.strict.checks.large.query}} controls the strict checks for restricting
> order bys with no limits, and scans of a partitioned table without a filter
> on the partition table.
> While both checks prevent "large" queries from being run, they both control
> very different behavior. It would be better if users could control these
> restrictions separately.
> Furthermore, many users make the mistake of abusing partitioned tables and
> often end up in a situation where they are running queries
[jira] [Updated] (HIVE-18516) load data should rename files consistent with insert statements for ACID Tables
[ https://issues.apache.org/jira/browse/HIVE-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-18516: -- Attachment: HIVE-18516.4.patch > load data should rename files consistent with insert statements for ACID > Tables > --- > > Key: HIVE-18516 > URL: https://issues.apache.org/jira/browse/HIVE-18516 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-18516.1.patch, HIVE-18516.2.patch, > HIVE-18516.3.patch, HIVE-18516.4.patch > > > h1. load data should rename files consistent with insert statements for ACID > Tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[
https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344475#comment-16344475
]
Hive QA commented on HIVE-18552:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / be8a451 |
| Default Java | 1.8.0_111 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8918/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Split hive.strict.checks.large.query into two configs
> -
>
> Key: HIVE-18552
> URL: https://issues.apache.org/jira/browse/HIVE-18552
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18552.1.patch
>
>
> {{hive.strict.checks.large.query}} controls the strict checks for restricting
> order bys with no limits, and scans of a partitioned table without a filter
> on the partition table.
> While both checks prevent "large" queries from being run, they both control
> very different behavior. It would be better if users could control these
> restrictions separately.
> Furthermore, many users make the mistake of abusing partitioned tables and
> often end up in a situation where they are running queries that are doing
> full-table scans of partitioned tables. This can lead to lots of issues for
> Hive - e.g. OOM issues because so many partitions are loaded in memory. So it
> would be good if we enabled this restriction by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18536) IOW + DP is broken for insert-only ACID
[
https://issues.apache.org/jira/browse/HIVE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344459#comment-16344459
]
Hive QA commented on HIVE-18536:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908241/HIVE-18536.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 12794 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[quotedid_basic]
(batchId=61)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8917/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8917/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8917/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12908241 - PreCommit-HIVE-Build
> IOW + DP is broken for insert-only ACID
> ---
>
> Key: HIVE-18536
> URL: https://issues.apache.org/jira/browse/HIVE-18536
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18536.01.patch, HIVE-18536.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18472) Beeline gives log4j warnings
[ https://issues.apache.org/jira/browse/HIVE-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344452#comment-16344452 ] Vihang Karajgaonkar commented on HIVE-18472: patch merged in master and branch-2. Thanks [~janulatha] for your contribution. > Beeline gives log4j warnings > > > Key: HIVE-18472 > URL: https://issues.apache.org/jira/browse/HIVE-18472 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Fix For: 3.0.0, 2.4.0 > > Attachments: HIVE-18472.1.patch, HIVE-18472.2.patch, > HIVE-18472.3.patch > > > Starting Beeline gives the following warnings multiple times: > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an > explanation. > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] > ERROR StatusLogger No log4j2 configuration file found. Using default > configuration: logging only errors to the console. Set system property > 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show > Log4j2 internal initialization logging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18472) Beeline gives log4j warnings
[ https://issues.apache.org/jira/browse/HIVE-18472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-18472: --- Resolution: Fixed Fix Version/s: 2.4.0 Status: Resolved (was: Patch Available) > Beeline gives log4j warnings > > > Key: HIVE-18472 > URL: https://issues.apache.org/jira/browse/HIVE-18472 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Fix For: 3.0.0, 2.4.0 > > Attachments: HIVE-18472.1.patch, HIVE-18472.2.patch, > HIVE-18472.3.patch > > > Starting Beeline gives the following warnings multiple times: > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an > explanation. > SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory] > ERROR StatusLogger No log4j2 configuration file found. Using default > configuration: logging only errors to the console. Set system property > 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show > Log4j2 internal initialization logging. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18571) stats issues for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1630#comment-1630 ] Sergey Shelukhin commented on HIVE-18571: - [~ekoifman] [~steveyeom2017] this one is for stats... first, it removes double aggregation of stats in case of insert overwrite. Then, it actually doesn't call populate quick stats for ACID tables anymore after create, because the logic there is completely bogus for ACID tables (# and size of the files). This may affect some existing out file stats for ACID. > stats issues for MM tables > -- > > Key: HIVE-18571 > URL: https://issues.apache.org/jira/browse/HIVE-18571 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18571.patch > > > There are multiple stats aggregation issues with MM tables. > Some simple stats are double counted and some stats (simple stats) are > invalid for ACID table dirs altogether. > I have a patch almost ready, need to fix some more stuff and clean up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18571) stats issues for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18571: Summary: stats issues for MM tables (was: stats issue for MM tables) > stats issues for MM tables > -- > > Key: HIVE-18571 > URL: https://issues.apache.org/jira/browse/HIVE-18571 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18571.patch > > > There are multiple stats aggregation issues with MM tables. > Some simple stats are double counted and some stats (simple stats) are > invalid for ACID table dirs altogether. > I have a patch almost ready, need to fix some more stuff and clean up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18571) stats issue for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18571: Attachment: HIVE-18571.patch > stats issue for MM tables > - > > Key: HIVE-18571 > URL: https://issues.apache.org/jira/browse/HIVE-18571 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18571.patch > > > There are multiple stats aggregation issues with MM tables. > Some simple stats are double counted and some stats (simple stats) are > invalid for ACID table dirs altogether. > I have a patch almost ready, need to fix some more stuff and clean up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18571) stats issue for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18571: Status: Patch Available (was: Open) > stats issue for MM tables > - > > Key: HIVE-18571 > URL: https://issues.apache.org/jira/browse/HIVE-18571 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18571.patch > > > There are multiple stats aggregation issues with MM tables. > Some simple stats are double counted and some stats (simple stats) are > invalid for ACID table dirs altogether. > I have a patch almost ready, need to fix some more stuff and clean up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18579) Changes from HIVE-18495 introduced import paths from shaded jars
[
https://issues.apache.org/jira/browse/HIVE-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deepesh Khandelwal reassigned HIVE-18579:
-
> Changes from HIVE-18495 introduced import paths from shaded jars
>
>
> Key: HIVE-18579
> URL: https://issues.apache.org/jira/browse/HIVE-18579
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.0.0
>Reporter: Deepesh Khandelwal
>Assignee: Zoltan Haindrich
>Priority: Critical
>
> When compiling the latest code after HIVE-18495 seeing the following issue:
> {noformat}
> -
> [ERROR] COMPILATION ERROR :
> -
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java:[29,64]
> package org.apache.hadoop.hbase.shaded.com.google.common.collect does not
> exist
> [ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[43,68]
> package org.apache.hadoop.hbase.shaded.com.google.common.annotations does
> not exist
> ERROR]
> /grid/0/jenkins/workspace/Zuul_HDP_Build_Job/build-support/SOURCES/hive/ql/src/test/org/apache/hive/testutils/MiniZooKeeperCluster.java:[100,4]
> cannot find symbol
> symbol: class VisibleForTesting
> location: class org.apache.hive.testutils.MiniZooKeeperCluster
> [INFO] 3 errors
> {noformat}
> Seems like org.apache.hadoop.hbase.shaded.com.google.* is being used, I am
> guessing the idea was to use com.google.*. Not sure why we didn't see this
> failing in Apache Hive build system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18578) Some class has missed the ASF header
[ https://issues.apache.org/jira/browse/HIVE-18578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang updated HIVE-18578: Status: Patch Available (was: Open) > Some class has missed the ASF header > > > Key: HIVE-18578 > URL: https://issues.apache.org/jira/browse/HIVE-18578 > Project: Hive > Issue Type: Bug >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-18578.1.patch > > > Some class has missed the ASF header -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18578) Some class has missed the ASF header
[ https://issues.apache.org/jira/browse/HIVE-18578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang updated HIVE-18578: Attachment: HIVE-18578.1.patch > Some class has missed the ASF header > > > Key: HIVE-18578 > URL: https://issues.apache.org/jira/browse/HIVE-18578 > Project: Hive > Issue Type: Bug >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-18578.1.patch > > > Some class has missed the ASF header -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18578) Some class has missed the ASF header
[ https://issues.apache.org/jira/browse/HIVE-18578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang updated HIVE-18578: Attachment: (was: HIVE-18578.1.patch) > Some class has missed the ASF header > > > Key: HIVE-18578 > URL: https://issues.apache.org/jira/browse/HIVE-18578 > Project: Hive > Issue Type: Bug >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > Fix For: 3.0.0 > > > Some class has missed the ASF header -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18578) Some class has missed the ASF header
[ https://issues.apache.org/jira/browse/HIVE-18578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang updated HIVE-18578: Attachment: HIVE-18578.1.patch > Some class has missed the ASF header > > > Key: HIVE-18578 > URL: https://issues.apache.org/jira/browse/HIVE-18578 > Project: Hive > Issue Type: Bug >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-18578.1.patch > > > Some class has missed the ASF header -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18578) Some class has missed the ASF header
[ https://issues.apache.org/jira/browse/HIVE-18578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saijin Huang reassigned HIVE-18578: --- > Some class has missed the ASF header > > > Key: HIVE-18578 > URL: https://issues.apache.org/jira/browse/HIVE-18578 > Project: Hive > Issue Type: Bug >Reporter: Saijin Huang >Assignee: Saijin Huang >Priority: Trivial > Fix For: 3.0.0 > > > Some class has missed the ASF header -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18048) Vectorization: Support Struct type with vectorization
[ https://issues.apache.org/jira/browse/HIVE-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HIVE-18048: Issue Type: Improvement (was: Sub-task) Parent: (was: HIVE-14826) > Vectorization: Support Struct type with vectorization > - > > Key: HIVE-18048 > URL: https://issues.apache.org/jira/browse/HIVE-18048 > Project: Hive > Issue Type: Improvement >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Major > Attachments: HIVE-18048.001.patch, HIVE-18048.002.patch, > HIVE-18048.003.patch, HIVE-18048.004.patch, HIVE-18048.005.patch > > > Struct type is not supported in MapWork with vectorization, it should be > supported to improve the performance. > New UDF will be added to access the field of Struct. > Note: > * Filter operator won't be supported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18048) Vectorization: Support Struct type with vectorization
[ https://issues.apache.org/jira/browse/HIVE-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HIVE-18048: Summary: Vectorization: Support Struct type with vectorization (was: Support Struct type with vectorization for Parquet file) > Vectorization: Support Struct type with vectorization > - > > Key: HIVE-18048 > URL: https://issues.apache.org/jira/browse/HIVE-18048 > Project: Hive > Issue Type: Sub-task >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Major > Attachments: HIVE-18048.001.patch, HIVE-18048.002.patch, > HIVE-18048.003.patch, HIVE-18048.004.patch, HIVE-18048.005.patch > > > Struct type is not supported in MapWork with vectorization, it should be > supported to improve the performance. > New UDF will be added to access the field of Struct. > Note: > * Filter operator won't be supported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18577) SemanticAnalyzer.validate has some pointless metastore calls
[ https://issues.apache.org/jira/browse/HIVE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18577: Status: Patch Available (was: Open) [~ashutoshc] can you take a look? these metastore calls look completely useless to me cause the code just above seems to access partition objects from the WriteEntity just fine. > SemanticAnalyzer.validate has some pointless metastore calls > > > Key: HIVE-18577 > URL: https://issues.apache.org/jira/browse/HIVE-18577 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18577.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18577) SemanticAnalyzer.validate has some pointless metastore calls
[ https://issues.apache.org/jira/browse/HIVE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18577: Attachment: HIVE-18577.patch > SemanticAnalyzer.validate has some pointless metastore calls > > > Key: HIVE-18577 > URL: https://issues.apache.org/jira/browse/HIVE-18577 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18577.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18536) IOW + DP is broken for insert-only ACID
[
https://issues.apache.org/jira/browse/HIVE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344429#comment-16344429
]
Hive QA commented on HIVE-18536:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} common: The patch generated 1 new + 6 unchanged - 1
fixed = 7 total (was 7) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 3 new + 359 unchanged - 3
fixed = 362 total (was 362) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 90f26bb |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8917/yetus/diff-checkstyle-common.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8917/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8917/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8917/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> IOW + DP is broken for insert-only ACID
> ---
>
> Key: HIVE-18536
> URL: https://issues.apache.org/jira/browse/HIVE-18536
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18536.01.patch, HIVE-18536.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18577) SemanticAnalyzer.validate has some pointless metastore calls
[ https://issues.apache.org/jira/browse/HIVE-18577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18577: --- > SemanticAnalyzer.validate has some pointless metastore calls > > > Key: HIVE-18577 > URL: https://issues.apache.org/jira/browse/HIVE-18577 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18449) Add configurable policy for choosing the HMS URI from hive.metastore.uris
[
https://issues.apache.org/jira/browse/HIVE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344422#comment-16344422
]
Vihang Karajgaonkar commented on HIVE-18449:
Thanks for the patch [~janulatha] I left a comment on the RB. Can you please
take a look? Thanks
> Add configurable policy for choosing the HMS URI from hive.metastore.uris
> -
>
> Key: HIVE-18449
> URL: https://issues.apache.org/jira/browse/HIVE-18449
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-18449.1.patch
>
>
> HIVE-10815 added logic to randomly choose a HMS URI from
> {{hive.metastore.uris}}. It would be nice if there was a configurable policy
> that determined how a URI is chosen from this list - e.g. one option can be
> to randomly pick a URI, another option can be to choose the first URI in the
> list (which was the behavior prior to HIVE-10815).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18516) load data should rename files consistent with insert statements for ACID Tables
[
https://issues.apache.org/jira/browse/HIVE-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344418#comment-16344418
]
Hive QA commented on HIVE-18516:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908221/HIVE-18516.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8916/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8916/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8916/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-01-30 02:31:32.455
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8916/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-01-30 02:31:32.458
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
455b2cc..6dbec04 branch-2 -> origin/branch-2
+ git reset --hard HEAD
HEAD is now at 90f26bb HIVE-18426: Memory leak in RoutingAppender for every
hive operation (kalyan kumar kalvagadda, reviewed by Aihua Xu)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 90f26bb HIVE-18426: Memory leak in RoutingAppender for every
hive operation (kalyan kumar kalvagadda, reviewed by Aihua Xu)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-01-30 02:31:35.708
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8916/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/itests/src/test/resources/testconfiguration.properties: does not exist
in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java:
does not exist in index
error: a/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveCopyFiles.java:
does not exist in index
error: a/ql/src/test/queries/clientpositive/smb_mapjoin_7.q: does not exist in
index
error: a/ql/src/test/results/clientpositive/beeline/smb_mapjoin_7.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/smb_mapjoin_7.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/spark/smb_mapjoin_7.q.out: does not
exist in index
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: protoc version: 250, detected platform: linux/amd64
protoc-jar: executing: [/tmp/protoc7530939041038954533.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
log4j:WARN No appenders could be found for logger
[jira] [Commented] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[
https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344414#comment-16344414
]
Hive QA commented on HIVE-18575:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908220/HIVE-18575.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 12794 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=104)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=104)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill
(batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8915/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8915/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8915/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 24 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12908220 - PreCommit-HIVE-Build
> ACID properties usage in jobconf is ambiguous for MM tables
> ---
>
> Key: HIVE-18575
> URL: https://issues.apache.org/jira/browse/HIVE-18575
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18575.patch
>
>
> Vectorization checks for ACID table trigger for MM tables where they don't
> apply. Other places seem to set the setting for transactional case while most
> of the code seems to assume it implies full acid.
> Overall, many places in the code use the settings directly or set the ACID
> flag without setting the ACID properties.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17552) Enable bucket map join by default
[ https://issues.apache.org/jira/browse/HIVE-17552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-17552: -- Attachment: HIVE-17552.7.patch > Enable bucket map join by default > - > > Key: HIVE-17552 > URL: https://issues.apache.org/jira/browse/HIVE-17552 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-17552.1.patch, HIVE-17552.2.patch, > HIVE-17552.3.patch, HIVE-17552.4.patch, HIVE-17552.5.patch, > HIVE-17552.6.patch, HIVE-17552.7.patch > > > Currently bucket map join is disabled by default, however, it is potentially > most optimal join we have. Need to enable it by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[
https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18552:
Status: Patch Available (was: Open)
> Split hive.strict.checks.large.query into two configs
> -
>
> Key: HIVE-18552
> URL: https://issues.apache.org/jira/browse/HIVE-18552
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18552.1.patch
>
>
> {{hive.strict.checks.large.query}} controls the strict checks for restricting
> order bys with no limits, and scans of a partitioned table without a filter
> on the partition table.
> While both checks prevent "large" queries from being run, they both control
> very different behavior. It would be better if users could control these
> restrictions separately.
> Furthermore, many users make the mistake of abusing partitioned tables and
> often end up in a situation where they are running queries that are doing
> full-table scans of partitioned tables. This can lead to lots of issues for
> Hive - e.g. OOM issues because so many partitions are loaded in memory. So it
> would be good if we enabled this restriction by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[
https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18552:
Attachment: HIVE-18552.1.patch
> Split hive.strict.checks.large.query into two configs
> -
>
> Key: HIVE-18552
> URL: https://issues.apache.org/jira/browse/HIVE-18552
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
> Attachments: HIVE-18552.1.patch
>
>
> {{hive.strict.checks.large.query}} controls the strict checks for restricting
> order bys with no limits, and scans of a partitioned table without a filter
> on the partition table.
> While both checks prevent "large" queries from being run, they both control
> very different behavior. It would be better if users could control these
> restrictions separately.
> Furthermore, many users make the mistake of abusing partitioned tables and
> often end up in a situation where they are running queries that are doing
> full-table scans of partitioned tables. This can lead to lots of issues for
> Hive - e.g. OOM issues because so many partitions are loaded in memory. So it
> would be good if we enabled this restriction by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18552) Split hive.strict.checks.large.query into two configs
[
https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18552:
Summary: Split hive.strict.checks.large.query into two configs (was: Split
hive.strict.checks.large.query into two configs, enable partition filter check
by default)
> Split hive.strict.checks.large.query into two configs
> -
>
> Key: HIVE-18552
> URL: https://issues.apache.org/jira/browse/HIVE-18552
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> {{hive.strict.checks.large.query}} controls the strict checks for restricting
> order bys with no limits, and scans of a partitioned table without a filter
> on the partition table.
> While both checks prevent "large" queries from being run, they both control
> very different behavior. It would be better if users could control these
> restrictions separately.
> Furthermore, many users make the mistake of abusing partitioned tables and
> often end up in a situation where they are running queries that are doing
> full-table scans of partitioned tables. This can lead to lots of issues for
> Hive - e.g. OOM issues because so many partitions are loaded in memory. So it
> would be good if we enabled this restriction by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18237) missing results for insert_only table after DP insert
[
https://issues.apache.org/jira/browse/HIVE-18237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344377#comment-16344377
]
Sergey Shelukhin commented on HIVE-18237:
-
+1
This will be modified partially by HIVE-18536, I can rebase later
> missing results for insert_only table after DP insert
> -
>
> Key: HIVE-18237
> URL: https://issues.apache.org/jira/browse/HIVE-18237
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Zoltan Haindrich
>Assignee: Steve Yeom
>Priority: Major
> Attachments: HIVE-18237.01.patch, HIVE-18237.02.patch,
> HIVE-18237.03.patch, HIVE-18237.04.patch
>
>
> {code}
> set hive.stats.column.autogather=false;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.max.dynamic.partitions.pernode=200;
> set hive.exec.max.dynamic.partitions=200;
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> create table i0 (p int,v int);
> insert into i0 values
> (0,0),
> (2,2),
> (3,3);
> create table p0 (v int) partitioned by (p int) stored as orc
> tblproperties ("transactional"="true",
> "transactional_properties"="insert_only");
> explain insert overwrite table p0 partition (p) select * from i0 where v < 3;
> insert overwrite table p0 partition (p) select * from i0 where v < 3;
> select count(*) from p0 where v!=1;
> {code}
> The table p0 should contain {{2}} rows at this point; but the result is {{0}}.
> * seems to be specific to insert_only tables
> * the existing data appears if an {{insert into}} is executed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[
https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344375#comment-16344375
]
Hive QA commented on HIVE-18575:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
48s{color} | {color:red} ql: The patch generated 7 new + 1205 unchanged - 9
fixed = 1212 total (was 1214) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch llap-server passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch core passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} hcatalog/streaming: The patch generated 0 new + 258
unchanged - 2 fixed = 258 total (was 260) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 90f26bb |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8915/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql llap-server hcatalog/core hcatalog/streaming
itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8915/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID properties usage in jobconf is ambiguous for MM tables
> ---
>
> Key: HIVE-18575
> URL: https://issues.apache.org/jira/browse/HIVE-18575
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18575.patch
>
>
> Vectorization checks for ACID table trigger for MM tables where they don't
> apply. Other places seem to set the setting for
[jira] [Commented] (HIVE-18552) Split hive.strict.checks.large.query into two configs, enable partition filter check by default
[
https://issues.apache.org/jira/browse/HIVE-18552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344366#comment-16344366
]
Sahil Takiar commented on HIVE-18552:
-
[~sershe] sounds reasonable. Didn't realize there was an existing config to
control the max # of partitions to scan. Will post a patch that splits
{{hive.strict.checks.large.query}} and keeps both the new ones false by default.
> Split hive.strict.checks.large.query into two configs, enable partition
> filter check by default
> ---
>
> Key: HIVE-18552
> URL: https://issues.apache.org/jira/browse/HIVE-18552
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Sahil Takiar
>Assignee: Sahil Takiar
>Priority: Major
>
> {{hive.strict.checks.large.query}} controls the strict checks for restricting
> order bys with no limits, and scans of a partitioned table without a filter
> on the partition table.
> While both checks prevent "large" queries from being run, they both control
> very different behavior. It would be better if users could control these
> restrictions separately.
> Furthermore, many users make the mistake of abusing partitioned tables and
> often end up in a situation where they are running queries that are doing
> full-table scans of partitioned tables. This can lead to lots of issues for
> Hive - e.g. OOM issues because so many partitions are loaded in memory. So it
> would be good if we enabled this restriction by default.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18576) Support to read nested complex type with Parquet in vectorization mode
[ https://issues.apache.org/jira/browse/HIVE-18576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma reassigned HIVE-18576: --- > Support to read nested complex type with Parquet in vectorization mode > -- > > Key: HIVE-18576 > URL: https://issues.apache.org/jira/browse/HIVE-18576 > Project: Hive > Issue Type: Sub-task >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Major > > Nested complex type is common used, eg: Struct, s2 > List>. Currently, nested complex type can't be parsed in vectorization > mode, this ticket is target to support it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18536) IOW + DP is broken for insert-only ACID
[ https://issues.apache.org/jira/browse/HIVE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344354#comment-16344354 ] Eugene Koifman commented on HIVE-18536: --- I imagine AcidUtils can just as well be in common > IOW + DP is broken for insert-only ACID > --- > > Key: HIVE-18536 > URL: https://issues.apache.org/jira/browse/HIVE-18536 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18536.01.patch, HIVE-18536.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18048) Support Struct type with vectorization for Parquet file
[ https://issues.apache.org/jira/browse/HIVE-18048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344350#comment-16344350 ] Colin Ma commented on HIVE-18048: - [~Ferd], the patch is updated as your comments and fix the related tests. The failed tests are not patch related. > Support Struct type with vectorization for Parquet file > --- > > Key: HIVE-18048 > URL: https://issues.apache.org/jira/browse/HIVE-18048 > Project: Hive > Issue Type: Sub-task >Reporter: Colin Ma >Assignee: Colin Ma >Priority: Major > Attachments: HIVE-18048.001.patch, HIVE-18048.002.patch, > HIVE-18048.003.patch, HIVE-18048.004.patch, HIVE-18048.005.patch > > > Struct type is not supported in MapWork with vectorization, it should be > supported to improve the performance. > New UDF will be added to access the field of Struct. > Note: > * Filter operator won't be supported. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18536) IOW + DP is broken for insert-only ACID
[ https://issues.apache.org/jira/browse/HIVE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344345#comment-16344345 ] Sergey Shelukhin commented on HIVE-18536: - Fixed the comment and whitespace. I think the reason things are parsed in JavaUtils/etc. is that they were/are accessed in places like metastore that don't have access to QL. Is there reason AcidUtils cannot be moved mostly to common, for all the properties, coniguration, etc. stuff? > IOW + DP is broken for insert-only ACID > --- > > Key: HIVE-18536 > URL: https://issues.apache.org/jira/browse/HIVE-18536 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18536.01.patch, HIVE-18536.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344344#comment-16344344
]
Hive QA commented on HIVE-18574:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908214/HIVE-18574.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 12794 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[0]
(batchId=193)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerSlowQueryExecutionTime
(batchId=235)
org.apache.hive.jdbc.TestTriggersNoTezSessionPool.testTriggerTotalLaunchedTasks
(batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8914/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8914/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8914/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 25 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12908214 - PreCommit-HIVE-Build
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18536) IOW + DP is broken for insert-only ACID
[ https://issues.apache.org/jira/browse/HIVE-18536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18536: Attachment: HIVE-18536.01.patch > IOW + DP is broken for insert-only ACID > --- > > Key: HIVE-18536 > URL: https://issues.apache.org/jira/browse/HIVE-18536 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18536.01.patch, HIVE-18536.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344303#comment-16344303
]
Hive QA commented on HIVE-18574:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
21s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 5m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 41m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 90f26bb |
| Default Java | 1.8.0_111 |
| modules | C: llap-server . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8914/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18573) Use proper Calcite operator instead of UDFs
[
https://issues.apache.org/jira/browse/HIVE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344223#comment-16344223
]
Hive QA commented on HIVE-18573:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908211/HIVE-18573.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 223 failed/errored test(s), 12793 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=240)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[allcolref_in_udf]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_7]
(batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ba_table_udfs]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_1]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_5]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_8]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_gby2_map_multi_distinct]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_udf_percentile2]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_udf_percentile]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[char_udf1] (batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[concat_op] (batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_3]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constantfolding]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_colname]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[date_udf] (batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_udf] (batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_basic2]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_timeseries]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[druid_topn] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[extract] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[floor_time] (batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby2_map]
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby2_map_multi_distinct]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby2_map_skew]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby2_noskew]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby2_noskew_multi_distinct]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_map_ppr]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_map_ppr_multi_distinct]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_ppr] (batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_ppr_multi_distinct]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert0] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insertoverwrite_bucket]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_2] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_alt] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_udf]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[localtimezone]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[macro_1] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[materialized_view_rewrite_ssb_2]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonmr_fetch_threshold]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_dictionary_threshold]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ctas]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_join]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_map_type_vectorization]
(batchId=84)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_10]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_11]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_12]
(batchId=23)
[jira] [Updated] (HIVE-18426) Memory leak in RoutingAppender for every hive operation
[ https://issues.apache.org/jira/browse/HIVE-18426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aihua Xu updated HIVE-18426: Resolution: Fixed Status: Resolved (was: Patch Available) Committed to master branch. Thanks [~kkalyan] > Memory leak in RoutingAppender for every hive operation > --- > > Key: HIVE-18426 > URL: https://issues.apache.org/jira/browse/HIVE-18426 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 3.0.0 >Reporter: kalyan kumar kalvagadda >Assignee: kalyan kumar kalvagadda >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18426.001.patch, HIVE-18426.002.patch, > HIVE-18426.003.patch, HIVE-18426.004.patch, HIVE-18426.005.patch, > HIVE-18426.006.patch, Screen Shot 2018-01-10 at 9.17.19 AM.png > > > Each new operation creates new entry in the ConcurrentMap in RoutingAppender > but when the operation ends, AppenderControl stored in the map is retrieved > and stopped but the entry in ConcurrentMap is never cleaned up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344172#comment-16344172 ] Sergey Shelukhin commented on HIVE-18575: - [~ekoifman] well, to avoid future ambiguity I'm making sure it's named explicitly and that properties must always be set (which they were anyway in all but one place). > ACID properties usage in jobconf is ambiguous for MM tables > --- > > Key: HIVE-18575 > URL: https://issues.apache.org/jira/browse/HIVE-18575 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18575.patch > > > Vectorization checks for ACID table trigger for MM tables where they don't > apply. Other places seem to set the setting for transactional case while most > of the code seems to assume it implies full acid. > Overall, many places in the code use the settings directly or set the ACID > flag without setting the ACID properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344170#comment-16344170 ] Sergey Shelukhin commented on HIVE-18575: - Hmm, it took a while to post an RB > ACID properties usage in jobconf is ambiguous for MM tables > --- > > Key: HIVE-18575 > URL: https://issues.apache.org/jira/browse/HIVE-18575 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18575.patch > > > Vectorization checks for ACID table trigger for MM tables where they don't > apply. Other places seem to set the setting for transactional case while most > of the code seems to assume it implies full acid. > Overall, many places in the code use the settings directly or set the ACID > flag without setting the ACID properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18516) load data should rename files consistent with insert statements for ACID Tables
[ https://issues.apache.org/jira/browse/HIVE-18516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Deepak Jaiswal updated HIVE-18516: -- Attachment: HIVE-18516.3.patch > load data should rename files consistent with insert statements for ACID > Tables > --- > > Key: HIVE-18516 > URL: https://issues.apache.org/jira/browse/HIVE-18516 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-18516.1.patch, HIVE-18516.2.patch, > HIVE-18516.3.patch > > > h1. load data should rename files consistent with insert statements for ACID > Tables. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18573) Use proper Calcite operator instead of UDFs
[
https://issues.apache.org/jira/browse/HIVE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344162#comment-16344162
]
Hive QA commented on HIVE-18573:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 1 new + 62 unchanged - 0 fixed
= 63 total (was 62) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / d2a14fe |
| Default Java | 1.8.0_111 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8913/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8913/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Use proper Calcite operator instead of UDFs
> ---
>
> Key: HIVE-18573
> URL: https://issues.apache.org/jira/browse/HIVE-18573
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: slim bouguerra
>Priority: Major
> Attachments: HIVE-18573.patch
>
>
> Currently, Hive is mostly using user-defined black box sql operators during
> Query planning. It will be more beneficial to use proper calcite operators.
> Also, Use a single name for Extract operator instead of a different name for
> every Unit,
> Same for Floor function. This will allow unifying the treatment per operator.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18550) Keep the hbase table name property as hbase.table.name
[ https://issues.apache.org/jira/browse/HIVE-18550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344144#comment-16344144 ] Yongzhi Chen commented on HIVE-18550: - I only see the place that set hbase.mapreduce.hfileoutputformat.table.name value, do we need read from this property ? > Keep the hbase table name property as hbase.table.name > -- > > Key: HIVE-18550 > URL: https://issues.apache.org/jira/browse/HIVE-18550 > Project: Hive > Issue Type: Sub-task > Components: HBase Handler >Affects Versions: 3.0.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > Attachments: HIVE-18550.1.patch > > > With hbase 2.0 support, I made some changes to the hbase table name property > change in HIVE-18366 and HIVE-18202. By checking the logic, seems the change > is not necessary since hbase.table.name is internal to hive hbase handler. We > just need to map hbase.table.name to > hbase.mapreduce.hfileoutputformat.table.name for HiveHFileOutputFormat. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344141#comment-16344141
]
Gopal V commented on HIVE-18574:
The follow up item is ugly - step #1 rewrite Zookeeper with Netty4.
One is org.jboss.netty and the other is io.netty, they're not exactly
replacements for each other.
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18575: Status: Patch Available (was: Open) > ACID properties usage in jobconf is ambiguous for MM tables > --- > > Key: HIVE-18575 > URL: https://issues.apache.org/jira/browse/HIVE-18575 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18575.patch > > > Vectorization checks for ACID table trigger for MM tables where they don't > apply. Other places seem to set the setting for transactional case while most > of the code seems to assume it implies full acid. > Overall, many places in the code use the settings directly or set the ACID > flag without setting the ACID properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18575: Attachment: HIVE-18575.patch > ACID properties usage in jobconf is ambiguous for MM tables > --- > > Key: HIVE-18575 > URL: https://issues.apache.org/jira/browse/HIVE-18575 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-18575.patch > > > Vectorization checks for ACID table trigger for MM tables where they don't > apply. Other places seem to set the setting for transactional case while most > of the code seems to assume it implies full acid. > Overall, many places in the code use the settings directly or set the ACID > flag without setting the ACID properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18529) Vectorization: Add a debug config option to disable scratch column reuse
[ https://issues.apache.org/jira/browse/HIVE-18529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-18529: --- Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master, thanks [~mmccline] > Vectorization: Add a debug config option to disable scratch column reuse > > > Key: HIVE-18529 > URL: https://issues.apache.org/jira/browse/HIVE-18529 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 3.0.0, 2.3.2 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18529-branch-2.patch, HIVE-18529.1.patch, > HIVE-18529.2.patch > > > Debugging scratch column reuse is particularly painful and slow, adding a > config allows for this to be done without rebuilds. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344129#comment-16344129 ] Eugene Koifman edited comment on HIVE-18575 at 1/29/18 10:36 PM: - hive.acid.table.scan and TABLE_IS_TRANSACTIONAL = "transactional"; as in AcidUtils.isTablePropertyTransactional when set on Configuration were always meant for full acid and only full acid The code running on the Grid uses these to know it's doing acid I think is the part that got broken - it's now set where it shouldn't be was (Author: ekoifman): hive.acid.table.scan and TABLE_IS_TRANSACTIONAL = "transactional"; as in AcidUtils.isTablePropertyTransactional when set on Configuration were always meant for full acid and only full acid I think is the part that got broken - it's now set where it shouldn't be > ACID properties usage in jobconf is ambiguous for MM tables > --- > > Key: HIVE-18575 > URL: https://issues.apache.org/jira/browse/HIVE-18575 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > Vectorization checks for ACID table trigger for MM tables where they don't > apply. Other places seem to set the setting for transactional case while most > of the code seems to assume it implies full acid. > Overall, many places in the code use the settings directly or set the ACID > flag without setting the ACID properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344129#comment-16344129 ] Eugene Koifman commented on HIVE-18575: --- hive.acid.table.scan and TABLE_IS_TRANSACTIONAL = "transactional"; as in AcidUtils.isTablePropertyTransactional when set on Configuration were always meant for full acid and only full acid I think is the part that got broken - it's now set where it shouldn't be > ACID properties usage in jobconf is ambiguous for MM tables > --- > > Key: HIVE-18575 > URL: https://issues.apache.org/jira/browse/HIVE-18575 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > Vectorization checks for ACID table trigger for MM tables where they don't > apply. Other places seem to set the setting for transactional case while most > of the code seems to assume it implies full acid. > Overall, many places in the code use the settings directly or set the ACID > flag without setting the ACID properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344128#comment-16344128
]
Ashutosh Chauhan edited comment on HIVE-18574 at 1/29/18 10:35 PM:
---
+1 LGTM
But I think we shall avoid depending on 2 netty versions. Can you create a
follow-up Jira to get rid of netty3 dependency ?
was (Author: ashutoshc):
+1 LGTM
But I we shall avoid depending on 2 netty versions. Can you create a follow-up
Jira to get rid of netty3 dependency ?
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344128#comment-16344128
]
Ashutosh Chauhan commented on HIVE-18574:
-
+1 LGTM
But I we shall avoid depending on 2 netty versions. Can you create a follow-up
Jira to get rid of netty3 dependency ?
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18449) Add configurable policy for choosing the HMS URI from hive.metastore.uris
[
https://issues.apache.org/jira/browse/HIVE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344122#comment-16344122
]
Hive QA commented on HIVE-18449:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908205/HIVE-18449.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 21 failed/errored test(s), 12777 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=150)
[intersect_all.q,unionDistinct_1.q,orc_ppd_schema_evol_3a.q,table_nonprintable.q,tez_union_dynamic_partition.q,tez_union_dynamic_partition_2.q,temp_table_external.q,global_limit.q,llap_udf.q,schemeAuthority.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,parallel_colstats.q]
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=78)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesGetExists.testGetAllTablesCaseInsensitive[Embedded]
(batchId=206)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8912/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8912/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8912/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 21 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12908205 - PreCommit-HIVE-Build
> Add configurable policy for choosing the HMS URI from hive.metastore.uris
> -
>
> Key: HIVE-18449
> URL: https://issues.apache.org/jira/browse/HIVE-18449
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-18449.1.patch
>
>
> HIVE-10815 added logic to randomly choose a HMS URI from
> {{hive.metastore.uris}}. It would be nice if there was a configurable policy
> that determined how a URI is chosen from this list - e.g. one option can be
> to randomly pick a URI, another option can be to choose the first URI in the
> list (which was the behavior prior to HIVE-10815).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18575) ACID properties usage in jobconf is ambiguous for MM tables
[ https://issues.apache.org/jira/browse/HIVE-18575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-18575: --- > ACID properties usage in jobconf is ambiguous for MM tables > --- > > Key: HIVE-18575 > URL: https://issues.apache.org/jira/browse/HIVE-18575 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > Vectorization checks for ACID table trigger for MM tables where they don't > apply. Other places seem to set the setting for transactional case while most > of the code seems to assume it implies full acid. > Overall, many places in the code use the settings directly or set the ACID > flag without setting the ACID properties. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-18574:
---
Status: Patch Available (was: Open)
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-18574:
---
Attachment: HIVE-18574.1.patch
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-18574:
---
Attachment: (was: HIVE-18574.1.patch)
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V reassigned HIVE-18574:
--
Assignee: Gopal V
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18574) LLAP: Ship netty3 as part of LLAP install tarball
[
https://issues.apache.org/jira/browse/HIVE-18574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-18574:
---
Attachment: HIVE-18574.1.patch
> LLAP: Ship netty3 as part of LLAP install tarball
> -
>
> Key: HIVE-18574
> URL: https://issues.apache.org/jira/browse/HIVE-18574
> Project: Hive
> Issue Type: Bug
> Components: llap
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Priority: Major
> Attachments: HIVE-18574.1.patch
>
>
> Removing netty from Tez libs causes
> {code}
> 2018-01-29T18:28:49,995 ERROR [main ()]
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon: Failed to start LLAP
> Daemon with exception
> java.lang.NoClassDefFoundError: org/jboss/netty/channel/group/ChannelGroup
> at
> org.apache.hadoop.hive.llap.daemon.impl.LlapDaemon.serviceStart(LlapDaemon.java:410)
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18558) Upgrade orc version to 1.4.2
[ https://issues.apache.org/jira/browse/HIVE-18558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-18558: - Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Test failures are unrelated to this patch. Committed patch to master. > Upgrade orc version to 1.4.2 > > > Key: HIVE-18558 > URL: https://issues.apache.org/jira/browse/HIVE-18558 > Project: Hive > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18558.1.patch > > > Upgrade orc version to latest 1.4.2 release. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18526) Backport HIVE-16886 to Hive 2
[ https://issues.apache.org/jira/browse/HIVE-18526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344085#comment-16344085 ] Alexander Kolbasov commented on HIVE-18526: --- [~vihangk1] [~anishek] Thanks for pointing this out. I checked actual source for DataNucleus RDBMS adapter for MS SQL and seems that it is actually supported - I put some details in HIVE-16886. I agree that keeping similar patches between branch-2 and branch-3 is nice. In this case the original patch is quite involved and is difficult to use for backports and patches. [~anishek] What do you think about reimplementing your original patch using DataNucleus mechanism? > Backport HIVE-16886 to Hive 2 > - > > Key: HIVE-18526 > URL: https://issues.apache.org/jira/browse/HIVE-18526 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 2.3.3 >Reporter: Alexander Kolbasov >Assignee: Alexander Kolbasov >Priority: Major > Attachments: HIVE-18526.01-branch-2.patch, > HIVE-18526.02-branch-2.patch > > > The fix for HIVE-16886 isn't in Hive 2. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18459) hive-exec.jar leaks contents fb303.jar into classpath
[ https://issues.apache.org/jira/browse/HIVE-18459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344073#comment-16344073 ] Aihua Xu commented on HIVE-18459: - +1. > hive-exec.jar leaks contents fb303.jar into classpath > - > > Key: HIVE-18459 > URL: https://issues.apache.org/jira/browse/HIVE-18459 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-18459.patch, HIVE-18459.patch, HIVE-18459.patch > > > thrift classes are now in the hive classpath in the hive-exec.jar > (HIVE-11553). This makes it hard to test with other versions of this library. > This library is already a declared dependency and is not required to be > included in the hive-exec.jar. > I am proposing that we not include these classes like we have done in the > past releases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18573) Use proper Calcite operator instead of UDFs
[ https://issues.apache.org/jira/browse/HIVE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-18573: -- Attachment: HIVE-18573.patch > Use proper Calcite operator instead of UDFs > --- > > Key: HIVE-18573 > URL: https://issues.apache.org/jira/browse/HIVE-18573 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: slim bouguerra >Priority: Major > Attachments: HIVE-18573.patch > > > Currently, Hive is mostly using user-defined black box sql operators during > Query planning. It will be more beneficial to use proper calcite operators. > Also, Use a single name for Extract operator instead of a different name for > every Unit, > Same for Floor function. This will allow unifying the treatment per operator. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18573) Use proper Calcite operator instead of UDFs
[ https://issues.apache.org/jira/browse/HIVE-18573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-18573: -- Status: Patch Available (was: Open) > Use proper Calcite operator instead of UDFs > --- > > Key: HIVE-18573 > URL: https://issues.apache.org/jira/browse/HIVE-18573 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: slim bouguerra >Priority: Major > > Currently, Hive is mostly using user-defined black box sql operators during > Query planning. It will be more beneficial to use proper calcite operators. > Also, Use a single name for Extract operator instead of a different name for > every Unit, > Same for Floor function. This will allow unifying the treatment per operator. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18449) Add configurable policy for choosing the HMS URI from hive.metastore.uris
[
https://issues.apache.org/jira/browse/HIVE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16344056#comment-16344056
]
Hive QA commented on HIVE-18449:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 6924b9c |
| Default Java | 1.8.0_111 |
| modules | C: common standalone-metastore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-8912/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add configurable policy for choosing the HMS URI from hive.metastore.uris
> -
>
> Key: HIVE-18449
> URL: https://issues.apache.org/jira/browse/HIVE-18449
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-18449.1.patch
>
>
> HIVE-10815 added logic to randomly choose a HMS URI from
> {{hive.metastore.uris}}. It would be nice if there was a configurable policy
> that determined how a URI is chosen from this list - e.g. one option can be
> to randomly pick a URI, another option can be to choose the first URI in the
> list (which was the behavior prior to HIVE-10815).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HIVE-18572) The record readers; InputFormat needs to be fixed for Tez as it generates 1 split
[
https://issues.apache.org/jira/browse/HIVE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
gurmukh singh resolved HIVE-18572.
--
Resolution: Invalid
Release Note: closing it as invalid, will create a new one with tests.
> The record readers; InputFormat needs to be fixed for Tez as it generates 1
> split
> -
>
> Key: HIVE-18572
> URL: https://issues.apache.org/jira/browse/HIVE-18572
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Affects Versions: 2.1.0
>Reporter: gurmukh singh
>Priority: Major
>
> The record reader needs to be fixed for tez, as it generates only 1 split due
> to the {color:#33}MRv2 CombineInputFormat broke that rule{color}.
> This has been fixed in MR but not Tez.
> == Issue
> Closing it as invalid, will create a new ticket with tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18572) The record readers; InputFormat needs to be fixed for Tez as it generates 1 split
[
https://issues.apache.org/jira/browse/HIVE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
gurmukh singh updated HIVE-18572:
-
Description:
The record reader needs to be fixed for tez, as it generates only 1 split due
to the {color:#33}MRv2 CombineInputFormat broke that rule{color}.
This has been fixed in MR but not Tez.
== Issue
Closing it as invalid, will create a new ticket with tests.
was:
The record reader needs to be fixed for tez, as it generates only 1 split due
to the {color:#33}MRv2 CombineInputFormat broke that rule{color}.
This has been fixed in MR but not Tez.
== Issue
I am seeing a strange behaviour in tez; it is seeing all data as a single split
under hive, where as MR see all 79 files. This is causing all the data to go to
a single map
TEZ Processing
INFO : Partition trusted.usage\{ds=20180126, periode=1200} stats:
[numFiles=1, numRows=79575067, totalSize=3.164.605.993,
rawDataSize=112439569671]
ELAPSED TIME: 1958.99 s
MR Processing
Partition trusted.usage\{ds=20180126, periode=1200} stats: [numFiles=79,
numRows=79575067, totalSize=3172280778, rawDataSize=112418416260]
ELAPSED TIME: 65 s
Log Tez
2018-01-29 16:50:04,825 [INFO] [InputInitializer \{Map 1} #0|#0]
|split.TezMapredSplitsGrouper|: Desired splits: 381 too large. Desired
splitLength: 8311476 Min splitLength: 50331648 New desired splits: 381 Final
desired splits: 381 All splits have localhost: false Total length: 19166265870
Original splits: 1
2018-01-29 16:50:04,825 [INFO] [InputInitializer \{Map 1} #0|#0]
|split.TezMapredSplitsGrouper|: Using original number of splits: 1 desired
splits: 381
2018-01-29 16:50:04,826 [INFO] [InputInitializer \{Map 1} #0|#0]
|tez.SplitGrouper|: Original split size is 1 grouped split size is 1, for
bucket: 1
2018-01-29 16:50:04,827 [INFO] [InputInitializer \{Map 1} #0|#0]
|tez.HiveSplitGenerator|: Number of grouped splits: 1
2018-01-29 16:50:04,846 [INFO] [InputInitializer \{Map 1} #0|#0]
|dag.RootInputInitializerManager|: Succeeded InputInitializer for Input: usage
on vertex vertex_1517207496169_0085_1_00 [Map 1]
2018-01-29 16:50:04,848 [INFO] [App Shared Pool - #0|#0] |impl.VertexImpl|:
Cannot init vertex: vertex_1517207496169_0085_1_00 [Map 1] numTasks: -1
numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers:
true
2018-01-29 16:50:04,848 [INFO] [App Shared Pool - #0|#0] |impl.VertexImpl|:
Got updated RootInputsSpecs: {usage=forAllWorkUnits=true, update=[1]}
2018-01-29 16:50:04,859 [INFO] [App Shared Pool - #0|#0] |impl.VertexImpl|:
Vertex vertex_1517207496169_0085_1_00 [Map 1] parallelism set to 1
As per discussion with Gopal Vijayaraghavan:
[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L494]
that line, right there MRv2 CombineInputFormat broke that rule, so the record
readers had to be fixed to handle it
[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java#L312]
> The record readers; InputFormat needs to be fixed for Tez as it generates 1
> split
> -
>
> Key: HIVE-18572
> URL: https://issues.apache.org/jira/browse/HIVE-18572
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Affects Versions: 2.1.0
>Reporter: gurmukh singh
>Priority: Major
>
> The record reader needs to be fixed for tez, as it generates only 1 split due
> to the {color:#33}MRv2 CombineInputFormat broke that rule{color}.
> This has been fixed in MR but not Tez.
> == Issue
> Closing it as invalid, will create a new ticket with tests.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18572) The record readers; InputFormat needs to be fixed for Tez as it generates 1 split
[
https://issues.apache.org/jira/browse/HIVE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
gurmukh singh updated HIVE-18572:
-
Description:
The record reader needs to be fixed for tez, as it generates only 1 split due
to the {color:#33}MRv2 CombineInputFormat broke that rule{color}.
This has been fixed in MR but not Tez.
== Issue
I am seeing a strange behaviour in tez; it is seeing all data as a single split
under hive, where as MR see all 79 files. This is causing all the data to go to
a single map
TEZ Processing
INFO : Partition trusted.usage\{ds=20180126, periode=1200} stats:
[numFiles=1, numRows=79575067, totalSize=3.164.605.993,
rawDataSize=112439569671]
ELAPSED TIME: 1958.99 s
MR Processing
Partition trusted.usage\{ds=20180126, periode=1200} stats: [numFiles=79,
numRows=79575067, totalSize=3172280778, rawDataSize=112418416260]
ELAPSED TIME: 65 s
Log Tez
2018-01-29 16:50:04,825 [INFO] [InputInitializer \{Map 1} #0|#0]
|split.TezMapredSplitsGrouper|: Desired splits: 381 too large. Desired
splitLength: 8311476 Min splitLength: 50331648 New desired splits: 381 Final
desired splits: 381 All splits have localhost: false Total length: 19166265870
Original splits: 1
2018-01-29 16:50:04,825 [INFO] [InputInitializer \{Map 1} #0|#0]
|split.TezMapredSplitsGrouper|: Using original number of splits: 1 desired
splits: 381
2018-01-29 16:50:04,826 [INFO] [InputInitializer \{Map 1} #0|#0]
|tez.SplitGrouper|: Original split size is 1 grouped split size is 1, for
bucket: 1
2018-01-29 16:50:04,827 [INFO] [InputInitializer \{Map 1} #0|#0]
|tez.HiveSplitGenerator|: Number of grouped splits: 1
2018-01-29 16:50:04,846 [INFO] [InputInitializer \{Map 1} #0|#0]
|dag.RootInputInitializerManager|: Succeeded InputInitializer for Input: usage
on vertex vertex_1517207496169_0085_1_00 [Map 1]
2018-01-29 16:50:04,848 [INFO] [App Shared Pool - #0|#0] |impl.VertexImpl|:
Cannot init vertex: vertex_1517207496169_0085_1_00 [Map 1] numTasks: -1
numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers:
true
2018-01-29 16:50:04,848 [INFO] [App Shared Pool - #0|#0] |impl.VertexImpl|:
Got updated RootInputsSpecs: {usage=forAllWorkUnits=true, update=[1]}
2018-01-29 16:50:04,859 [INFO] [App Shared Pool - #0|#0] |impl.VertexImpl|:
Vertex vertex_1517207496169_0085_1_00 [Map 1] parallelism set to 1
As per discussion with Gopal Vijayaraghavan:
[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L494]
that line, right there MRv2 CombineInputFormat broke that rule, so the record
readers had to be fixed to handle it
[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java#L312]
was:
The record reader needs to be fixed for tez, as it generates only 1 split due
to the {color:#33}MRv2 CombineInputFormat broke that rule{color}.
This has been fixed in MR but not Tez.
I am seeing a strange behaviour in tez; it is seeing all data as a single split
under hive, where as MR see all 79 files. This is causing all the data to go to
a single map
TEZ Processing
INFO : Partition trusted.usage\{ds=20180126, periode=1200} stats: [numFiles=1,
numRows=79575067, totalSize=3.164.605.993, rawDataSize=112439569671]
ELAPSED TIME: 1958.99 s
MR Processing
Partition trusted.usage\{ds=20180126, periode=1200} stats: [numFiles=79,
numRows=79575067, totalSize=3172280778, rawDataSize=112418416260]
ELAPSED TIME: 65 s
Log Tez
2018-01-29 16:50:04,825 [INFO] [InputInitializer \{Map 1} #0]
|split.TezMapredSplitsGrouper|: Desired splits: 381 too large. Desired
splitLength: 8311476 Min splitLength: 50331648 New desired splits: 381 Final
desired splits: 381 All splits have localhost: false Total length: 19166265870
Original splits: 1
2018-01-29 16:50:04,825 [INFO] [InputInitializer \{Map 1} #0]
|split.TezMapredSplitsGrouper|: Using original number of splits: 1 desired
splits: 381
2018-01-29 16:50:04,826 [INFO] [InputInitializer \{Map 1} #0]
|tez.SplitGrouper|: Original split size is 1 grouped split size is 1, for
bucket: 1
2018-01-29 16:50:04,827 [INFO] [InputInitializer \{Map 1} #0]
|tez.HiveSplitGenerator|: Number of grouped splits: 1
2018-01-29 16:50:04,846 [INFO] [InputInitializer \{Map 1} #0]
|dag.RootInputInitializerManager|: Succeeded InputInitializer for Input: usage
on vertex vertex_1517207496169_0085_1_00 [Map 1]
2018-01-29 16:50:04,848 [INFO] [App Shared Pool - #0] |impl.VertexImpl|: Cannot
init vertex: vertex_1517207496169_0085_1_00 [Map 1] numTasks: -1
numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers:
true
2018-01-29 16:50:04,848 [INFO] [App Shared Pool - #0] |impl.VertexImpl|: Got
updated RootInputsSpecs: \{usage=forAllWorkUnits=true, update=[1]}
2018-01-29 16:50:04,859 [INFO] [App Shared Pool - #0] |impl.VertexImpl|: Vertex
[jira] [Updated] (HIVE-18572) The record readers; InputFormat needs to be fixed for Tez as it generates 1 split
[
https://issues.apache.org/jira/browse/HIVE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
gurmukh singh updated HIVE-18572:
-
Component/s: Tez
> The record readers; InputFormat needs to be fixed for Tez as it generates 1
> split
> -
>
> Key: HIVE-18572
> URL: https://issues.apache.org/jira/browse/HIVE-18572
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Affects Versions: 2.1.0
>Reporter: gurmukh singh
>Priority: Major
>
> The record reader needs to be fixed for tez, as it generates only 1 split due
> to the {color:#33}MRv2 CombineInputFormat broke that rule{color}.
> This has been fixed in MR but not Tez.
> I am seeing a strange behaviour in tez; it is seeing all data as a single
> split under hive, where as MR see all 79 files. This is causing all the data
> to go to a single map
> TEZ Processing
> INFO : Partition trusted.usage\{ds=20180126, periode=1200} stats:
> [numFiles=1, numRows=79575067, totalSize=3.164.605.993,
> rawDataSize=112439569671]
> ELAPSED TIME: 1958.99 s
> MR Processing
> Partition trusted.usage\{ds=20180126, periode=1200} stats: [numFiles=79,
> numRows=79575067, totalSize=3172280778, rawDataSize=112418416260]
> ELAPSED TIME: 65 s
> Log Tez
> 2018-01-29 16:50:04,825 [INFO] [InputInitializer \{Map 1} #0]
> |split.TezMapredSplitsGrouper|: Desired splits: 381 too large. Desired
> splitLength: 8311476 Min splitLength: 50331648 New desired splits: 381 Final
> desired splits: 381 All splits have localhost: false Total length:
> 19166265870 Original splits: 1
> 2018-01-29 16:50:04,825 [INFO] [InputInitializer \{Map 1} #0]
> |split.TezMapredSplitsGrouper|: Using original number of splits: 1 desired
> splits: 381
> 2018-01-29 16:50:04,826 [INFO] [InputInitializer \{Map 1} #0]
> |tez.SplitGrouper|: Original split size is 1 grouped split size is 1, for
> bucket: 1
> 2018-01-29 16:50:04,827 [INFO] [InputInitializer \{Map 1} #0]
> |tez.HiveSplitGenerator|: Number of grouped splits: 1
> 2018-01-29 16:50:04,846 [INFO] [InputInitializer \{Map 1} #0]
> |dag.RootInputInitializerManager|: Succeeded InputInitializer for Input:
> usage on vertex vertex_1517207496169_0085_1_00 [Map 1]
> 2018-01-29 16:50:04,848 [INFO] [App Shared Pool - #0] |impl.VertexImpl|:
> Cannot init vertex: vertex_1517207496169_0085_1_00 [Map 1] numTasks: -1
> numUnitializedEdges: 0 numInitializedInputs: 1 initWaitsForRootInitializers:
> true
> 2018-01-29 16:50:04,848 [INFO] [App Shared Pool - #0] |impl.VertexImpl|: Got
> updated RootInputsSpecs: \{usage=forAllWorkUnits=true, update=[1]}
> 2018-01-29 16:50:04,859 [INFO] [App Shared Pool - #0] |impl.VertexImpl|:
> Vertex vertex_1517207496169_0085_1_00 [Map 1] parallelism set to 1
> As per discussion with Gopal Vijayaraghavan:
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L494
> that line, right there MRv2 CombineInputFormat broke that rule, so the
> record readers had to be fixed to handle it
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java#L312
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18449) Add configurable policy for choosing the HMS URI from hive.metastore.uris
[
https://issues.apache.org/jira/browse/HIVE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janaki Lahorani updated HIVE-18449:
---
Attachment: HIVE-18449.1.patch
> Add configurable policy for choosing the HMS URI from hive.metastore.uris
> -
>
> Key: HIVE-18449
> URL: https://issues.apache.org/jira/browse/HIVE-18449
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-18449.1.patch
>
>
> HIVE-10815 added logic to randomly choose a HMS URI from
> {{hive.metastore.uris}}. It would be nice if there was a configurable policy
> that determined how a URI is chosen from this list - e.g. one option can be
> to randomly pick a URI, another option can be to choose the first URI in the
> list (which was the behavior prior to HIVE-10815).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18449) Add configurable policy for choosing the HMS URI from hive.metastore.uris
[
https://issues.apache.org/jira/browse/HIVE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janaki Lahorani updated HIVE-18449:
---
Attachment: (was: HIVE-18449.1.patch)
> Add configurable policy for choosing the HMS URI from hive.metastore.uris
> -
>
> Key: HIVE-18449
> URL: https://issues.apache.org/jira/browse/HIVE-18449
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-18449.1.patch
>
>
> HIVE-10815 added logic to randomly choose a HMS URI from
> {{hive.metastore.uris}}. It would be nice if there was a configurable policy
> that determined how a URI is chosen from this list - e.g. one option can be
> to randomly pick a URI, another option can be to choose the first URI in the
> list (which was the behavior prior to HIVE-10815).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18449) Add configurable policy for choosing the HMS URI from hive.metastore.uris
[
https://issues.apache.org/jira/browse/HIVE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janaki Lahorani updated HIVE-18449:
---
Attachment: HIVE-18499.1.patch
> Add configurable policy for choosing the HMS URI from hive.metastore.uris
> -
>
> Key: HIVE-18449
> URL: https://issues.apache.org/jira/browse/HIVE-18449
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-18449.1.patch
>
>
> HIVE-10815 added logic to randomly choose a HMS URI from
> {{hive.metastore.uris}}. It would be nice if there was a configurable policy
> that determined how a URI is chosen from this list - e.g. one option can be
> to randomly pick a URI, another option can be to choose the first URI in the
> list (which was the behavior prior to HIVE-10815).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18449) Add configurable policy for choosing the HMS URI from hive.metastore.uris
[
https://issues.apache.org/jira/browse/HIVE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janaki Lahorani updated HIVE-18449:
---
Attachment: (was: HIVE-18499.1.patch)
> Add configurable policy for choosing the HMS URI from hive.metastore.uris
> -
>
> Key: HIVE-18449
> URL: https://issues.apache.org/jira/browse/HIVE-18449
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-18449.1.patch
>
>
> HIVE-10815 added logic to randomly choose a HMS URI from
> {{hive.metastore.uris}}. It would be nice if there was a configurable policy
> that determined how a URI is chosen from this list - e.g. one option can be
> to randomly pick a URI, another option can be to choose the first URI in the
> list (which was the behavior prior to HIVE-10815).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18449) Add configurable policy for choosing the HMS URI from hive.metastore.uris
[
https://issues.apache.org/jira/browse/HIVE-18449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janaki Lahorani updated HIVE-18449:
---
Status: Patch Available (was: Open)
> Add configurable policy for choosing the HMS URI from hive.metastore.uris
> -
>
> Key: HIVE-18449
> URL: https://issues.apache.org/jira/browse/HIVE-18449
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
>Reporter: Sahil Takiar
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-18449.1.patch
>
>
> HIVE-10815 added logic to randomly choose a HMS URI from
> {{hive.metastore.uris}}. It would be nice if there was a configurable policy
> that determined how a URI is chosen from this list - e.g. one option can be
> to randomly pick a URI, another option can be to choose the first URI in the
> list (which was the behavior prior to HIVE-10815).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
