[jira] [Commented] (HIVE-22995) Add support for location for managed tables on database
[
https://issues.apache.org/jira/browse/HIVE-22995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059988#comment-17059988
]
Hive QA commented on HIVE-22995:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996805/HIVE-22995.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 617 failed/errored test(s), 17910 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_custom_key2]
(batchId=298)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown]
(batchId=298)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=298)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=298)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[colstats_all_nulls]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[create_merge_compressed]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[desc_table_formatted]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[drop_with_concurrency]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[escape_comments]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[explain_outputs]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[insert_overwrite_local_directory_1]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[mapjoin2]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[select_dummy_source]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_10]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_11]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_12]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_13]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_16]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_1]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_2]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_3]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_7]
(batchId=306)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[udf_unix_timestamp]
(batchId=306)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[create_database]
(batchId=309)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_blobstore_to_hdfs]
(batchId=309)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[add_part_exist]
(batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter1] (batchId=100)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter2] (batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter3] (batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter4] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter5] (batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_change_db_location]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_char1] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_db_owner]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_coltype]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_onto_nocurrent_db]
(batchId=101)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_partition_update_status]
(batchId=104)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_rename_partition]
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_rename_table]
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_skewed_table]
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_column_stats]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_stats_status]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status_disable_bitvector]
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_varchar1]
(batchId=96)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_view_as_select]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_view_rename]
(bat
[jira] [Commented] (HIVE-23029) LLAP: Shuffle Handler should support Index Cache configuration
[ https://issues.apache.org/jira/browse/HIVE-23029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059987#comment-17059987 ] Rajesh Balamohan commented on HIVE-23029: - Ref: TEZ-3887 > LLAP: Shuffle Handler should support Index Cache configuration > -- > > Key: HIVE-23029 > URL: https://issues.apache.org/jira/browse/HIVE-23029 > Project: Hive > Issue Type: Improvement > Components: llap >Reporter: Rajesh Balamohan >Priority: Trivial > Attachments: Screenshot 2020-03-16 at 12.08.44 PM.jpg > > > !Screenshot 2020-03-16 at 12.08.44 PM.jpg|width=580,height=405! > > Queries like Q78 at large scale misses index cache with unordered edges. (24 > * 1009 = 24216. With the default 10 MB cache size, it can accommodate only > 400+ entries). > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403743&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403743
]
ASF GitHub Bot logged work on HIVE-22990:
-
Author: ASF GitHub Bot
Created on: 16/Mar/20 06:21
Start Date: 16/Mar/20 06:21
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on pull request #949: HIVE-22990
Add file based ack for replication
URL: https://github.com/apache/hive/pull/949#discussion_r392800355
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##
@@ -168,6 +185,42 @@ private Long getEventFromPreviousDumpMetadata(Path
dumpRoot) throws IOException,
return 0L;
}
+ private Path getPreviousDumpMetadataPath(Path dumpRoot) throws IOException {
+FileSystem fs = dumpRoot.getFileSystem(conf);
+if (fs.exists(dumpRoot)) {
+ FileStatus[] statuses = fs.listStatus(dumpRoot);
+ if (statuses.length > 0) {
+FileStatus latestUpdatedStatus = statuses[0];
+for (FileStatus status : statuses) {
+ if (status.getModificationTime() >
latestUpdatedStatus.getModificationTime()) {
+latestUpdatedStatus = status;
+ }
+}
+return latestUpdatedStatus.getPath();
+ }
+}
+return null;
+ }
+
+ private boolean shouldDump(Path previousDumpPath) throws IOException {
+//If no previous dump means bootstrap. So return true as there was no
+//previous dump to load
+if (previousDumpPath == null) {
+ return true;
+} else {
+ FileSystem fs = previousDumpPath.getFileSystem(conf);
+ if (fs.exists(previousDumpPath)) {
Review comment:
Should we just use return fs.exists(new Path(previousDumpPath,
ReplUtils.LOAD_ACKNOWLEDGEMENT));
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 403743)
Time Spent: 3h (was: 2h 50m)
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403742&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403742
]
ASF GitHub Bot logged work on HIVE-22990:
-
Author: ASF GitHub Bot
Created on: 16/Mar/20 06:21
Start Date: 16/Mar/20 06:21
Worklog Time Spent: 10m
Work Description: pkumarsinha commented on pull request #949: HIVE-22990
Add file based ack for replication
URL: https://github.com/apache/hive/pull/949#discussion_r392800936
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##
@@ -126,21 +126,30 @@ public int execute() {
Path dumpRoot = new Path(conf.getVar(HiveConf.ConfVars.REPLDIR),
Base64.getEncoder().encodeToString(work.dbNameOrPattern.toLowerCase()
.getBytes(StandardCharsets.UTF_8.name(;
- Path currentDumpPath = new Path(dumpRoot, getNextDumpDir());
- Path hiveDumpRoot = new Path(currentDumpPath,
ReplUtils.REPL_HIVE_BASE_DIR);
- DumpMetaData dmd = new DumpMetaData(hiveDumpRoot, conf);
- // Initialize ReplChangeManager instance since we will require it to
encode file URI.
- ReplChangeManager.getInstance(conf);
- Path cmRoot = new Path(conf.getVar(HiveConf.ConfVars.REPLCMDIR));
- Long lastReplId;
- if (!dumpRoot.getFileSystem(conf).exists(dumpRoot)
- || dumpRoot.getFileSystem(conf).listStatus(dumpRoot).length ==
0) {
-lastReplId = bootStrapDump(hiveDumpRoot, dmd, cmRoot, hiveDb);
+ Path previousDumpMetaPath = getPreviousDumpMetadataPath(dumpRoot);
Review comment:
Does this mean that. user is not supposed to Alter the dump directory in the
middle of replication? Any way to restrict that operation?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 403742)
Time Spent: 3h (was: 2h 50m)
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 3h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22995) Add support for location for managed tables on database
[
https://issues.apache.org/jira/browse/HIVE-22995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059967#comment-17059967
]
Hive QA commented on HIVE-22995:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 16s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-21129/patches/PreCommit-HIVE-Build-21129.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21129/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add support for location for managed tables on database
> ---
>
> Key: HIVE-22995
> URL: https://issues.apache.org/jira/browse/HIVE-22995
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22995.1.patch, Hive Metastore Support for
> Tenant-based storage heirarchy.pdf
>
>
> I have attached the initial spec to this jira.
> Default location for database would be the external table base directory.
> Managed location can be optionally specified.
> {code}
> CREATE (DATABASE|SCHEMA) [IF NOT EXISTS] database_name
> [COMMENT database_comment]
> [LOCATION hdfs_path]
> [MANAGEDLOCATION hdfs_path]
> [WITH DBPROPERTIES (property_name=property_value, ...)];
> ALTER (DATABASE|SCHEMA) database_name SET
> MANAGEDLOCATION
> hdfs_path;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403733&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403733
]
ASF GitHub Bot logged work on HIVE-22990:
-
Author: ASF GitHub Bot
Created on: 16/Mar/20 05:33
Start Date: 16/Mar/20 05:33
Worklog Time Spent: 10m
Work Description: aasha commented on pull request #949: HIVE-22990 Add
file based ack for replication
URL: https://github.com/apache/hive/pull/949#discussion_r392790094
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##
@@ -146,7 +146,7 @@ public int execute() {
}
prepareReturnValues(Arrays.asList(currentDumpPath.toUri().toString(),
String.valueOf(lastReplId)));
writeDumpCompleteAck(hiveDumpRoot);
-deletePreviousDumpMeta(previousDumpMetaPath);
+deleteAllPreviousDumpMeta(dumpRoot, currentDumpPath);
Review comment:
Load should read only the latest one.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 403733)
Time Spent: 2h 50m (was: 2h 40m)
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403732&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403732
]
ASF GitHub Bot logged work on HIVE-22990:
-
Author: ASF GitHub Bot
Created on: 16/Mar/20 05:33
Start Date: 16/Mar/20 05:33
Worklog Time Spent: 10m
Work Description: aasha commented on pull request #949: HIVE-22990 Add
file based ack for replication
URL: https://github.com/apache/hive/pull/949#discussion_r392789944
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadCompleteAckTask.java
##
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec.repl;
+
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hive.ql.ErrorMsg;
+import org.apache.hadoop.hive.ql.exec.Task;
+import org.apache.hadoop.hive.ql.exec.repl.util.ReplUtils;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+import org.apache.hadoop.hive.ql.parse.repl.dump.Utils;
+import org.apache.hadoop.hive.ql.plan.api.StageType;
+
+import java.io.Serializable;
+
+/**
+ * ReplLoadCompleteAckTask.
+ *
+ * Add the load complete acknoledgement.
+ **/
+public class ReplLoadCompleteAckTask extends Task
implements Serializable {
Review comment:
If we directly add the ack at the ReplLoadTask complete, its not correct.
There can be DDLM Tasks or Copy Tasks which are not yet finished but the Repl
Load Task is complete. So we need to add this task at the end of the DAG so
that once all other tasks are complete without any error only then it writes
the ack
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 403732)
Time Spent: 2h 40m (was: 2.5h)
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403731&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403731
]
ASF GitHub Bot logged work on HIVE-22990:
-
Author: ASF GitHub Bot
Created on: 16/Mar/20 05:31
Start Date: 16/Mar/20 05:31
Worklog Time Spent: 10m
Work Description: aasha commented on pull request #949: HIVE-22990 Add
file based ack for replication
URL: https://github.com/apache/hive/pull/949#discussion_r392789614
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java
##
@@ -584,6 +578,7 @@ private int executeIncrementalLoad() {
DAGTraversal.traverse(childTasks, new
AddDependencyToLeaves(TaskFactory.get(work, conf)));
}
this.childTasks = childTasks;
+ createReplLoadCompleteAckTask();
Review comment:
The check is inside the method
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 403731)
Time Spent: 2.5h (was: 2h 20m)
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-21164) ACID: explore how we can avoid a move step during inserts/compaction
[ https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059963#comment-17059963 ] Gopal Vijayaraghavan edited comment on HIVE-21164 at 3/16/20, 5:30 AM: --- bq. it deals with inserting data to existing tables and contains no code specific to execution engines. Actually, that might not be true because of the file commit handling code which is specific to each execution engine. You can try replacing the "insert overwrite" with a simple "insert" to explore that theory. was (Author: gopalv): bq. it deals with inserting data to existing tables and contains no code specific to execution engines. Actually, that might not be true because of the file commit handling code which is specific to each execution engine. > ACID: explore how we can avoid a move step during inserts/compaction > > > Key: HIVE-21164 > URL: https://issues.apache.org/jira/browse/HIVE-21164 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Marta Kuczora >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21164.1.patch, HIVE-21164.10.patch, > HIVE-21164.11.patch, HIVE-21164.11.patch, HIVE-21164.12.patch, > HIVE-21164.13.patch, HIVE-21164.14.patch, HIVE-21164.14.patch, > HIVE-21164.15.patch, HIVE-21164.16.patch, HIVE-21164.17.patch, > HIVE-21164.18.patch, HIVE-21164.19.patch, HIVE-21164.2.patch, > HIVE-21164.20.patch, HIVE-21164.21.patch, HIVE-21164.22.patch, > HIVE-21164.3.patch, HIVE-21164.4.patch, HIVE-21164.5.patch, > HIVE-21164.6.patch, HIVE-21164.7.patch, HIVE-21164.8.patch, HIVE-21164.9.patch > > > Currently, we write compacted data to a temporary location and then move the > files to a final location, which is an expensive operation on some cloud file > systems. Since HIVE-20823 is already in, it can control the visibility of > compacted data for the readers. Therefore, we can perhaps avoid writing data > to a temporary location and directly write compacted data to the intended > final path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21164) ACID: explore how we can avoid a move step during inserts/compaction
[ https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059963#comment-17059963 ] Gopal Vijayaraghavan commented on HIVE-21164: - bq. it deals with inserting data to existing tables and contains no code specific to execution engines. Actually, that might not be true because of the file commit handling code which is specific to each execution engine. > ACID: explore how we can avoid a move step during inserts/compaction > > > Key: HIVE-21164 > URL: https://issues.apache.org/jira/browse/HIVE-21164 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Marta Kuczora >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21164.1.patch, HIVE-21164.10.patch, > HIVE-21164.11.patch, HIVE-21164.11.patch, HIVE-21164.12.patch, > HIVE-21164.13.patch, HIVE-21164.14.patch, HIVE-21164.14.patch, > HIVE-21164.15.patch, HIVE-21164.16.patch, HIVE-21164.17.patch, > HIVE-21164.18.patch, HIVE-21164.19.patch, HIVE-21164.2.patch, > HIVE-21164.20.patch, HIVE-21164.21.patch, HIVE-21164.22.patch, > HIVE-21164.3.patch, HIVE-21164.4.patch, HIVE-21164.5.patch, > HIVE-21164.6.patch, HIVE-21164.7.patch, HIVE-21164.8.patch, HIVE-21164.9.patch > > > Currently, we write compacted data to a temporary location and then move the > files to a final location, which is an expensive operation on some cloud file > systems. Since HIVE-20823 is already in, it can control the visibility of > compacted data for the readers. Therefore, we can perhaps avoid writing data > to a temporary location and directly write compacted data to the intended > final path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23028) Should not use group parameter when run tests in standalone-metastore-common
[
https://issues.apache.org/jira/browse/HIVE-23028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059962#comment-17059962
]
Hive QA commented on HIVE-23028:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996799/HIVE-23028.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 18101 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.org.apache.hadoop.hive.ql.TestWarehouseExternalDir
(batchId=270)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testExternalDefaultPaths
(batchId=270)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21128/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21128/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21128/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996799 - PreCommit-HIVE-Build
> Should not use group parameter when run tests in standalone-metastore-common
>
>
> Key: HIVE-23028
> URL: https://issues.apache.org/jira/browse/HIVE-23028
> Project: Hive
> Issue Type: Bug
>Reporter: Zhenyu Zheng
>Assignee: Zhenyu Zheng
>Priority: Major
> Attachments: HIVE-23028.1.patch
>
>
> Should not use group parameter when run tests in standalone-metastore-common
> we inherited the `group` parameter from ‘standalone-metastore’s pom.xml and
> there the parameter
> is set to : org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest
> ([https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61])
> which only exists in metastore-server package:
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/annotation/MetastoreUnitTest.java]
> and for tests for metastore-common package does not have the @Catagory header:
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/test/java/org/apache/hadoop/hive/metastore/utils/RetryTest.java]
>
> and it was actually introduced by:
> [https://github.com/apache/hive/commit/7411d42579ffa0bad96e8da731a1a35afc9ff614#diff-171fcb0dda3bcba577fa13720d5b6571]
> we should remove the group paramenter in
> [https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml|https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22995) Add support for location for managed tables on database
[
https://issues.apache.org/jira/browse/HIVE-22995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam updated HIVE-22995:
-
Status: Patch Available (was: Open)
Initial patch with changes, to get the test run going. I expect test failures.
> Add support for location for managed tables on database
> ---
>
> Key: HIVE-22995
> URL: https://issues.apache.org/jira/browse/HIVE-22995
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22995.1.patch, Hive Metastore Support for
> Tenant-based storage heirarchy.pdf
>
>
> I have attached the initial spec to this jira.
> Default location for database would be the external table base directory.
> Managed location can be optionally specified.
> {code}
> CREATE (DATABASE|SCHEMA) [IF NOT EXISTS] database_name
> [COMMENT database_comment]
> [LOCATION hdfs_path]
> [MANAGEDLOCATION hdfs_path]
> [WITH DBPROPERTIES (property_name=property_value, ...)];
> ALTER (DATABASE|SCHEMA) database_name SET
> MANAGEDLOCATION
> hdfs_path;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22995) Add support for location for managed tables on database
[
https://issues.apache.org/jira/browse/HIVE-22995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam updated HIVE-22995:
-
Attachment: HIVE-22995.1.patch
> Add support for location for managed tables on database
> ---
>
> Key: HIVE-22995
> URL: https://issues.apache.org/jira/browse/HIVE-22995
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Major
> Attachments: HIVE-22995.1.patch, Hive Metastore Support for
> Tenant-based storage heirarchy.pdf
>
>
> I have attached the initial spec to this jira.
> Default location for database would be the external table base directory.
> Managed location can be optionally specified.
> {code}
> CREATE (DATABASE|SCHEMA) [IF NOT EXISTS] database_name
> [COMMENT database_comment]
> [LOCATION hdfs_path]
> [MANAGEDLOCATION hdfs_path]
> [WITH DBPROPERTIES (property_name=property_value, ...)];
> ALTER (DATABASE|SCHEMA) database_name SET
> MANAGEDLOCATION
> hdfs_path;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23027) Fix syntax error in llap package.py
[
https://issues.apache.org/jira/browse/HIVE-23027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059955#comment-17059955
]
Gopal Vijayaraghavan commented on HIVE-23027:
-
LGTM - +1
The test failure is definitely unrelated.
{code}
Client Execution succeeded but contained differences (error code = 1) after
executing sysdb_schq.q
128c128
< 1 1 EXECUTING true NULL NULL true
---
> 1 1 FINISHED true true NULL NULL
{code}
Looks like a timing issue.
> Fix syntax error in llap package.py
> ---
>
> Key: HIVE-23027
> URL: https://issues.apache.org/jira/browse/HIVE-23027
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-23027.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23022) Arrow deserializer should ensure size of hive vector equal to arrow vector
[
https://issues.apache.org/jira/browse/HIVE-23022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059943#comment-17059943
]
mahesh kumar behera commented on HIVE-23022:
+1
[^HIVE-23022.01.patch] looks fine to me.
> Arrow deserializer should ensure size of hive vector equal to arrow vector
> --
>
> Key: HIVE-23022
> URL: https://issues.apache.org/jira/browse/HIVE-23022
> Project: Hive
> Issue Type: Bug
> Components: llap, Serializers/Deserializers
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23022.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Arrow deserializer - {{org.apache.hadoop.hive.ql.io.arrow.Deserializer}} in
> some cases does not set the size of hive vector correctly. Size of hive
> vector should be set at least equal to arrow vector to be able to read
> (accommodate) it fully.
> Following exception can be seen when we try to read (using
> {{LlapArrowRowInputFormat}} ) some table which contains complex types (struct
> nested in array to be specific) and number of rows in table is more than
> default (1024) batch/vector size.
> {code:java}
> Caused by: java.lang.ArrayIndexOutOfBoundsException: 1024
> at
> org.apache.hadoop.hive.ql.io.arrow.Deserializer.readStruct(Deserializer.java:440)
> at
> org.apache.hadoop.hive.ql.io.arrow.Deserializer.read(Deserializer.java:143)
> at
> org.apache.hadoop.hive.ql.io.arrow.Deserializer.readList(Deserializer.java:394)
> at
> org.apache.hadoop.hive.ql.io.arrow.Deserializer.read(Deserializer.java:137)
> at
> org.apache.hadoop.hive.ql.io.arrow.Deserializer.deserialize(Deserializer.java:122)
> at
> org.apache.hadoop.hive.ql.io.arrow.ArrowColumnarBatchSerDe.deserialize(ArrowColumnarBatchSerDe.java:284)
> at
> org.apache.hadoop.hive.llap.LlapArrowRowRecordReader.next(LlapArrowRowRecordReader.java:75)
> ... 23 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23028) Should not use group parameter when run tests in standalone-metastore-common
[
https://issues.apache.org/jira/browse/HIVE-23028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059936#comment-17059936
]
Hive QA commented on HIVE-23028:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21128/dev-support/hive-personality.sh
|
| git revision | master / 632ce66 |
| Default Java | 1.8.0_111 |
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21128/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Should not use group parameter when run tests in standalone-metastore-common
>
>
> Key: HIVE-23028
> URL: https://issues.apache.org/jira/browse/HIVE-23028
> Project: Hive
> Issue Type: Bug
>Reporter: Zhenyu Zheng
>Assignee: Zhenyu Zheng
>Priority: Major
> Attachments: HIVE-23028.1.patch
>
>
> Should not use group parameter when run tests in standalone-metastore-common
> we inherited the `group` parameter from ‘standalone-metastore’s pom.xml and
> there the parameter
> is set to : org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest
> ([https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61])
> which only exists in metastore-server package:
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/annotation/MetastoreUnitTest.java]
> and for tests for metastore-common package does not have the @Catagory header:
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/test/java/org/apache/hadoop/hive/metastore/utils/RetryTest.java]
>
> and it was actually introduced by:
> [https://github.com/apache/hive/commit/7411d42579ffa0bad96e8da731a1a35afc9ff614#diff-171fcb0dda3bcba577fa13720d5b6571]
> we should remove the group paramenter in
> [https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml|https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403718&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403718
]
ASF GitHub Bot logged work on HIVE-22990:
-
Author: ASF GitHub Bot
Created on: 16/Mar/20 04:02
Start Date: 16/Mar/20 04:02
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #949: HIVE-22990
Add file based ack for replication
URL: https://github.com/apache/hive/pull/949#discussion_r392772991
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplDumpTask.java
##
@@ -146,7 +146,7 @@ public int execute() {
}
prepareReturnValues(Arrays.asList(currentDumpPath.toUri().toString(),
String.valueOf(lastReplId)));
writeDumpCompleteAck(hiveDumpRoot);
-deletePreviousDumpMeta(previousDumpMetaPath);
+deleteAllPreviousDumpMeta(dumpRoot, currentDumpPath);
Review comment:
As load and dump are operating on same path ..how will it behave when load
is reading these dump directory while dump is reading it ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 403718)
Time Spent: 2h 20m (was: 2h 10m)
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403716&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403716 ] ASF GitHub Bot logged work on HIVE-22990: - Author: ASF GitHub Bot Created on: 16/Mar/20 04:02 Start Date: 16/Mar/20 04:02 Worklog Time Spent: 10m Work Description: maheshk114 commented on pull request #949: HIVE-22990 Add file based ack for replication URL: https://github.com/apache/hive/pull/949#discussion_r392770512 ## File path: ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/Utils.java ## @@ -84,10 +85,17 @@ public static void writeOutput(List> listValues, Path outputFile, H public static void write(Path outputFile, HiveConf hiveConf) Review comment: change the name to create This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 403716) Time Spent: 2h 10m (was: 2h) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch, > HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch, > HIVE-22990.15.patch, HIVE-22990.patch > > Time Spent: 2h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403715&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403715
]
ASF GitHub Bot logged work on HIVE-22990:
-
Author: ASF GitHub Bot
Created on: 16/Mar/20 04:02
Start Date: 16/Mar/20 04:02
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #949: HIVE-22990
Add file based ack for replication
URL: https://github.com/apache/hive/pull/949#discussion_r392772730
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadCompleteAckTask.java
##
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.exec.repl;
+
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hive.ql.ErrorMsg;
+import org.apache.hadoop.hive.ql.exec.Task;
+import org.apache.hadoop.hive.ql.exec.repl.util.ReplUtils;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+import org.apache.hadoop.hive.ql.parse.repl.dump.Utils;
+import org.apache.hadoop.hive.ql.plan.api.StageType;
+
+import java.io.Serializable;
+
+/**
+ * ReplLoadCompleteAckTask.
+ *
+ * Add the load complete acknoledgement.
+ **/
+public class ReplLoadCompleteAckTask extends Task
implements Serializable {
Review comment:
i don't think a separate task is required for this
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 403715)
Time Spent: 2h 10m (was: 2h)
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403717&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403717
]
ASF GitHub Bot logged work on HIVE-22990:
-
Author: ASF GitHub Bot
Created on: 16/Mar/20 04:02
Start Date: 16/Mar/20 04:02
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #949: HIVE-22990
Add file based ack for replication
URL: https://github.com/apache/hive/pull/949#discussion_r392772602
##
File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/ReplLoadTask.java
##
@@ -584,6 +578,7 @@ private int executeIncrementalLoad() {
DAGTraversal.traverse(childTasks, new
AddDependencyToLeaves(TaskFactory.get(work, conf)));
}
this.childTasks = childTasks;
+ createReplLoadCompleteAckTask();
Review comment:
This should be done only once when no more tasks are there to execute .. "if
(builder.hasMoreWork() || work.getPathsToCopyIterator().hasNext() ||
work.hasBootstrapLoadTasks()) {'
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 403717)
Time Spent: 2h 10m (was: 2h)
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23027) Fix syntax error in llap package.py
[
https://issues.apache.org/jira/browse/HIVE-23027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059930#comment-17059930
]
Hive QA commented on HIVE-23027:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996791/HIVE-23027.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 18100 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=182)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21127/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21127/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21127/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996791 - PreCommit-HIVE-Build
> Fix syntax error in llap package.py
> ---
>
> Key: HIVE-23027
> URL: https://issues.apache.org/jira/browse/HIVE-23027
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-23027.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23027) Fix syntax error in llap package.py
[
https://issues.apache.org/jira/browse/HIVE-23027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059909#comment-17059909
]
Hive QA commented on HIVE-23027:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21127/dev-support/hive-personality.sh
|
| git revision | master / 632ce66 |
| Default Java | 1.8.0_111 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21127/yetus/whitespace-tabs.txt
|
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21127/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix syntax error in llap package.py
> ---
>
> Key: HIVE-23027
> URL: https://issues.apache.org/jira/browse/HIVE-23027
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-23027.1.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?focusedWorklogId=403702&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-403702
]
ASF GitHub Bot logged work on HIVE-22990:
-
Author: ASF GitHub Bot
Created on: 16/Mar/20 02:38
Start Date: 16/Mar/20 02:38
Worklog Time Spent: 10m
Work Description: maheshk114 commented on pull request #949: HIVE-22990
Add file based ack for replication
URL: https://github.com/apache/hive/pull/949#discussion_r392758479
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenarios.java
##
@@ -2387,39 +2427,33 @@ public void testTruncateWithCM() throws IOException {
run("INSERT INTO TABLE " + dbName + ".unptned values('" + unptn_data[0] +
"')", driver);
Tuple firstInsert = replDumpDb(dbName);
Integer numOfEventsIns1 = Integer.valueOf(firstInsert.lastReplId) -
Integer.valueOf(replDumpId);
+// load only first insert (1 record)
+loadAndVerify(replDbName, dbName, firstInsert.lastReplId);
Review comment:
i think ..the old repl dump and load mechanism should work fine.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 403702)
Time Spent: 2h (was: 1h 50m)
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23028) Should not use group parameter when run tests in standalone-metastore-common
[ https://issues.apache.org/jira/browse/HIVE-23028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059903#comment-17059903 ] Zhenyu Zheng commented on HIVE-23028: - Uploaded a patch to remove test group limitation for standalone-metastore/metastore-common/pom.xml since it is not correct for the tests in that package. > Should not use group parameter when run tests in standalone-metastore-common > > > Key: HIVE-23028 > URL: https://issues.apache.org/jira/browse/HIVE-23028 > Project: Hive > Issue Type: Bug >Reporter: Zhenyu Zheng >Assignee: Zhenyu Zheng >Priority: Major > Attachments: HIVE-23028.1.patch > > > Should not use group parameter when run tests in standalone-metastore-common > we inherited the `group` parameter from ‘standalone-metastore’s pom.xml and > there the parameter > is set to : org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest > ([https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61]) > which only exists in metastore-server package: > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/annotation/MetastoreUnitTest.java] > and for tests for metastore-common package does not have the @Catagory header: > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/test/java/org/apache/hadoop/hive/metastore/utils/RetryTest.java] > > and it was actually introduced by: > [https://github.com/apache/hive/commit/7411d42579ffa0bad96e8da731a1a35afc9ff614#diff-171fcb0dda3bcba577fa13720d5b6571] > we should remove the group paramenter in > [https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml|https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23028) Should not use group parameter when run tests in standalone-metastore-common
[ https://issues.apache.org/jira/browse/HIVE-23028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng updated HIVE-23028: Attachment: HIVE-23028.1.patch Status: Patch Available (was: Open) > Should not use group parameter when run tests in standalone-metastore-common > > > Key: HIVE-23028 > URL: https://issues.apache.org/jira/browse/HIVE-23028 > Project: Hive > Issue Type: Bug >Reporter: Zhenyu Zheng >Assignee: Zhenyu Zheng >Priority: Major > Attachments: HIVE-23028.1.patch > > > Should not use group parameter when run tests in standalone-metastore-common > we inherited the `group` parameter from ‘standalone-metastore’s pom.xml and > there the parameter > is set to : org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest > ([https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61]) > which only exists in metastore-server package: > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/annotation/MetastoreUnitTest.java] > and for tests for metastore-common package does not have the @Catagory header: > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/test/java/org/apache/hadoop/hive/metastore/utils/RetryTest.java] > > and it was actually introduced by: > [https://github.com/apache/hive/commit/7411d42579ffa0bad96e8da731a1a35afc9ff614#diff-171fcb0dda3bcba577fa13720d5b6571] > we should remove the group paramenter in > [https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml|https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23028) Should not use group parameter when run tests in standalone-metastore-common
[ https://issues.apache.org/jira/browse/HIVE-23028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenyu Zheng reassigned HIVE-23028: --- > Should not use group parameter when run tests in standalone-metastore-common > > > Key: HIVE-23028 > URL: https://issues.apache.org/jira/browse/HIVE-23028 > Project: Hive > Issue Type: Bug >Reporter: Zhenyu Zheng >Assignee: Zhenyu Zheng >Priority: Major > > Should not use group parameter when run tests in standalone-metastore-common > we inherited the `group` parameter from ‘standalone-metastore’s pom.xml and > there the parameter > is set to : org.apache.hadoop.hive.metastore.annotation.MetastoreUnitTest > ([https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61]) > which only exists in metastore-server package: > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/annotation/MetastoreUnitTest.java] > and for tests for metastore-common package does not have the @Catagory header: > [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/test/java/org/apache/hadoop/hive/metastore/utils/RetryTest.java] > > and it was actually introduced by: > [https://github.com/apache/hive/commit/7411d42579ffa0bad96e8da731a1a35afc9ff614#diff-171fcb0dda3bcba577fa13720d5b6571] > we should remove the group paramenter in > [https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml|https://github.com/apache/hive/blob/master/standalone-metastore/pom.xml#L61] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23011) Shared work optimizer should check residual predicates when comparing joins
[
https://issues.apache.org/jira/browse/HIVE-23011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059900#comment-17059900
]
Hive QA commented on HIVE-23011:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996789/HIVE-23011.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 18101 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query30]
(batchId=306)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21126/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21126/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21126/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996789 - PreCommit-HIVE-Build
> Shared work optimizer should check residual predicates when comparing joins
> ---
>
> Key: HIVE-23011
> URL: https://issues.apache.org/jira/browse/HIVE-23011
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch,
> HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch,
> HIVE-23011.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23011) Shared work optimizer should check residual predicates when comparing joins
[
https://issues.apache.org/jira/browse/HIVE-23011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059881#comment-17059881
]
Hive QA commented on HIVE-23011:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 3 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21126/dev-support/hive-personality.sh
|
| git revision | master / 632ce66 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21126/yetus/whitespace-eol.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21126/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Shared work optimizer should check residual predicates when comparing joins
> ---
>
> Key: HIVE-23011
> URL: https://issues.apache.org/jira/browse/HIVE-23011
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch,
> HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch,
> HIVE-23011.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23027) Fix syntax error in llap package.py
[ https://issues.apache.org/jira/browse/HIVE-23027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-23027: Status: Patch Available (was: Open) > Fix syntax error in llap package.py > --- > > Key: HIVE-23027 > URL: https://issues.apache.org/jira/browse/HIVE-23027 > Project: Hive > Issue Type: Bug >Reporter: Rajesh Balamohan >Priority: Trivial > Attachments: HIVE-23027.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23027) Fix syntax error in llap package.py
[ https://issues.apache.org/jira/browse/HIVE-23027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-23027: Attachment: HIVE-23027.1.patch > Fix syntax error in llap package.py > --- > > Key: HIVE-23027 > URL: https://issues.apache.org/jira/browse/HIVE-23027 > Project: Hive > Issue Type: Bug >Reporter: Rajesh Balamohan >Priority: Trivial > Attachments: HIVE-23027.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23011) Shared work optimizer should check residual predicates when comparing joins
[ https://issues.apache.org/jira/browse/HIVE-23011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-23011: --- Attachment: HIVE-23011.patch > Shared work optimizer should check residual predicates when comparing joins > --- > > Key: HIVE-23011 > URL: https://issues.apache.org/jira/browse/HIVE-23011 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch, > HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch, HIVE-23011.patch, > HIVE-23011.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23013) Fix UnitTestPropertiesParser creation log message
[ https://issues.apache.org/jira/browse/HIVE-23013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23013: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Fix UnitTestPropertiesParser creation log message > - > > Key: HIVE-23013 > URL: https://issues.apache.org/jira/browse/HIVE-23013 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23013.01.patch > > > UnitTestPropertiesParser has a bug preventing from logging it's creation info > appropriately. Instead of string concatenation ( + ) it has a ( , ), thus the > end of the message is considered as the first argument, making a mess. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23007) Do Not Consider Client Session For Default Fetch Size
[
https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059825#comment-17059825
]
Hive QA commented on HIVE-23007:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996786/HIVE-23017.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 18099 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcDriver2.testURLWithFetchSize (batchId=292)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testFetchSize (batchId=293)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21125/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21125/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21125/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996786 - PreCommit-HIVE-Build
> Do Not Consider Client Session For Default Fetch Size
> -
>
> Key: HIVE-23007
> URL: https://issues.apache.org/jira/browse/HIVE-23007
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch,
> HIVE-23017.3.patch
>
>
> Right now, when a client makes a new session, it's value override's the
> server's value for {{hive.server2.thrift.resultset.default.fetch.size}}.
> This is a server-side configuration and the client should not dictate what
> the server's default (preferred) size is.
> * Remove superfluous null check
> * Use a Collection's Singleton Map instead of creating a full-sized HashMap
> * Code simplification
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22974) Metastore's table location check should be applied when location changed
[ https://issues.apache.org/jira/browse/HIVE-22974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22974: - Resolution: Fixed Status: Resolved (was: Patch Available) Fix has been committed to master. Thank you for the patch [~amagyar] > Metastore's table location check should be applied when location changed > > > Key: HIVE-22974 > URL: https://issues.apache.org/jira/browse/HIVE-22974 > Project: Hive > Issue Type: Sub-task > Components: Metastore >Reporter: Attila Magyar >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22974.2.patch, HIVE-22974.3.patch, > HIVE-22974.4.patch > > > In HIVE-22189 a check was introduced to make sure managed and external tables > are located at the proper space. This condition cannot be satisfied during an > upgrade. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23007) Do Not Consider Client Session For Default Fetch Size
[
https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059801#comment-17059801
]
Hive QA commented on HIVE-23007:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} service in master has 51 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} service: The patch generated 0 new + 26 unchanged -
2 fixed = 26 total (was 28) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21125/dev-support/hive-personality.sh
|
| git revision | master / 2baf21b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21125/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Do Not Consider Client Session For Default Fetch Size
> -
>
> Key: HIVE-23007
> URL: https://issues.apache.org/jira/browse/HIVE-23007
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch,
> HIVE-23017.3.patch
>
>
> Right now, when a client makes a new session, it's value override's the
> server's value for {{hive.server2.thrift.resultset.default.fetch.size}}.
> This is a server-side configuration and the client should not dictate what
> the server's default (preferred) size is.
> * Remove superfluous null check
> * Use a Collection's Singleton Map instead of creating a full-sized HashMap
> * Code simplification
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Do Not Consider Client Session For Default Fetch Size
[
https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23007:
--
Description:
Right now, when a client makes a new session, it's value override's the
server's value for {{hive.server2.thrift.resultset.default.fetch.size}}. This
is a server-side configuration and the client should not dictate what the
server's default (preferred) size is.
* Remove superfluous null check
* Use a Collection's Singleton Map instead of creating a full-sized HashMap
* Code simplification
was:
* Remove superfluous null check
* Use a Collection's Singleton Map instead of creating a full-sized HashMap
* Code simplification
> Do Not Consider Client Session For Default Fetch Size
> -
>
> Key: HIVE-23007
> URL: https://issues.apache.org/jira/browse/HIVE-23007
> Project: Hive
> Issue Type: Sub-task
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch,
> HIVE-23017.3.patch
>
>
> Right now, when a client makes a new session, it's value override's the
> server's value for {{hive.server2.thrift.resultset.default.fetch.size}}.
> This is a server-side configuration and the client should not dictate what
> the server's default (preferred) size is.
> * Remove superfluous null check
> * Use a Collection's Singleton Map instead of creating a full-sized HashMap
> * Code simplification
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Do Not Consider Client Session For Default Fetch Size
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Summary: Do Not Consider Client Session For Default Fetch Size (was: Cleanup Default Fetch Size If One Is Not Sent By Client) > Do Not Consider Client Session For Default Fetch Size > - > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch, > HIVE-23017.3.patch > > > * Remove superfluous null check > * Use a Collection's Singleton Map instead of creating a full-sized HashMap > * Code simplification -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23007) Do Not Consider Client Session For Default Fetch Size
[ https://issues.apache.org/jira/browse/HIVE-23007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-23007: -- Attachment: HIVE-23017.3.patch > Do Not Consider Client Session For Default Fetch Size > - > > Key: HIVE-23007 > URL: https://issues.apache.org/jira/browse/HIVE-23007 > Project: Hive > Issue Type: Sub-task >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-23007.1.patch, HIVE-23007.2.patch, > HIVE-23017.3.patch > > > * Remove superfluous null check > * Use a Collection's Singleton Map instead of creating a full-sized HashMap > * Code simplification -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059757#comment-17059757
]
Hive QA commented on HIVE-22990:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12996780/HIVE-22990.15.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 18099 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[repl_dump_requires_admin]
(batchId=107)
org.apache.hadoop.hive.ql.parse.TestReplAcidTablesBootstrapWithJsonMessage.testRetryAcidTablesBootstrapFromDifferentDump
(batchId=259)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testBootStrapDumpOfWarehouse
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testIfBootstrapReplLoadFailWhenRetryAfterBootstrapComplete
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testMoveOptimizationBootstrapReplLoadRetryAfterFailure
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testMoveOptimizationIncrementalFailureAfterCopy
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplAcrossInstancesWithJsonMessageFormat.testMoveOptimizationIncrementalFailureAfterCopyReplace
(batchId=268)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcidTablesBootstrap.testRetryAcidTablesBootstrapFromDifferentDump
(batchId=257)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testBootStrapDumpOfWarehouse
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testIfBootstrapReplLoadFailWhenRetryAfterBootstrapComplete
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testMoveOptimizationBootstrapReplLoadRetryAfterFailure
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testMoveOptimizationIncrementalFailureAfterCopy
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosAcrossInstances.testMoveOptimizationIncrementalFailureAfterCopyReplace
(batchId=273)
org.apache.hadoop.hive.ql.parse.TestReplicationScenariosExternalTables.retryBootstrapExternalTablesFromDifferentDump
(batchId=267)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.testIncLoadPenFlagPropAlterDB
(batchId=274)
org.apache.hive.jdbc.TestJdbcDriver2.testGetQueryLogForReplCommands
(batchId=292)
org.apache.hive.jdbc.TestJdbcGenericUDTFGetSplits2.testGenericUDTFOrderBySplitCount1
(batchId=291)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/21124/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/21124/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-21124/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12996780 - PreCommit-HIVE-Build
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[
https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059737#comment-17059737
]
Hive QA commented on HIVE-22990:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 693
unchanged - 3 fixed = 693 total (was 696) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 8s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-21124/dev-support/hive-personality.sh
|
| git revision | master / 2baf21b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-21124/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Build acknowledgement mechanism for repl dump and load
> --
>
> Key: HIVE-22990
> URL: https://issues.apache.org/jira/browse/HIVE-22990
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch,
> HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch,
> HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch,
> HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch,
> HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch,
> HIVE-22990.15.patch, HIVE-22990.patch
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Attachment: HIVE-22990.15.patch Status: Patch Available (was: In Progress) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch, > HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch, > HIVE-22990.15.patch, HIVE-22990.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22990) Build acknowledgement mechanism for repl dump and load
[ https://issues.apache.org/jira/browse/HIVE-22990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-22990: --- Status: In Progress (was: Patch Available) > Build acknowledgement mechanism for repl dump and load > -- > > Key: HIVE-22990 > URL: https://issues.apache.org/jira/browse/HIVE-22990 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22990.01.patch, HIVE-22990.02.patch, > HIVE-22990.03.patch, HIVE-22990.04.patch, HIVE-22990.05.patch, > HIVE-22990.06.patch, HIVE-22990.07.patch, HIVE-22990.08.patch, > HIVE-22990.09.patch, HIVE-22990.10.patch, HIVE-22990.11.patch, > HIVE-22990.12.patch, HIVE-22990.13.patch, HIVE-22990.14.patch, > HIVE-22990.15.patch, HIVE-22990.patch > > Time Spent: 1h 50m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HIVE-23025) Support add a yarn application name for tez on hiveserver2
[ https://issues.apache.org/jira/browse/HIVE-23025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jake Xie resolved HIVE-23025. - Resolution: Abandoned > Support add a yarn application name for tez on hiveserver2 > -- > > Key: HIVE-23025 > URL: https://issues.apache.org/jira/browse/HIVE-23025 > Project: Hive > Issue Type: New Feature > Components: Tez >Affects Versions: 2.3.5 >Reporter: Jake Xie >Priority: Major > Fix For: 2.3.5 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20232) Support add a yarn application name for tez on hiveserver2
[ https://issues.apache.org/jira/browse/HIVE-20232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jake Xie updated HIVE-20232: Description: Currently tez on hiveServer2 cannot specify yarn application name, which is not very convenient for locating the problem SQL, so i added a configuration item to support setting tez job name > Support add a yarn application name for tez on hiveserver2 > -- > > Key: HIVE-20232 > URL: https://issues.apache.org/jira/browse/HIVE-20232 > Project: Hive > Issue Type: Improvement > Components: Operators >Reporter: Michael Lee >Priority: Major > > Currently tez on hiveServer2 cannot specify yarn application name, which is > not very convenient for locating the problem SQL, so i added a configuration > item to support setting tez job name -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20232) Support add a yarn application name for tez on hiveserver2
[ https://issues.apache.org/jira/browse/HIVE-20232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jake Xie updated HIVE-20232: Description: (was: Hello, I am trying to divide the values of two fields that have been joined together on multiple criteria (offerlevelid, visit_date, days_to_action). For some rows, the quotient is correct, but for other rows, the result is zero. See below: TABLE A: mlee.mit_test1 select * from mlee.mit_test1 limit 5; ||offerlevelid||action_date||visit_date||days_to_action||cluster||cnt|| |29992|_2018-07-11_|_2018-06-28_|13|11158|1| |_29991_|_2018-07-12_|_2018-06-18_|24 |11158 |0 | |_5279_|_2018-07-01_|_2018-05-30_|32|11158 |10 | |_5279_|_2018-07-01_|_2018-06-02_ |29 |11158 |1 | |_5279_|_2018-07-02_|_2018-06-29_ |3 |11158 |3 | TABLE B: mlee_p2p.num_at_visit_vd select * from mlee_p2p.num_at_visit_vd limit 5; ||offerlevelid||action_date||visit_date||days_to_action||cnt|| |5279|2018-07-06|_2018-06-17_| 19|1696 | |_5279_|_2018-07-07_|_2018-06-07_| 30|2072 | |_29991_|_2018-07-11_|_2018-07-09_| 2|361| |_29991_|_2018-07-10_|_2018-06-10_| 30|116| |29992 |_2018-07-02_|_2018-06-27_| 5|0 | When I attempt to perform division on a.cnt / b.cnt, the results do not make sense. Specifically, there are results of zero where a.cnt and b.cnt are integer values. I tried casting both as doubles, but that did not work either. See below, where I've bolded the "prob" values that do not make sense. Please advise! select a.offerlevelid, a.days_to_action, a.visit_date, a.cluster, a.cnt at_cluster_vd_dta_cnt, b.cnt at_vd_dta_cnt, a.cnt/b.cnt prob from mlee.mit_test1 a join mlee_p2p.num_at_visit_vd b on a.offerlevelid=b.offerlevelid and a.visit_date = b.visit_date and a.days_to_action = b.days_to_action order by a.days_to_action,a.visit_date limit 2000; ||offerlevelid||days_to_action||visit_date||cluster||at_cluster_vd_dta_cnt||at_vd_dta_cnt||prob|| |29991|0|2018-07-01 |11158|1|111|.009009009009009009| |5279|0|2018-07-01|11158|8|3255|_0.002457757296466974_| |_29992_|0|_2018-07-02_ |11158|0|1|0.0| |_29991_|0|_2018-07-02_ |11158|2|247|*0.0*| |_5279_|0|_2018-07-02_ |11158|3|2268|_0.0013227513227513227_| |_5279_|0|_2018-07-03_|11158|4|3206|_0.0012476606363069245_| |_29991_|0|_2018-07-03_|11158|1|293|*0.0*| |_5279_|0|_2018-07-04_|11158|4|3523|_0.0011353959693443088_| |_29991_|0|_2018-07-04_|11158|2|203|_0.009852216748768473_| |_29992_|0|_2018-07-05_|11158|0|2|*0.0*| ) Issue Type: Improvement (was: Bug) Priority: Major (was: Blocker) > Support add a yarn application name for tez on hiveserver2 > -- > > Key: HIVE-20232 > URL: https://issues.apache.org/jira/browse/HIVE-20232 > Project: Hive > Issue Type: Improvement > Components: Operators >Reporter: Michael Lee >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20232) Support add a yarn application name for tez on hiveserver2
[ https://issues.apache.org/jira/browse/HIVE-20232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jake Xie updated HIVE-20232: Summary: Support add a yarn application name for tez on hiveserver2 (was: Basic division operator not working for select statement with join) > Support add a yarn application name for tez on hiveserver2 > -- > > Key: HIVE-20232 > URL: https://issues.apache.org/jira/browse/HIVE-20232 > Project: Hive > Issue Type: Bug > Components: Operators >Reporter: Michael Lee >Priority: Blocker > > Hello, > I am trying to divide the values of two fields that have been joined together > on multiple criteria (offerlevelid, visit_date, days_to_action). For some > rows, the quotient is correct, but for other rows, the result is zero. See > below: > TABLE A: mlee.mit_test1 > select * from mlee.mit_test1 limit 5; > > ||offerlevelid||action_date||visit_date||days_to_action||cluster||cnt|| > |29992|_2018-07-11_|_2018-06-28_|13|11158|1| > |_29991_|_2018-07-12_|_2018-06-18_|24 |11158 |0 | > |_5279_|_2018-07-01_|_2018-05-30_|32|11158 |10 | > |_5279_|_2018-07-01_|_2018-06-02_ |29 |11158 |1 | > |_5279_|_2018-07-02_|_2018-06-29_ |3 |11158 |3 | > > TABLE B: mlee_p2p.num_at_visit_vd > select * from mlee_p2p.num_at_visit_vd limit 5; > ||offerlevelid||action_date||visit_date||days_to_action||cnt|| > |5279|2018-07-06|_2018-06-17_| 19|1696 | > |_5279_|_2018-07-07_|_2018-06-07_| 30|2072 | > |_29991_|_2018-07-11_|_2018-07-09_| 2|361| > |_29991_|_2018-07-10_|_2018-06-10_| 30|116| > |29992 |_2018-07-02_|_2018-06-27_| 5|0 | > > When I attempt to perform division on a.cnt / b.cnt, the results do not make > sense. Specifically, there are results of zero where a.cnt and b.cnt are > integer values. I tried casting both as doubles, but that did not work > either. See below, where I've bolded the "prob" values that do not make > sense. Please advise! > > select > a.offerlevelid, > a.days_to_action, > a.visit_date, > a.cluster, > a.cnt at_cluster_vd_dta_cnt, > b.cnt at_vd_dta_cnt, > a.cnt/b.cnt prob > from mlee.mit_test1 a > join mlee_p2p.num_at_visit_vd b on a.offerlevelid=b.offerlevelid > and a.visit_date = b.visit_date > and a.days_to_action = b.days_to_action > order by a.days_to_action,a.visit_date > limit 2000; > ||offerlevelid||days_to_action||visit_date||cluster||at_cluster_vd_dta_cnt||at_vd_dta_cnt||prob|| > |29991|0|2018-07-01 |11158|1|111|.009009009009009009| > |5279|0|2018-07-01|11158|8|3255|_0.002457757296466974_| > |_29992_|0|_2018-07-02_ |11158|0|1|0.0| > |_29991_|0|_2018-07-02_ |11158|2|247|*0.0*| > |_5279_|0|_2018-07-02_ |11158|3|2268|_0.0013227513227513227_| > |_5279_|0|_2018-07-03_|11158|4|3206|_0.0012476606363069245_| > |_29991_|0|_2018-07-03_|11158|1|293|*0.0*| > |_5279_|0|_2018-07-04_|11158|4|3523|_0.0011353959693443088_| > |_29991_|0|_2018-07-04_|11158|2|203|_0.009852216748768473_| > |_29992_|0|_2018-07-05_|11158|0|2|*0.0*| > > > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
