[jira] [Commented] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899029#comment-16899029
]
Hive QA commented on HIVE-21241:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} common: The patch generated 0 new + 0 unchanged - 14
fixed = 0 total (was 14) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} common generated 0 new + 61 unchanged - 1 fixed = 61
total (was 62) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} common generated 0 new + 26 unchanged - 1 fixed = 26
total (was 27) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18241/dev-support/hive-personality.sh
|
| git revision | master / d7475aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18241/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22068) Add more logging to notification cleaner and replication to track events
[
https://issues.apache.org/jira/browse/HIVE-22068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899122#comment-16899122
]
Hive QA commented on HIVE-22068:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
180 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
20s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 180 unchanged - 0 fixed = 181 total (was 180) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
23s{color} | {color:red} ql generated 3 new + 2250 unchanged - 0 fixed = 2253
total (was 2250) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Redundant nullcheck of lastMEvent which is known to be null in
org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(int)
Redundant null check at ObjectStore.java:is known to be null in
org.apache.hadoop.hive.metastore.ObjectStore.cleanNotificationEvents(int)
Redundant null check at ObjectStore.java:[line 10193] |
| FindBugs | module:ql |
| | Unread field:IncrementalDumpBegin.java:[line 52] |
| | Unread field:IncrementalDumpBegin.java:[line 54] |
| | Unread field:IncrementalDumpBegin.java:[line 53] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18242/dev-support/hive-personality.sh
|
| git revision | master / d7475aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18242/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18242/yetus/new-findbugs-ql.html
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18242/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add more logging to notification cleaner and replication to track events
>
[jira] [Updated] (HIVE-22068) Add more logging to notification cleaner and replication to track events
[ https://issues.apache.org/jira/browse/HIVE-22068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-22068: -- Status: In Progress (was: Patch Available) > Add more logging to notification cleaner and replication to track events > > > Key: HIVE-22068 > URL: https://issues.apache.org/jira/browse/HIVE-22068 > Project: Hive > Issue Type: Improvement >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22068.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > * Add more logging to DB notification listener cleaner thread > ** The time when it considered cleaning, the interval and time before which > events were cleared, the min and max id at that time > ** how many events were cleared > ** min and max id after the cleaning. > * In REPL::START document the starting event, end event if specified and the > maximum number of events, if specified. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-22069) joda-time binary conflict between druid-handler and phoenix-hive jars.
[ https://issues.apache.org/jira/browse/HIVE-22069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898966#comment-16898966 ] David Mollitor commented on HIVE-22069: --- Can also file a JIRA against Pheonix to remove Joda Time from there as well. > joda-time binary conflict between druid-handler and phoenix-hive jars. > -- > > Key: HIVE-22069 > URL: https://issues.apache.org/jira/browse/HIVE-22069 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.0, 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > > Hive's druid storage handler uses 2.8.1 version of the joda time library > where as the phoenix-hive.jar uses 1.6 version of this library. When both > jars are in the classpath, bad things happen. > Apache phoenix has its own release cycle and them uptaking a new version is > not what hive should count on. Besides they could decide to move to a new > version of this library and we would still have this problem. > So its best we use shaded jars in hive for the version we are on. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-22069) joda-time binary conflict between druid-handler and phoenix-hive jars.
[ https://issues.apache.org/jira/browse/HIVE-22069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898967#comment-16898967 ] David Mollitor commented on HIVE-22069: --- https://github.com/apache/incubator-druid/issues/8225 > joda-time binary conflict between druid-handler and phoenix-hive jars. > -- > > Key: HIVE-22069 > URL: https://issues.apache.org/jira/browse/HIVE-22069 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.0, 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > > Hive's druid storage handler uses 2.8.1 version of the joda time library > where as the phoenix-hive.jar uses 1.6 version of this library. When both > jars are in the classpath, bad things happen. > Apache phoenix has its own release cycle and them uptaking a new version is > not what hive should count on. Besides they could decide to move to a new > version of this library and we would still have this problem. > So its best we use shaded jars in hive for the version we are on. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22068) Add more logging to notification cleaner and replication to track events
[ https://issues.apache.org/jira/browse/HIVE-22068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Bapat updated HIVE-22068: -- Attachment: HIVE-22068.02.patch Status: Patch Available (was: In Progress) > Add more logging to notification cleaner and replication to track events > > > Key: HIVE-22068 > URL: https://issues.apache.org/jira/browse/HIVE-22068 > Project: Hive > Issue Type: Improvement >Reporter: Ashutosh Bapat >Assignee: Ashutosh Bapat >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22068.01.patch, HIVE-22068.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > * Add more logging to DB notification listener cleaner thread > ** The time when it considered cleaning, the interval and time before which > events were cleared, the min and max id at that time > ** how many events were cleared > ** min and max id after the cleaning. > * In REPL::START document the starting event, end event if specified and the > maximum number of events, if specified. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati updated HIVE-21198:
Attachment: HIVE-21198.8.patch
Status: Patch Available (was: Open)
I still had to fix a couple cans of worms and some cases may still need to be
addressed.
> Introduce a database object reference class
> ---
>
> Key: HIVE-21198
> URL: https://issues.apache.org/jira/browse/HIVE-21198
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21198.1.patch, HIVE-21198.2.patch,
> HIVE-21198.3.patch, HIVE-21198.4.patch, HIVE-21198.5.patch,
> HIVE-21198.6.patch, HIVE-21198.7.patch, HIVE-21198.8.patch
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> There are many places in which "{databasename}.{tablename}" is passed as a
> single string; there are some places where the they travel as 2 separate
> arguments.
> Idea would be to introduce a simple immutable class with 2 fields ; and pass
> these informations together. Making this better is required if we would be
> wanting to enable dot in tablenames
> HIVE-16907, HIVE-21151
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22069) joda-time binary conflict between druid-handler and phoenix-hive jars.
[ https://issues.apache.org/jira/browse/HIVE-22069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899080#comment-16899080 ] Naveen Gangam commented on HIVE-22069: -- Filed a task for Apache Phoenix as well. https://issues.apache.org/jira/browse/PHOENIX-5422 > joda-time binary conflict between druid-handler and phoenix-hive jars. > -- > > Key: HIVE-22069 > URL: https://issues.apache.org/jira/browse/HIVE-22069 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.0, 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > > Hive's druid storage handler uses 2.8.1 version of the joda time library > where as the phoenix-hive.jar uses 1.6 version of this library. When both > jars are in the classpath, bad things happen. > Apache phoenix has its own release cycle and them uptaking a new version is > not what hive should count on. Besides they could decide to move to a new > version of this library and we would still have this problem. > So its best we use shaded jars in hive for the version we are on. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-22069) joda-time binary conflict between druid-handler and phoenix-hive jars.
[
https://issues.apache.org/jira/browse/HIVE-22069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898957#comment-16898957
]
David Mollitor commented on HIVE-22069:
---
So, this is a bit tricky.
The {{druid-handler}} project depends on {{hive-exec}} which depends on ...
{{hive-serde}} which depends on {{arrow-vector}} which depends on
{{joda-time}}. Need to update {{arrow-vector}} first to remove this dependency.
> joda-time binary conflict between druid-handler and phoenix-hive jars.
> --
>
> Key: HIVE-22069
> URL: https://issues.apache.org/jira/browse/HIVE-22069
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
>
> Hive's druid storage handler uses 2.8.1 version of the joda time library
> where as the phoenix-hive.jar uses 1.6 version of this library. When both
> jars are in the classpath, bad things happen.
> Apache phoenix has its own release cycle and them uptaking a new version is
> not what hive should count on. Besides they could decide to move to a new
> version of this library and we would still have this problem.
> So its best we use shaded jars in hive for the version we are on.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21241:
--
Status: Patch Available (was: Open)
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21241:
--
Attachment: HIVE-21241.3.patch
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HIVE-22069) joda-time binary conflict between druid-handler and phoenix-hive jars.
[
https://issues.apache.org/jira/browse/HIVE-22069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898957#comment-16898957
]
David Mollitor edited comment on HIVE-22069 at 8/2/19 2:56 PM:
---
So, this is a bit tricky.
The {{druid-handler}} project depends on {{hive-exec}} which depends on ...
{{hive-serde}} which depends on {{arrow-vector}} which depends on
{{joda-time}}. Need to update {{arrow-vector}} first to remove this dependency.
Also {{druid-core}} is a dependency of {{druid-handler}} depends on
{{joda-time}}. Need to address that too.
was (Author: belugabehr):
So, this is a bit tricky.
The {{druid-handler}} project depends on {{hive-exec}} which depends on ...
{{hive-serde}} which depends on {{arrow-vector}} which depends on
{{joda-time}}. Need to update {{arrow-vector}} first to remove this dependency.
> joda-time binary conflict between druid-handler and phoenix-hive jars.
> --
>
> Key: HIVE-22069
> URL: https://issues.apache.org/jira/browse/HIVE-22069
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
>
> Hive's druid storage handler uses 2.8.1 version of the joda time library
> where as the phoenix-hive.jar uses 1.6 version of this library. When both
> jars are in the classpath, bad things happen.
> Apache phoenix has its own release cycle and them uptaking a new version is
> not what hive should count on. Besides they could decide to move to a new
> version of this library and we would still have this problem.
> So its best we use shaded jars in hive for the version we are on.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22075) Fix the max-reducers=1 regression from HIVE-14200
[
https://issues.apache.org/jira/browse/HIVE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899051#comment-16899051
]
Jesus Camacho Rodriguez commented on HIVE-22075:
+1
> Fix the max-reducers=1 regression from HIVE-14200
> -
>
> Key: HIVE-22075
> URL: https://issues.apache.org/jira/browse/HIVE-22075
> Project: Hive
> Issue Type: Bug
> Components: llap, Physical Optimizer
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Jesus Camacho Rodriguez
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22075.1.patch
>
>
> The condition does not kick in when
> {code}
> minPartition=1, maxPartition=1, nReducers=1, maxReducers=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899077#comment-16899077
]
Hive QA commented on HIVE-21241:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976546/HIVE-21241.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 16731 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=232)
org.apache.hadoop.hive.serde2.TestJsonSerDe.testTimestampWithLocalTimeZone
(batchId=356)
org.apache.hive.spark.client.rpc.TestRpc.testClientTimeout (batchId=353)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18241/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18241/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18241/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976546 - PreCommit-HIVE-Build
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21241:
--
Status: Open (was: Patch Available)
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-21637) Synchronized metastore cache
[ https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-21637: -- Attachment: HIVE-21637.56.patch > Synchronized metastore cache > > > Key: HIVE-21637 > URL: https://issues.apache.org/jira/browse/HIVE-21637 > Project: Hive > Issue Type: New Feature >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch, > HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch, > HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch, > HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch, > HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch, > HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.23.patch, > HIVE-21637.24.patch, HIVE-21637.25.patch, HIVE-21637.26.patch, > HIVE-21637.27.patch, HIVE-21637.28.patch, HIVE-21637.29.patch, > HIVE-21637.3.patch, HIVE-21637.30.patch, HIVE-21637.31.patch, > HIVE-21637.32.patch, HIVE-21637.33.patch, HIVE-21637.34.patch, > HIVE-21637.35.patch, HIVE-21637.36.patch, HIVE-21637.37.patch, > HIVE-21637.38.patch, HIVE-21637.39.patch, HIVE-21637.4.patch, > HIVE-21637.40.patch, HIVE-21637.41.patch, HIVE-21637.42.patch, > HIVE-21637.43.patch, HIVE-21637.44.patch, HIVE-21637.45.patch, > HIVE-21637.46.patch, HIVE-21637.47.patch, HIVE-21637.48.patch, > HIVE-21637.49.patch, HIVE-21637.5.patch, HIVE-21637.50.patch, > HIVE-21637.51.patch, HIVE-21637.52.patch, HIVE-21637.53.patch, > HIVE-21637.54.patch, HIVE-21637.55.patch, HIVE-21637.56.patch, > HIVE-21637.6.patch, HIVE-21637.7.patch, HIVE-21637.8.patch, HIVE-21637.9.patch > > > Currently, HMS has a cache implemented by CachedStore. The cache is > asynchronized and in HMS HA setting, we can only get eventual consistency. In > this Jira, we try to make it synchronized. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-22069) joda-time binary conflict between druid-handler and phoenix-hive jars.
[
https://issues.apache.org/jira/browse/HIVE-22069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898964#comment-16898964
]
David Mollitor commented on HIVE-22069:
---
{{arrow-vector}} dropped Joda time with [ARROW-2015]. Fixed in {{arrow-vector}}
{{0.13.0}}.
> joda-time binary conflict between druid-handler and phoenix-hive jars.
> --
>
> Key: HIVE-22069
> URL: https://issues.apache.org/jira/browse/HIVE-22069
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
>
> Hive's druid storage handler uses 2.8.1 version of the joda time library
> where as the phoenix-hive.jar uses 1.6 version of this library. When both
> jars are in the classpath, bad things happen.
> Apache phoenix has its own release cycle and them uptaking a new version is
> not what hive should count on. Besides they could decide to move to a new
> version of this library and we would still have this problem.
> So its best we use shaded jars in hive for the version we are on.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22068) Add more logging to notification cleaner and replication to track events
[
https://issues.apache.org/jira/browse/HIVE-22068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899150#comment-16899150
]
Hive QA commented on HIVE-22068:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976548/HIVE-22068.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16723 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18242/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18242/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18242/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976548 - PreCommit-HIVE-Build
> Add more logging to notification cleaner and replication to track events
>
>
> Key: HIVE-22068
> URL: https://issues.apache.org/jira/browse/HIVE-22068
> Project: Hive
> Issue Type: Improvement
>Reporter: Ashutosh Bapat
>Assignee: Ashutosh Bapat
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22068.01.patch, HIVE-22068.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> * Add more logging to DB notification listener cleaner thread
> ** The time when it considered cleaning, the interval and time before which
> events were cleared, the min and max id at that time
> ** how many events were cleared
> ** min and max id after the cleaning.
> * In REPL::START document the starting event, end event if specified and the
> maximum number of events, if specified.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899193#comment-16899193
]
Hive QA commented on HIVE-21637:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976550/HIVE-21637.56.patch
{color:green}SUCCESS:{color} +1 due to 124 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16717 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testNoStatsUpdateForTxnReplTable
(batchId=318)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18243/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18243/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18243/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976550 - PreCommit-HIVE-Build
> Synchronized metastore cache
>
>
> Key: HIVE-21637
> URL: https://issues.apache.org/jira/browse/HIVE-21637
> Project: Hive
> Issue Type: New Feature
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch,
> HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch,
> HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch,
> HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch,
> HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch,
> HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.23.patch,
> HIVE-21637.24.patch, HIVE-21637.25.patch, HIVE-21637.26.patch,
> HIVE-21637.27.patch, HIVE-21637.28.patch, HIVE-21637.29.patch,
> HIVE-21637.3.patch, HIVE-21637.30.patch, HIVE-21637.31.patch,
> HIVE-21637.32.patch, HIVE-21637.33.patch, HIVE-21637.34.patch,
> HIVE-21637.35.patch, HIVE-21637.36.patch, HIVE-21637.37.patch,
> HIVE-21637.38.patch, HIVE-21637.39.patch, HIVE-21637.4.patch,
> HIVE-21637.40.patch, HIVE-21637.41.patch, HIVE-21637.42.patch,
> HIVE-21637.43.patch, HIVE-21637.44.patch, HIVE-21637.45.patch,
> HIVE-21637.46.patch, HIVE-21637.47.patch, HIVE-21637.48.patch,
> HIVE-21637.49.patch, HIVE-21637.5.patch, HIVE-21637.50.patch,
> HIVE-21637.51.patch, HIVE-21637.52.patch, HIVE-21637.53.patch,
> HIVE-21637.54.patch, HIVE-21637.55.patch, HIVE-21637.56.patch,
> HIVE-21637.6.patch, HIVE-21637.7.patch, HIVE-21637.8.patch, HIVE-21637.9.patch
>
>
> Currently, HMS has a cache implemented by CachedStore. The cache is
> asynchronized and in HMS HA setting, we can only get eventual consistency. In
> this Jira, we try to make it synchronized.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899199#comment-16899199
]
Hive QA commented on HIVE-21637:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
31s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
30s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
16s{color} | {color:blue} standalone-metastore/metastore-server in master has
180 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} beeline in master has 44 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue}
standalone-metastore/metastore-tools/metastore-benchmarks in master has 3
extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/util in master has 44 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} storage-api: The patch generated 2 new + 15 unchanged
- 0 fixed = 17 total (was 15) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 9 new + 487 unchanged - 4 fixed = 496 total (was 491) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 178 new + 1910 unchanged - 65 fixed = 2088 total (was 1975) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
4s{color} | {color:red} ql: The patch generated 64 new + 2289 unchanged - 32
fixed = 2353 total (was 2321) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} standalone-metastore/metastore-tools/tools-common: The
patch generated 5 new + 31 unchanged - 0 fixed = 36 total (was 31) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} itests/hcatalog-unit: The
[jira] [Commented] (HIVE-22074) Slow compilation due to IN to OR transformation
[
https://issues.apache.org/jira/browse/HIVE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899318#comment-16899318
]
Hive QA commented on HIVE-22074:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 2 new + 260 unchanged - 1
fixed = 262 total (was 261) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18247/dev-support/hive-personality.sh
|
| git revision | master / d7475aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18247/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18247/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Slow compilation due to IN to OR transformation
> ---
>
> Key: HIVE-22074
> URL: https://issues.apache.org/jira/browse/HIVE-22074
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22074.1.patch, HIVE-22074.2.patch
>
>
> Currently Hive transform IN expressions to OR to apply various CBO rules.
> This incur significant performance hit if IN consist of large number of
> expressions.
> It is better to not transform IN expressions to OR in such cases because
> overall benefit of various optimizations/transformations is unrealized due to
> the compilation overhead
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22080) Prevent implicit conversion from String/char/varchar to double/decimal
[
https://issues.apache.org/jira/browse/HIVE-22080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899353#comment-16899353
]
Hive QA commented on HIVE-22080:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
30s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} standalone-metastore/metastore-common: The patch
generated 0 new + 70 unchanged - 1 fixed = 70 total (was 71) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18249/dev-support/hive-personality.sh
|
| git revision | master / d7475aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18249/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Prevent implicit conversion from String/char/varchar to double/decimal
> --
>
> Key: HIVE-22080
> URL: https://issues.apache.org/jira/browse/HIVE-22080
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22080.1.patch
>
>
> Implicit conversion from String family types to any non-string family types
> are invalid. User can force the conversion by turning off the setting
> hive.metastore.disallow.incompatible.col.type.changes. If not turned off,
> such a conversion should throw error.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22080) Prevent implicit conversion from String/char/varchar to double/decimal
[
https://issues.apache.org/jira/browse/HIVE-22080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899357#comment-16899357
]
Hive QA commented on HIVE-22080:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976590/HIVE-22080.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 40 failed/errored test(s), 16723 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck]
(batchId=46)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_part]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_part_llap_io]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_table]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_table_llap_io]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_part_llap_io]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_table]
(batchId=185)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acidvec_table_llap_io]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_part]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_part_llap_io]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_table]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_nonvec_table_llap_io]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_part]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_table]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_vec_table_llap_io]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_part]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_part_llap_io]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_table]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_nonvec_table_llap_io]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_table_llap_io]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_part]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_part_llap_io]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_table]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vecrow_table_llap_io]
(batchId=170)
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=232)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=232)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18249/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18249/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18249/
Messages:
{noformat}
[jira] [Commented] (HIVE-22074) Slow compilation due to IN to OR transformation
[
https://issues.apache.org/jira/browse/HIVE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899326#comment-16899326
]
Hive QA commented on HIVE-22074:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976580/HIVE-22074.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 282 failed/errored test(s), 16723 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_insert_overwrite]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_directories_test]
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_simple_select]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_simple_select]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_3]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_groupby]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_cond_pushdown]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_in_or_dup]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_multi_single_reducer3]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[implicit_cast_during_insert]
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_char]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_mixed]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_varchar]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input49] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[interval_alt] (batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join45] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join47] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin47] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_buckets] (batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_ppd_char]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_vectorization_ppd]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_char]
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_date]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_decimal]
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_timestamp]
(batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_varchar]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_condition_remover]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[partition_type_in_plan]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcs] (batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pointlookup5]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_deterministic_expr]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[remove_exprs_stats]
(batchId=97)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[selectindate]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_47]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stat_estimate_drill]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats_part2] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[structin] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_in] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_notop] (batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_date_1]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_mr_diff_schema_alias]
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_non_constant_in_expr]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_struct_in]
(batchId=50)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_parquet_ppd_decimal]
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_context]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_mapjoin3]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_timestamp]
(batchId=85)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_partition_static]
(batchId=187)
org.apache.hadoop.hive.cli.TestKuduCliDriver.testCliDriver[kudu_queries]
(batchId=287)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_ts]
(batchId=197)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899334#comment-16899334
]
Hive QA commented on HIVE-22079:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 7 new + 16 unchanged - 0 fixed
= 23 total (was 16) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18248/dev-support/hive-personality.sh
|
| git revision | master / d7475aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18248/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18248/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-21637) Synchronized metastore cache
[ https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-21637: -- Attachment: HIVE-21637.58.patch > Synchronized metastore cache > > > Key: HIVE-21637 > URL: https://issues.apache.org/jira/browse/HIVE-21637 > Project: Hive > Issue Type: New Feature >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch, > HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch, > HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch, > HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch, > HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch, > HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.23.patch, > HIVE-21637.24.patch, HIVE-21637.25.patch, HIVE-21637.26.patch, > HIVE-21637.27.patch, HIVE-21637.28.patch, HIVE-21637.29.patch, > HIVE-21637.3.patch, HIVE-21637.30.patch, HIVE-21637.31.patch, > HIVE-21637.32.patch, HIVE-21637.33.patch, HIVE-21637.34.patch, > HIVE-21637.35.patch, HIVE-21637.36.patch, HIVE-21637.37.patch, > HIVE-21637.38.patch, HIVE-21637.39.patch, HIVE-21637.4.patch, > HIVE-21637.40.patch, HIVE-21637.41.patch, HIVE-21637.42.patch, > HIVE-21637.43.patch, HIVE-21637.44.patch, HIVE-21637.45.patch, > HIVE-21637.46.patch, HIVE-21637.47.patch, HIVE-21637.48.patch, > HIVE-21637.49.patch, HIVE-21637.5.patch, HIVE-21637.50.patch, > HIVE-21637.51.patch, HIVE-21637.52.patch, HIVE-21637.53.patch, > HIVE-21637.54.patch, HIVE-21637.55.patch, HIVE-21637.56.patch, > HIVE-21637.57.patch, HIVE-21637.58.patch, HIVE-21637.6.patch, > HIVE-21637.7.patch, HIVE-21637.8.patch, HIVE-21637.9.patch > > > Currently, HMS has a cache implemented by CachedStore. The cache is > asynchronized and in HMS HA setting, we can only get eventual consistency. In > this Jira, we try to make it synchronized. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-22081) Hivemetastore Performance: Compaction Initiator Thread overwhelmed if there are too many Table/partitions are eligible for compaction
[
https://issues.apache.org/jira/browse/HIVE-22081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899363#comment-16899363
]
Hive QA commented on HIVE-22081:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
9s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 15 new + 23 unchanged - 2
fixed = 38 total (was 25) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18250/dev-support/hive-personality.sh
|
| git revision | master / d7475aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18250/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18250/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hivemetastore Performance: Compaction Initiator Thread overwhelmed if there
> are too many Table/partitions are eligible for compaction
> --
>
> Key: HIVE-22081
> URL: https://issues.apache.org/jira/browse/HIVE-22081
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Rajkumar Singh
>Assignee: Rajkumar Singh
>Priority: Major
> Attachments: HIVE-22081.patch
>
>
> if Automatic Compaction is turned on, Initiator thread check for potential
> table/partitions which are eligible for compactions and run some checks in
> for loop before requesting compaction for eligibles. Though initiator thread
> is configured to run at interval 5 min default, in case of many objects it
> keeps on running as these checks are IO intensive and hog cpu.
> In the proposed changes, I am planning to do
> 1. passing less object to for loop by filtering out the objects based on the
> condition which we are checking within the loop.
> 2. Doing Async call using future to determine compaction type(this is where
> we do FileSystem calls)
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899344#comment-16899344
]
Hive QA commented on HIVE-22079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976586/HIVE-22079.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 231 failed/errored test(s), 16723 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bool_unknown]
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_join0]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_subq_exists]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constant_prop_3]
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog_partitioner]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_1] (batchId=95)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_unqual1]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_unqual2]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_cond_pushdown_unqual4]
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[keep_uniform]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_12] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_3] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_4] (batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonmr_fetch] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin4] (batchId=95)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[semijoin5] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subq_where_serialization]
(batchId=96)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_exists]
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_exists_having]
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notexists]
(batchId=99)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notexists_having]
(batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notin_having]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_unqualcolumnrefs]
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_subquery1]
(batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_mapjoin]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_mapjoin3]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[view_cbo] (batchId=75)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_gby]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_join]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_gby]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_join]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_semijoin]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_semijoin]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_exists]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_in]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_subq_not_in]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cte_1]
(batchId=185)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_partition_pruning]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_2]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[external_jdbc_table_perf]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[get_splits_0]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_groupingset_bug]
(batchId=184)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[keep_uniform]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage3]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mapjoin_hint]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[optimize_join_ptp]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
(batchId=179)
[jira] [Updated] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21241:
--
Status: Patch Available (was: Open)
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch, HIVE-21241.4.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21241:
--
Status: Open (was: Patch Available)
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch, HIVE-21241.4.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-21241:
--
Attachment: HIVE-21241.4.patch
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch, HIVE-21241.4.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-21637) Synchronized metastore cache
[ https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Dai updated HIVE-21637: -- Attachment: HIVE-21637.57.patch > Synchronized metastore cache > > > Key: HIVE-21637 > URL: https://issues.apache.org/jira/browse/HIVE-21637 > Project: Hive > Issue Type: New Feature >Reporter: Daniel Dai >Assignee: Daniel Dai >Priority: Major > Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch, > HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch, > HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch, > HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch, > HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch, > HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.23.patch, > HIVE-21637.24.patch, HIVE-21637.25.patch, HIVE-21637.26.patch, > HIVE-21637.27.patch, HIVE-21637.28.patch, HIVE-21637.29.patch, > HIVE-21637.3.patch, HIVE-21637.30.patch, HIVE-21637.31.patch, > HIVE-21637.32.patch, HIVE-21637.33.patch, HIVE-21637.34.patch, > HIVE-21637.35.patch, HIVE-21637.36.patch, HIVE-21637.37.patch, > HIVE-21637.38.patch, HIVE-21637.39.patch, HIVE-21637.4.patch, > HIVE-21637.40.patch, HIVE-21637.41.patch, HIVE-21637.42.patch, > HIVE-21637.43.patch, HIVE-21637.44.patch, HIVE-21637.45.patch, > HIVE-21637.46.patch, HIVE-21637.47.patch, HIVE-21637.48.patch, > HIVE-21637.49.patch, HIVE-21637.5.patch, HIVE-21637.50.patch, > HIVE-21637.51.patch, HIVE-21637.52.patch, HIVE-21637.53.patch, > HIVE-21637.54.patch, HIVE-21637.55.patch, HIVE-21637.56.patch, > HIVE-21637.57.patch, HIVE-21637.6.patch, HIVE-21637.7.patch, > HIVE-21637.8.patch, HIVE-21637.9.patch > > > Currently, HMS has a cache implemented by CachedStore. The cache is > asynchronized and in HMS HA setting, we can only get eventual consistency. In > this Jira, we try to make it synchronized. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-22079:
--
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-22081) Hivemetastore Performance: Compaction Initiator Thread overwhelmed if there are too many Table/partitions are eligible for compaction
[ https://issues.apache.org/jira/browse/HIVE-22081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-22081: -- Summary: Hivemetastore Performance: Compaction Initiator Thread overwhelmed if there are too many Table/partitions are eligible for compaction (was: Hivemetastore Performance: Compaction Initiator thread overwhelmed if no there are too many Table/partitions are eligible for compaction ) > Hivemetastore Performance: Compaction Initiator Thread overwhelmed if there > are too many Table/partitions are eligible for compaction > -- > > Key: HIVE-22081 > URL: https://issues.apache.org/jira/browse/HIVE-22081 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-22081.patch > > > if Automatic Compaction is turned on, Initiator thread check for potential > table/partitions which are eligible for compactions and run some checks in > for loop before requesting compaction for eligibles. Though initiator thread > is configured to run at interval 5 min default, in case of many objects it > keeps on running as these checks are IO intensive and hog cpu. > In the proposed changes, I am planning to do > 1. passing less object to for loop by filtering out the objects based on the > condition which we are checking within the loop. > 2. Doing Async call using future to determine compaction type(this is where > we do FileSystem calls) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899306#comment-16899306
]
Hive QA commented on HIVE-21637:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976579/HIVE-21637.57.patch
{color:green}SUCCESS:{color} +1 due to 124 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 16718 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.org.apache.hadoop.hive.ql.TestWarehouseExternalDir
(batchId=273)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testExternalDefaultPaths
(batchId=273)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testNoStatsUpdateForTxnReplTable
(batchId=318)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18246/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18246/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18246/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976579 - PreCommit-HIVE-Build
> Synchronized metastore cache
>
>
> Key: HIVE-21637
> URL: https://issues.apache.org/jira/browse/HIVE-21637
> Project: Hive
> Issue Type: New Feature
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch,
> HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch,
> HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch,
> HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch,
> HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch,
> HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.23.patch,
> HIVE-21637.24.patch, HIVE-21637.25.patch, HIVE-21637.26.patch,
> HIVE-21637.27.patch, HIVE-21637.28.patch, HIVE-21637.29.patch,
> HIVE-21637.3.patch, HIVE-21637.30.patch, HIVE-21637.31.patch,
> HIVE-21637.32.patch, HIVE-21637.33.patch, HIVE-21637.34.patch,
> HIVE-21637.35.patch, HIVE-21637.36.patch, HIVE-21637.37.patch,
> HIVE-21637.38.patch, HIVE-21637.39.patch, HIVE-21637.4.patch,
> HIVE-21637.40.patch, HIVE-21637.41.patch, HIVE-21637.42.patch,
> HIVE-21637.43.patch, HIVE-21637.44.patch, HIVE-21637.45.patch,
> HIVE-21637.46.patch, HIVE-21637.47.patch, HIVE-21637.48.patch,
> HIVE-21637.49.patch, HIVE-21637.5.patch, HIVE-21637.50.patch,
> HIVE-21637.51.patch, HIVE-21637.52.patch, HIVE-21637.53.patch,
> HIVE-21637.54.patch, HIVE-21637.55.patch, HIVE-21637.56.patch,
> HIVE-21637.57.patch, HIVE-21637.6.patch, HIVE-21637.7.patch,
> HIVE-21637.8.patch, HIVE-21637.9.patch
>
>
> Currently, HMS has a cache implemented by CachedStore. The cache is
> asynchronized and in HMS HA setting, we can only get eventual consistency. In
> this Jira, we try to make it synchronized.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22079:
---
Status: Patch Available (was: Open)
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22079:
---
Attachment: HIVE-22079.1.patch
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899276#comment-16899276
]
Hive QA commented on HIVE-21241:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976562/HIVE-21241.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16731 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.serde2.TestJsonSerDe.testTimestampWithLocalTimeZone
(batchId=356)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18245/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18245/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18245/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976562 - PreCommit-HIVE-Build
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch, HIVE-21241.4.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-22074) Slow compilation due to IN to OR transformation
[ https://issues.apache.org/jira/browse/HIVE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22074: --- Attachment: HIVE-22074.2.patch > Slow compilation due to IN to OR transformation > --- > > Key: HIVE-22074 > URL: https://issues.apache.org/jira/browse/HIVE-22074 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22074.1.patch, HIVE-22074.2.patch > > > Currently Hive transform IN expressions to OR to apply various CBO rules. > This incur significant performance hit if IN consist of large number of > expressions. > It is better to not transform IN expressions to OR in such cases because > overall benefit of various optimizations/transformations is unrealized due to > the compilation overhead -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22074) Slow compilation due to IN to OR transformation
[ https://issues.apache.org/jira/browse/HIVE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22074: --- Status: Patch Available (was: Open) > Slow compilation due to IN to OR transformation > --- > > Key: HIVE-22074 > URL: https://issues.apache.org/jira/browse/HIVE-22074 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-22074.1.patch, HIVE-22074.2.patch > > > Currently Hive transform IN expressions to OR to apply various CBO rules. > This incur significant performance hit if IN consist of large number of > expressions. > It is better to not transform IN expressions to OR in such cases because > overall benefit of various optimizations/transformations is unrealized due to > the compilation overhead -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-21241) Migrate TimeStamp Parser From Joda Time
[
https://issues.apache.org/jira/browse/HIVE-21241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899245#comment-16899245
]
Hive QA commented on HIVE-21241:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} common: The patch generated 0 new + 0 unchanged - 14
fixed = 0 total (was 14) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s{color} | {color:green} common generated 0 new + 61 unchanged - 1 fixed = 61
total (was 62) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} common generated 0 new + 26 unchanged - 1 fixed = 26
total (was 27) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18245/dev-support/hive-personality.sh
|
| git revision | master / d7475aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common U: common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18245/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Migrate TimeStamp Parser From Joda Time
> ---
>
> Key: HIVE-21241
> URL: https://issues.apache.org/jira/browse/HIVE-21241
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
>Affects Versions: 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-21241.1.patch, HIVE-21241.2.patch,
> HIVE-21241.3.patch, HIVE-21241.4.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive uses Joda time for its TimeStampParser.
> {quote}
> Joda-Time is the de facto standard date and time library for Java prior to
> Java SE 8. Users are now asked to migrate to java.time (JSR-310).
> https://www.joda.org/joda-time/
> {quote}
> Migrate TimeStampParser to {{java.time}}
> I also added a couple new pre-canned timestamp parsers for convenience:
> * ISO 8601
> * RFC 1123
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-22080) Prevent implicit conversion from String/char/varchar to double/decimal
[ https://issues.apache.org/jira/browse/HIVE-22080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22080: Attachment: HIVE-22080.1.patch > Prevent implicit conversion from String/char/varchar to double/decimal > -- > > Key: HIVE-22080 > URL: https://issues.apache.org/jira/browse/HIVE-22080 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22080.1.patch > > > Implicit conversion from String family types to any non-string family types > are invalid. User can force the conversion by turning off the setting > hive.metastore.disallow.incompatible.col.type.changes. If not turned off, > such a conversion should throw error. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899212#comment-16899212
]
Hive QA commented on HIVE-21198:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
13s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} beeline in master has 44 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} hcatalog/core in master has 29 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} hcatalog/webhcat/java-client in master has 3 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 11s{color}
| {color:red} storage-api generated 2 new + 0 unchanged - 2 fixed = 2 total
(was 2) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} storage-api: The patch generated 0 new + 4 unchanged
- 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
56s{color} | {color:red} ql: The patch generated 23 new + 1782 unchanged - 64
fixed = 1805 total (was 1846) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch accumulo-handler passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch beeline passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch core passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch java-client passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
21s{color} | {color:red} ql generated 4 new + 2248 unchanged - 2 fixed = 2252
total (was 2250) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m
2s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total
(was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 45m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to table in
[jira] [Commented] (HIVE-21198) Introduce a database object reference class
[
https://issues.apache.org/jira/browse/HIVE-21198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899233#comment-16899233
]
Hive QA commented on HIVE-21198:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976558/HIVE-21198.8.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 288 failed/errored test(s), 16724 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=288)
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert]
(batchId=288)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_blobstore_to_blobstore]
(batchId=299)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_blobstore_to_hdfs]
(batchId=299)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_hdfs_to_blobstore]
(batchId=299)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[allcolref_in_udf]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[allow_change_col_type_par]
(batchId=79)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_table]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_7]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_alter_table_update_columns]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_SortUnionTransposeRule]
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_union_table]
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cross_product_check_1]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cross_product_check_2]
(batchId=98)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_colname]
(batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_uses_database_location]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_3] (batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_3] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_4] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_5] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_join2]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_serde]
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[distinct_66] (batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[exec_parallel_column_stats]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_ddl] (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fileformat_sequencefile]
(batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fileformat_text]
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_duplicate_key]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input10] (batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input15] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input1] (batchId=76)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input3] (batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[inputddl1] (batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[inputddl2] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[inputddl3] (batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[inputddl6] (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join41] (batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join42] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_filters_overlap]
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[lateral_view_outer]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_10] (batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_12] (batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_13] (batchId=81)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_1] (batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_1_newdb]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_3] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_4] (batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_5] (batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_8] (batchId=8)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_9] (batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_disablecbo_1]
[jira] [Updated] (HIVE-22080) Prevent implicit conversion from String/char/varchar to double/decimal
[ https://issues.apache.org/jira/browse/HIVE-22080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22080: Attachment: (was: DWX-684_1.patch) > Prevent implicit conversion from String/char/varchar to double/decimal > -- > > Key: HIVE-22080 > URL: https://issues.apache.org/jira/browse/HIVE-22080 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Minor > Fix For: 4.0.0 > > > Implicit conversion from String family types to any non-string family types > are invalid. User can force the conversion by turning off the setting > hive.metastore.disallow.incompatible.col.type.changes. If not turned off, > such a conversion should throw error. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22081) Hivemetastore Performance: Compaction Initiator thread overwhelmed if no there are too many Table/partitions are eligible for compaction
[ https://issues.apache.org/jira/browse/HIVE-22081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh updated HIVE-22081: -- Attachment: HIVE-22081.patch Status: Patch Available (was: Open) > Hivemetastore Performance: Compaction Initiator thread overwhelmed if no > there are too many Table/partitions are eligible for compaction > - > > Key: HIVE-22081 > URL: https://issues.apache.org/jira/browse/HIVE-22081 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > Attachments: HIVE-22081.patch > > > if Automatic Compaction is turned on, Initiator thread check for potential > table/partitions which are eligible for compactions and run some checks in > for loop before requesting compaction for eligibles. Though initiator thread > is configured to run at interval 5 min default, in case of many objects it > keeps on running as these checks are IO intensive and hog cpu. > In the proposed changes, I am planning to do > 1. passing less object to for loop by filtering out the objects based on the > condition which we are checking within the loop. > 2. Doing Async call using future to determine compaction type(this is where > we do FileSystem calls) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (HIVE-22081) Hivemetastore Performance: Compaction Initiator thread overwhelmed if no there are too many Table/partitions are eligible for compaction
[ https://issues.apache.org/jira/browse/HIVE-22081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajkumar Singh reassigned HIVE-22081: - > Hivemetastore Performance: Compaction Initiator thread overwhelmed if no > there are too many Table/partitions are eligible for compaction > - > > Key: HIVE-22081 > URL: https://issues.apache.org/jira/browse/HIVE-22081 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Major > > if Automatic Compaction is turned on, Initiator thread check for potential > table/partitions which are eligible for compactions and run some checks in > for loop before requesting compaction for eligibles. Though initiator thread > is configured to run at interval 5 min default, in case of many objects it > keeps on running as these checks are IO intensive and hog cpu. > In the proposed changes, I am planning to do > 1. passing less object to for loop by filtering out the objects based on the > condition which we are checking within the loop. > 2. Doing Async call using future to determine compaction type(this is where > we do FileSystem calls) -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16899311#comment-16899311
]
Hive QA commented on HIVE-21637:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
32s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
27s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
180 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} beeline in master has 44 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue}
standalone-metastore/metastore-tools/metastore-benchmarks in master has 3
extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
49s{color} | {color:blue} itests/util in master has 44 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} storage-api: The patch generated 2 new + 15 unchanged
- 0 fixed = 17 total (was 15) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 9 new + 487 unchanged - 4 fixed = 496 total (was 491) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 178 new + 1910 unchanged - 65 fixed = 2088 total (was 1975) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
2s{color} | {color:red} ql: The patch generated 64 new + 2289 unchanged - 32
fixed = 2353 total (was 2321) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} standalone-metastore/metastore-tools/tools-common: The
patch generated 5 new + 31 unchanged - 0 fixed = 36 total (was 31) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} itests/hcatalog-unit: The
[jira] [Updated] (HIVE-22080) Prevent implicit conversion from String/char/varchar to double/decimal
[ https://issues.apache.org/jira/browse/HIVE-22080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22080: Attachment: HIVE-22080.1.patch Status: Patch Available (was: Open) > Prevent implicit conversion from String/char/varchar to double/decimal > -- > > Key: HIVE-22080 > URL: https://issues.apache.org/jira/browse/HIVE-22080 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22080.1.patch > > > Implicit conversion from String family types to any non-string family types > are invalid. User can force the conversion by turning off the setting > hive.metastore.disallow.incompatible.col.type.changes. If not turned off, > such a conversion should throw error. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22080) Prevent implicit conversion from String/char/varchar to double/decimal
[ https://issues.apache.org/jira/browse/HIVE-22080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22080: Attachment: (was: HIVE-22080.1.patch) > Prevent implicit conversion from String/char/varchar to double/decimal > -- > > Key: HIVE-22080 > URL: https://issues.apache.org/jira/browse/HIVE-22080 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Minor > Fix For: 4.0.0 > > > Implicit conversion from String family types to any non-string family types > are invalid. User can force the conversion by turning off the setting > hive.metastore.disallow.incompatible.col.type.changes. If not turned off, > such a conversion should throw error. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (HIVE-22080) Prevent implicit conversion from String/char/varchar to double/decimal
[ https://issues.apache.org/jira/browse/HIVE-22080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan reassigned HIVE-22080: --- > Prevent implicit conversion from String/char/varchar to double/decimal > -- > > Key: HIVE-22080 > URL: https://issues.apache.org/jira/browse/HIVE-22080 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Minor > Fix For: 4.0.0 > > Attachments: DWX-684_1.patch > > > Implicit conversion from String family types to any non-string family types > are invalid. User can force the conversion by turning off the setting > hive.metastore.disallow.incompatible.col.type.changes. If not turned off, > such a conversion should throw error. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22080) Prevent implicit conversion from String/char/varchar to double/decimal
[ https://issues.apache.org/jira/browse/HIVE-22080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22080: Attachment: DWX-684_1.patch > Prevent implicit conversion from String/char/varchar to double/decimal > -- > > Key: HIVE-22080 > URL: https://issues.apache.org/jira/browse/HIVE-22080 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Minor > Fix For: 4.0.0 > > Attachments: DWX-684_1.patch > > > Implicit conversion from String family types to any non-string family types > are invalid. User can force the conversion by turning off the setting > hive.metastore.disallow.incompatible.col.type.changes. If not turned off, > such a conversion should throw error. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-22075) Fix the max-reducers=1 regression from HIVE-14200
[
https://issues.apache.org/jira/browse/HIVE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898901#comment-16898901
]
Hive QA commented on HIVE-22075:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976497/HIVE-22075.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16723 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18239/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18239/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18239/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976497 - PreCommit-HIVE-Build
> Fix the max-reducers=1 regression from HIVE-14200
> -
>
> Key: HIVE-22075
> URL: https://issues.apache.org/jira/browse/HIVE-22075
> Project: Hive
> Issue Type: Bug
> Components: llap, Physical Optimizer
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Jesus Camacho Rodriguez
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22075.1.patch
>
>
> The condition does not kick in when
> {code}
> minPartition=1, maxPartition=1, nReducers=1, maxReducers=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[
https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898903#comment-16898903
]
Hive QA commented on HIVE-21828:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976504/HIVE-21828.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18240/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18240/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18240/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-08-02 13:48:09.038
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18240/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-08-02 13:48:09.041
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at d7475aa HIVE-22008: LIKE Operator should match multi-line input
(Gopal V, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at d7475aa HIVE-22008: LIKE Operator should match multi-line input
(Gopal V, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-08-02 13:48:09.642
+ rm -rf ../yetus_PreCommit-HIVE-Build-18240
+ mkdir ../yetus_PreCommit-HIVE-Build-18240
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18240
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18240/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: a/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java: does not
exist in index
error:
a/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRegExp.java: does
not exist in index
error: a/ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezTask.java: does
not exist in index
error: a/ql/src/test/org/apache/hive/testutils/HiveTestEnvSetup.java: does not
exist in index
error: a/ql/src/test/queries/clientpositive/mm_loaddata.q: does not exist in
index
error:
a/ql/src/test/results/clientpositive/llap/tez_fixed_bucket_pruning.q.out: does
not exist in index
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:5432
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java:44
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java'
cleanly.
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezConfigurationFactory.java'
cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java:174
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezTask.java'
with conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRegExp.java:64
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFRegExp.java'
cleanly.
error: patch failed:
ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezTask.java:28
Falling back to three-way merge...
Applied patch to
[jira] [Updated] (HIVE-22008) LIKE Operator should match multi-line input
[
https://issues.apache.org/jira/browse/HIVE-22008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-22008:
---
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
> LIKE Operator should match multi-line input
>
>
> Key: HIVE-22008
> URL: https://issues.apache.org/jira/browse/HIVE-22008
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0, 2.3.4
>Reporter: Shankar
>Assignee: Gopal V
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22008.1.patch, HIVE-22008.2.patch
>
>
> I am facing some issues while using *Like* operator & *newline* (\n)
> character. Below is the in details description :
>
>
> {color:#263238}*-- Hive
> Queries *{color}
> {color:#263238} – consider these are the reproduce steps.{color}
>
> {color:#263238}create table default.withdraw({color}
> {color:#263238}id string{color}
> {color:#263238}) stored as parquet;{color}
> {color:#263238} {color}
>
> {color:#263238}
> *insert into default.withdraw select 'withdraw\ncash';*{color}
> -- note here, added '\n' character
>
> *{color:#263238}--1) result = {color}{color:#6aa84f}success{color}*
> {color:#263238}hive> select * from default.withdraw where id like
> '%withdraw%';{color}
> {color:#263238}OK{color}
> {color:#263238}withdraw{color}
> {color:#263238}cash{color}
> {color:#263238}Time taken: 0.078 seconds, Fetched: *1 row(s)*{color}
>
> *--2)* *{color:#263238}result = {color}{color:#cc}wrong{color}*
> {color:#263238}hive> select * from default.withdraw where id like
> '%withdraw%cash';{color}
> {color:#263238}OK{color}
> {color:#263238}Time taken: 0.066 seconds{color}
>
> *--3)* *{color:#263238}result = {color}{color:#6aa84f}success{color}*
> {color:#263238}hive> select * from default.withdraw where id like
> '%cash%';{color}
> {color:#263238}OK{color}
> {color:#263238}withdraw{color}
> {color:#263238}cash{color}
> {color:#263238}Time taken: 0.086 seconds, Fetched: 1 row(s){color}
>
>
>
>
> {color:#263238}*-- Presto
> Queries -*{color}
> {color:#263238}FYI - Presto (v0.221) is using above table meta store. We
> tested above queries on presto too. {color}
> {color:#263238} {color}
>
> *--1)* *{color:#263238}result ={color}* *{color:#6aa84f}success{color}*
> presto> select * from default.withdraw where id like '%withdraw%';
> id
> --
> withdraw
> cash
> (1 row)
>
> *--2)* *{color:#263238}result ={color}* *{color:#6aa84f}success{color}*
> presto> select * from default.withdraw where id like '%withdraw%cash';
> id
> --
> withdraw
> cash
> (1 row)
>
> *--3)* *{color:#263238}result ={color}* *{color:#6aa84f}success{color}*
> presto> select * from default.withdraw where id like '%cash%';
> id
> --
> withdraw
> cash
> (1 row){color:#263238}
> {color}
>
>
> *--*
> *--*
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (HIVE-22076) JDK11: Remove ParallelGC in debug.sh
[ https://issues.apache.org/jira/browse/HIVE-22076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V reassigned HIVE-22076: -- Assignee: Gopal V > JDK11: Remove ParallelGC in debug.sh > > > Key: HIVE-22076 > URL: https://issues.apache.org/jira/browse/HIVE-22076 > Project: Hive > Issue Type: Bug > Components: Diagnosability >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > > The JDK debug mode no longer depends on ParallelGC -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22076) JDK11: Remove ParallelGC in debug.sh
[ https://issues.apache.org/jira/browse/HIVE-22076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-22076: --- Description: The JDK debug mode no longer depends on ParallelGC This was a workaround for JDK6 bug - https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6862295 was:The JDK debug mode no longer depends on ParallelGC > JDK11: Remove ParallelGC in debug.sh > > > Key: HIVE-22076 > URL: https://issues.apache.org/jira/browse/HIVE-22076 > Project: Hive > Issue Type: Bug > Components: Diagnosability >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > > The JDK debug mode no longer depends on ParallelGC > This was a workaround for JDK6 bug - > https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6862295 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22075) Fix the max-reducers=1 regression from HIVE-14200
[
https://issues.apache.org/jira/browse/HIVE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-22075:
---
Description:
The condition does not kick in when
{code}
minPartition=1, maxPartition=1, nReducers=1, maxReducers=1
{code}
> Fix the max-reducers=1 regression from HIVE-14200
> -
>
> Key: HIVE-22075
> URL: https://issues.apache.org/jira/browse/HIVE-22075
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer, Tez
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Jesus Camacho Rodriguez
>Assignee: Gopal V
>Priority: Major
>
> The condition does not kick in when
> {code}
> minPartition=1, maxPartition=1, nReducers=1, maxReducers=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-22076) JDK11: Remove ParallelGC in debug.sh
[ https://issues.apache.org/jira/browse/HIVE-22076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-22076: --- Status: Patch Available (was: Open) > JDK11: Remove ParallelGC in debug.sh > > > Key: HIVE-22076 > URL: https://issues.apache.org/jira/browse/HIVE-22076 > Project: Hive > Issue Type: Bug > Components: Diagnosability >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Attachments: HIVE-22076.1.patch > > > The JDK debug mode no longer depends on ParallelGC > This was a workaround for JDK6 bug - > https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6862295 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22076) JDK11: Remove ParallelGC in debug.sh
[ https://issues.apache.org/jira/browse/HIVE-22076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-22076: --- Attachment: HIVE-22076.1.patch > JDK11: Remove ParallelGC in debug.sh > > > Key: HIVE-22076 > URL: https://issues.apache.org/jira/browse/HIVE-22076 > Project: Hive > Issue Type: Bug > Components: Diagnosability >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Minor > Attachments: HIVE-22076.1.patch > > > The JDK debug mode no longer depends on ParallelGC > This was a workaround for JDK6 bug - > https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6862295 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-22008) LIKE Operator should match multi-line input
[
https://issues.apache.org/jira/browse/HIVE-22008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-22008:
---
Summary: LIKE Operator should match multi-line input (was: Apache Hive
2.3.4 - Issue with combination of Like operator & newline (\n) character in
data)
> LIKE Operator should match multi-line input
>
>
> Key: HIVE-22008
> URL: https://issues.apache.org/jira/browse/HIVE-22008
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0, 2.3.4
>Reporter: Shankar
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22008.1.patch, HIVE-22008.2.patch
>
>
> I am facing some issues while using *Like* operator & *newline* (\n)
> character. Below is the in details description :
>
>
> {color:#263238}*-- Hive
> Queries *{color}
> {color:#263238} – consider these are the reproduce steps.{color}
>
> {color:#263238}create table default.withdraw({color}
> {color:#263238}id string{color}
> {color:#263238}) stored as parquet;{color}
> {color:#263238} {color}
>
> {color:#263238}
> *insert into default.withdraw select 'withdraw\ncash';*{color}
> -- note here, added '\n' character
>
> *{color:#263238}--1) result = {color}{color:#6aa84f}success{color}*
> {color:#263238}hive> select * from default.withdraw where id like
> '%withdraw%';{color}
> {color:#263238}OK{color}
> {color:#263238}withdraw{color}
> {color:#263238}cash{color}
> {color:#263238}Time taken: 0.078 seconds, Fetched: *1 row(s)*{color}
>
> *--2)* *{color:#263238}result = {color}{color:#cc}wrong{color}*
> {color:#263238}hive> select * from default.withdraw where id like
> '%withdraw%cash';{color}
> {color:#263238}OK{color}
> {color:#263238}Time taken: 0.066 seconds{color}
>
> *--3)* *{color:#263238}result = {color}{color:#6aa84f}success{color}*
> {color:#263238}hive> select * from default.withdraw where id like
> '%cash%';{color}
> {color:#263238}OK{color}
> {color:#263238}withdraw{color}
> {color:#263238}cash{color}
> {color:#263238}Time taken: 0.086 seconds, Fetched: 1 row(s){color}
>
>
>
>
> {color:#263238}*-- Presto
> Queries -*{color}
> {color:#263238}FYI - Presto (v0.221) is using above table meta store. We
> tested above queries on presto too. {color}
> {color:#263238} {color}
>
> *--1)* *{color:#263238}result ={color}* *{color:#6aa84f}success{color}*
> presto> select * from default.withdraw where id like '%withdraw%';
> id
> --
> withdraw
> cash
> (1 row)
>
> *--2)* *{color:#263238}result ={color}* *{color:#6aa84f}success{color}*
> presto> select * from default.withdraw where id like '%withdraw%cash';
> id
> --
> withdraw
> cash
> (1 row)
>
> *--3)* *{color:#263238}result ={color}* *{color:#6aa84f}success{color}*
> presto> select * from default.withdraw where id like '%cash%';
> id
> --
> withdraw
> cash
> (1 row){color:#263238}
> {color}
>
>
> *--*
> *--*
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22076) JDK11: Remove ParallelGC in debug.sh
[
https://issues.apache.org/jira/browse/HIVE-22076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898802#comment-16898802
]
Hive QA commented on HIVE-22076:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 0m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18238/dev-support/hive-personality.sh
|
| git revision | master / d7475aa |
| modules | C: . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18238/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> JDK11: Remove ParallelGC in debug.sh
>
>
> Key: HIVE-22076
> URL: https://issues.apache.org/jira/browse/HIVE-22076
> Project: Hive
> Issue Type: Bug
> Components: Diagnosability
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Minor
> Attachments: HIVE-22076.1.patch
>
>
> The JDK debug mode no longer depends on ParallelGC
> This was a workaround for JDK6 bug -
> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6862295
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898797#comment-16898797
]
Hive QA commented on HIVE-21637:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976488/HIVE-21637.55.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18237/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18237/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18237/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12976488/HIVE-21637.55.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976488 - PreCommit-HIVE-Build
> Synchronized metastore cache
>
>
> Key: HIVE-21637
> URL: https://issues.apache.org/jira/browse/HIVE-21637
> Project: Hive
> Issue Type: New Feature
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch,
> HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch,
> HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch,
> HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch,
> HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch,
> HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.23.patch,
> HIVE-21637.24.patch, HIVE-21637.25.patch, HIVE-21637.26.patch,
> HIVE-21637.27.patch, HIVE-21637.28.patch, HIVE-21637.29.patch,
> HIVE-21637.3.patch, HIVE-21637.30.patch, HIVE-21637.31.patch,
> HIVE-21637.32.patch, HIVE-21637.33.patch, HIVE-21637.34.patch,
> HIVE-21637.35.patch, HIVE-21637.36.patch, HIVE-21637.37.patch,
> HIVE-21637.38.patch, HIVE-21637.39.patch, HIVE-21637.4.patch,
> HIVE-21637.40.patch, HIVE-21637.41.patch, HIVE-21637.42.patch,
> HIVE-21637.43.patch, HIVE-21637.44.patch, HIVE-21637.45.patch,
> HIVE-21637.46.patch, HIVE-21637.47.patch, HIVE-21637.48.patch,
> HIVE-21637.49.patch, HIVE-21637.5.patch, HIVE-21637.50.patch,
> HIVE-21637.51.patch, HIVE-21637.52.patch, HIVE-21637.53.patch,
> HIVE-21637.54.patch, HIVE-21637.55.patch, HIVE-21637.6.patch,
> HIVE-21637.7.patch, HIVE-21637.8.patch, HIVE-21637.9.patch
>
>
> Currently, HMS has a cache implemented by CachedStore. The cache is
> asynchronized and in HMS HA setting, we can only get eventual consistency. In
> this Jira, we try to make it synchronized.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898763#comment-16898763
]
Hive QA commented on HIVE-21637:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976488/HIVE-21637.55.patch
{color:green}SUCCESS:{color} +1 due to 124 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16717 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_partition_cluster]
(batchId=182)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testNoStatsUpdateForTxnReplTable
(batchId=318)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18236/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18236/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18236/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976488 - PreCommit-HIVE-Build
> Synchronized metastore cache
>
>
> Key: HIVE-21637
> URL: https://issues.apache.org/jira/browse/HIVE-21637
> Project: Hive
> Issue Type: New Feature
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-21637-1.patch, HIVE-21637.10.patch,
> HIVE-21637.11.patch, HIVE-21637.12.patch, HIVE-21637.13.patch,
> HIVE-21637.14.patch, HIVE-21637.15.patch, HIVE-21637.16.patch,
> HIVE-21637.17.patch, HIVE-21637.18.patch, HIVE-21637.19.patch,
> HIVE-21637.19.patch, HIVE-21637.2.patch, HIVE-21637.20.patch,
> HIVE-21637.21.patch, HIVE-21637.22.patch, HIVE-21637.23.patch,
> HIVE-21637.24.patch, HIVE-21637.25.patch, HIVE-21637.26.patch,
> HIVE-21637.27.patch, HIVE-21637.28.patch, HIVE-21637.29.patch,
> HIVE-21637.3.patch, HIVE-21637.30.patch, HIVE-21637.31.patch,
> HIVE-21637.32.patch, HIVE-21637.33.patch, HIVE-21637.34.patch,
> HIVE-21637.35.patch, HIVE-21637.36.patch, HIVE-21637.37.patch,
> HIVE-21637.38.patch, HIVE-21637.39.patch, HIVE-21637.4.patch,
> HIVE-21637.40.patch, HIVE-21637.41.patch, HIVE-21637.42.patch,
> HIVE-21637.43.patch, HIVE-21637.44.patch, HIVE-21637.45.patch,
> HIVE-21637.46.patch, HIVE-21637.47.patch, HIVE-21637.48.patch,
> HIVE-21637.49.patch, HIVE-21637.5.patch, HIVE-21637.50.patch,
> HIVE-21637.51.patch, HIVE-21637.52.patch, HIVE-21637.53.patch,
> HIVE-21637.54.patch, HIVE-21637.55.patch, HIVE-21637.6.patch,
> HIVE-21637.7.patch, HIVE-21637.8.patch, HIVE-21637.9.patch
>
>
> Currently, HMS has a cache implemented by CachedStore. The cache is
> asynchronized and in HMS HA setting, we can only get eventual consistency. In
> this Jira, we try to make it synchronized.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-21637) Synchronized metastore cache
[
https://issues.apache.org/jira/browse/HIVE-21637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898766#comment-16898766
]
Hive QA commented on HIVE-21637:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
32s{color} | {color:blue} standalone-metastore/metastore-common in master has
31 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
14s{color} | {color:blue} standalone-metastore/metastore-server in master has
180 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} beeline in master has 44 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue}
standalone-metastore/metastore-tools/metastore-benchmarks in master has 3
extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
48s{color} | {color:blue} itests/util in master has 44 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
45s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} storage-api: The patch generated 2 new + 15 unchanged
- 0 fixed = 17 total (was 15) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 9 new + 487 unchanged - 4 fixed = 496 total (was 491) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 178 new + 1910 unchanged - 65 fixed = 2088 total (was 1975) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
3s{color} | {color:red} ql: The patch generated 64 new + 2289 unchanged - 32
fixed = 2353 total (was 2321) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} standalone-metastore/metastore-tools/tools-common: The
patch generated 5 new + 31 unchanged - 0 fixed = 36 total (was 31) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} itests/hcatalog-unit: The
[jira] [Updated] (HIVE-22075) Fix the max-reducers=1 regression from HIVE-14200
[
https://issues.apache.org/jira/browse/HIVE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-22075:
---
Component/s: (was: Tez)
llap
> Fix the max-reducers=1 regression from HIVE-14200
> -
>
> Key: HIVE-22075
> URL: https://issues.apache.org/jira/browse/HIVE-22075
> Project: Hive
> Issue Type: Bug
> Components: llap, Physical Optimizer
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Jesus Camacho Rodriguez
>Assignee: Gopal V
>Priority: Major
>
> The condition does not kick in when
> {code}
> minPartition=1, maxPartition=1, nReducers=1, maxReducers=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22075) Fix the max-reducers=1 regression from HIVE-14200
[
https://issues.apache.org/jira/browse/HIVE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898677#comment-16898677
]
Gopal V commented on HIVE-22075:
Disabling auto-reducer parallelism triggers the LlapDecisionDispatcher to
ignore maxReducers.
> Fix the max-reducers=1 regression from HIVE-14200
> -
>
> Key: HIVE-22075
> URL: https://issues.apache.org/jira/browse/HIVE-22075
> Project: Hive
> Issue Type: Bug
> Components: llap, Physical Optimizer
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Jesus Camacho Rodriguez
>Assignee: Gopal V
>Priority: Major
>
> The condition does not kick in when
> {code}
> minPartition=1, maxPartition=1, nReducers=1, maxReducers=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HIVE-22077) Inserting overwrite partitions clause does not clean directories while partitions' info is not stored in metadata
[
https://issues.apache.org/jira/browse/HIVE-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898685#comment-16898685
]
Hui An edited comment on HIVE-22077 at 8/2/19 8:12 AM:
---
This issue is caused by method loadPartitionInternal of Hive.java
{code:java}
Path oldPartPath = (oldPart != null) ? oldPart.getDataLocation() : null;
Path newPartPath = null;
if (inheritLocation) {
newPartPath = genPartPathFromTable(tbl, partSpec, tblDataLocationPath);
if(oldPart != null) {
/*
* If we are moving the partition across filesystem boundaries
* inherit from the table properties. Otherwise (same filesystem) use the
* original partition location.
*
* See: HIVE-1707 and HIVE-2117 for background
*/
FileSystem oldPartPathFS = oldPartPath.getFileSystem(getConf());
FileSystem loadPathFS = loadPath.getFileSystem(getConf());
if (FileUtils.equalsFileSystem(oldPartPathFS,loadPathFS)) {
newPartPath = oldPartPath;
}
}
} else {
newPartPath = oldPartPath == null
? genPartPathFromTable(tbl, partSpec, tblDataLocationPath) : oldPartPath;
}
{code}
Actually, oldPart is null does not mean oldPartPath is not exists in HDFS, but
it just set oldPartPath is null, and give null value to following method
replaceFiles.
I think we could just give newPartPath value to the oldPartPath when oldPart is
null, but may this causes other problems, or should we check partitions
directory before mr work and throw errors to the end user if there are files
under it?
was (Author: bone an):
This issue is caused by method loadPartitionInternal of Hive.java
{code:java}
Path oldPartPath = (oldPart != null) ? oldPart.getDataLocation() : null;
Path newPartPath = null;
if (inheritLocation) {
newPartPath = genPartPathFromTable(tbl, partSpec, tblDataLocationPath);
if(oldPart != null) {
/*
* If we are moving the partition across filesystem boundaries
* inherit from the table properties. Otherwise (same filesystem) use the
* original partition location.
*
* See: HIVE-1707 and HIVE-2117 for background
*/
FileSystem oldPartPathFS = oldPartPath.getFileSystem(getConf());
FileSystem loadPathFS = loadPath.getFileSystem(getConf());
if (FileUtils.equalsFileSystem(oldPartPathFS,loadPathFS)) {
newPartPath = oldPartPath;
}
}
} else {
newPartPath = oldPartPath == null
? genPartPathFromTable(tbl, partSpec, tblDataLocationPath) : oldPartPath;
}
{code}
Actually, oldPart is null does not mean oldPartPath is not exists in HDFS, but
it just set oldPartPath is null, and give null value to following method
replaceFiles.
> Inserting overwrite partitions clause does not clean directories while
> partitions' info is not stored in metadata
> -
>
> Key: HIVE-22077
> URL: https://issues.apache.org/jira/browse/HIVE-22077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1, 4.0.0, 2.3.4
>Reporter: Hui An
>Assignee: Hui An
>Priority: Major
>
> Inserting overwrite static partitions may not clean related HDFS location if
> partitions' info is not stored in metadata.
> Steps to Reproduce this issue :
>
> 1. Create a managed table :
>
> {code:sql}
> CREATE TABLE `test`(
>`id` string)
> PARTITIONED BY (
>`dayno` string)
> ROW FORMAT SERDE
>'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION |
>'hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test'
> TBLPROPERTIES (
>'transient_lastDdlTime'='1564731656')
> {code}
>
> 2. Create partition's directory and put some data under it
> ----
> {code:java}
> hdfs dfs -mkdir
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> hdfs dfs -put test.data
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> {code}
>
> 3. Insert overwrite partition dayno=20190802
[jira] [Updated] (HIVE-22077) Inserting overwrite partitions clause does not clean directories while partitions' info is not stored in metadata
[
https://issues.apache.org/jira/browse/HIVE-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hui An updated HIVE-22077:
--
Description:
Inserting overwrite static partitions may not clean related HDFS location if
partitions' info is not stored in metadata.
Steps to Reproduce this issue :
1. Create a managed table :
{code:sql}
CREATE TABLE `test`(
`id` string)
PARTITIONED BY (
`dayno` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION |
'hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test'
TBLPROPERTIES (
'transient_lastDdlTime'='1564731656')
{code}
2. Create partition's directory and put some data under it
{code:java}
hdfs dfs -mkdir
hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
hdfs dfs -put test.data
hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
{code}
3. Insert overwrite partition dayno=20190802
{code:sql}
INSERT OVERWRITE TABLE test PARTITION(dayno='20190802')
SELECT "some value";
{code}
4. We could see the test.data under partition directory is not deleted.
was:
Inserting overwrite static partitions may not clean related HDFS location if
partitions' info is not stored in metadata.
Steps to Reproduce this issue :
1. Create a managed table :
{code:sql}
CREATE TABLE `test`(
`id` string)
PARTITIONED BY (
`dayno` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
STORED AS INPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
LOCATION |
'hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test'
TBLPROPERTIES (
'transient_lastDdlTime'='1564731656')
{code}
2. Create partition's directory and put some data under it
{code:java}
hdfs dfs -mkdir
hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
hdfs dfs -put test.data
hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
{code}
3. Insert overwrite partition dayno=20190802
{code:sql}
INSERT OVERWRITE TABLE test PARTITION(dayno='20190802')
SELECT 1;
{code}
4. We could see the test.data under partition directory is not deleted.
> Inserting overwrite partitions clause does not clean directories while
> partitions' info is not stored in metadata
> -
>
> Key: HIVE-22077
> URL: https://issues.apache.org/jira/browse/HIVE-22077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1, 4.0.0, 2.3.4
>Reporter: Hui An
>Assignee: Hui An
>Priority: Major
>
> Inserting overwrite static partitions may not clean related HDFS location if
> partitions' info is not stored in metadata.
> Steps to Reproduce this issue :
>
> 1. Create a managed table :
>
> {code:sql}
> CREATE TABLE `test`(
>`id` string)
> PARTITIONED BY (
[jira] [Commented] (HIVE-21349) TIMESTAMP WITHOUT TIME ZONE
[
https://issues.apache.org/jira/browse/HIVE-21349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898722#comment-16898722
]
Piotr Findeisen commented on HIVE-21349:
Do i understand correctly that {{TIMESTAMP WITHOUT TIME ZONE}} is going to be
an alias for {{TIMESTAMP}}?
I.e. it's not going to be distinguishable whether I declared a column as
"{{TIMESTAMP}}" or "{{TIMESTAMP WITHOUT TIME ZONE}}"?
> TIMESTAMP WITHOUT TIME ZONE
> ---
>
> Key: HIVE-21349
> URL: https://issues.apache.org/jira/browse/HIVE-21349
> Project: Hive
> Issue Type: Task
>Reporter: Zoltan Ivanfi
>Priority: Major
>
> As specified in the [design doc for TIMESTAMP
> types|https://cwiki.apache.org/confluence/display/Hive/Different+TIMESTAMP+types],
> the TIMESTAMP WITHOUT TIME ZONE type shall behave like the
> [LocalDateTime|https://docs.oracle.com/javase/8/docs/api/java/time/LocalDateTime.html]
> class of Java, i.e., each value is a recording of what can be seen on a
> calendar and a clock hanging on the wall, for example "1969-07-20 16:17:39".
> It can be decomposed into year, month, day, hour, minute and seconds fields,
> but with no time zone information available, it does not correspond to any
> specific point in time.
> This behaviour is consistent with the SQL standard (revisions 2003 and
> higher).
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (HIVE-22077) Inserting overwrite partitions clause does not clean directories while partitions' info is not stored in metadata
[
https://issues.apache.org/jira/browse/HIVE-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hui An reassigned HIVE-22077:
-
> Inserting overwrite partitions clause does not clean directories while
> partitions' info is not stored in metadata
> -
>
> Key: HIVE-22077
> URL: https://issues.apache.org/jira/browse/HIVE-22077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.4, 1.1.1, 4.0.0
>Reporter: Hui An
>Assignee: Hui An
>Priority: Major
>
> Inserting overwrite static partitions may not clean related HDFS location if
> partitions' info is not stored in metadata.
> Steps to Reproduce this issue :
>
> 1. Create a managed table :
>
> {code:sql}
> CREATE TABLE `test`(
>`id` string)
> PARTITIONED BY (
>`dayno` string)
> ROW FORMAT SERDE
>'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION |
>'hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test'
> TBLPROPERTIES (
>'transient_lastDdlTime'='1564731656')
> {code}
>
> 2. Create partition's directory and put some data under it
> ----
> {code:java}
> hdfs dfs -mkdir
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> hdfs dfs -put test.data
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> {code}
>
> 3. Insert overwrite partition dayno=20190802
>
> {code:sql}
> INSERT OVERWRITE TABLE test PARTITION(dayno='20190802')
> SELECT 1;
> {code}
>
> 4. We could see the test.data under partition directory is not deleted.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22077) Inserting overwrite partitions clause does not clean directories while partitions' info is not stored in metadata
[
https://issues.apache.org/jira/browse/HIVE-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898685#comment-16898685
]
Hui An commented on HIVE-22077:
---
This issue is caused by method loadPartitionInternal of Hive.java
{code:java}
Path oldPartPath = (oldPart != null) ? oldPart.getDataLocation() : null;
Path newPartPath = null;
if (inheritLocation) {
newPartPath = genPartPathFromTable(tbl, partSpec, tblDataLocationPath);
if(oldPart != null) {
/*
* If we are moving the partition across filesystem boundaries
* inherit from the table properties. Otherwise (same filesystem) use the
* original partition location.
*
* See: HIVE-1707 and HIVE-2117 for background
*/
FileSystem oldPartPathFS = oldPartPath.getFileSystem(getConf());
FileSystem loadPathFS = loadPath.getFileSystem(getConf());
if (FileUtils.equalsFileSystem(oldPartPathFS,loadPathFS)) {
newPartPath = oldPartPath;
}
}
} else {
newPartPath = oldPartPath == null
? genPartPathFromTable(tbl, partSpec, tblDataLocationPath) : oldPartPath;
}
{code}
Actually, oldPart is null does not mean oldPartPath is not exists in HDFS, but
it just set oldPartPath is null, and give null value to following method
replaceFiles.
> Inserting overwrite partitions clause does not clean directories while
> partitions' info is not stored in metadata
> -
>
> Key: HIVE-22077
> URL: https://issues.apache.org/jira/browse/HIVE-22077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1, 4.0.0, 2.3.4
>Reporter: Hui An
>Assignee: Hui An
>Priority: Major
>
> Inserting overwrite static partitions may not clean related HDFS location if
> partitions' info is not stored in metadata.
> Steps to Reproduce this issue :
>
> 1. Create a managed table :
>
> {code:sql}
> CREATE TABLE `test`(
>`id` string)
> PARTITIONED BY (
>`dayno` string)
> ROW FORMAT SERDE
>'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION |
>'hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test'
> TBLPROPERTIES (
>'transient_lastDdlTime'='1564731656')
> {code}
>
> 2. Create partition's directory and put some data under it
> ----
> {code:java}
> hdfs dfs -mkdir
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> hdfs dfs -put test.data
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test/dayno=20190802
> {code}
>
> 3. Insert overwrite partition dayno=20190802
>
> {code:sql}
> INSERT OVERWRITE TABLE test PARTITION(dayno='20190802')
> SELECT 1;
> {code}
>
> 4. We could see the test.data under partition directory is not deleted.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-22075) Fix the max-reducers=1 regression from HIVE-14200
[
https://issues.apache.org/jira/browse/HIVE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-22075:
---
Status: Patch Available (was: Open)
> Fix the max-reducers=1 regression from HIVE-14200
> -
>
> Key: HIVE-22075
> URL: https://issues.apache.org/jira/browse/HIVE-22075
> Project: Hive
> Issue Type: Bug
> Components: llap, Physical Optimizer
>Affects Versions: 3.1.1, 4.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22075.1.patch
>
>
> The condition does not kick in when
> {code}
> minPartition=1, maxPartition=1, nReducers=1, maxReducers=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (HIVE-22075) Fix the max-reducers=1 regression from HIVE-14200
[
https://issues.apache.org/jira/browse/HIVE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gopal V updated HIVE-22075:
---
Attachment: HIVE-22075.1.patch
> Fix the max-reducers=1 regression from HIVE-14200
> -
>
> Key: HIVE-22075
> URL: https://issues.apache.org/jira/browse/HIVE-22075
> Project: Hive
> Issue Type: Bug
> Components: llap, Physical Optimizer
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Jesus Camacho Rodriguez
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22075.1.patch
>
>
> The condition does not kick in when
> {code}
> minPartition=1, maxPartition=1, nReducers=1, maxReducers=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (HIVE-22077) Inserting overwrite partitions clause does not clean directories while partitions' info is not stored in metadata
[
https://issues.apache.org/jira/browse/HIVE-22077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898685#comment-16898685
]
Hui An edited comment on HIVE-22077 at 8/2/19 8:14 AM:
---
This issue is caused by method loadPartitionInternal of Hive.java
{code:java}
Path oldPartPath = (oldPart != null) ? oldPart.getDataLocation() : null;
Path newPartPath = null;
if (inheritLocation) {
newPartPath = genPartPathFromTable(tbl, partSpec, tblDataLocationPath);
if(oldPart != null) {
/*
* If we are moving the partition across filesystem boundaries
* inherit from the table properties. Otherwise (same filesystem) use the
* original partition location.
*
* See: HIVE-1707 and HIVE-2117 for background
*/
FileSystem oldPartPathFS = oldPartPath.getFileSystem(getConf());
FileSystem loadPathFS = loadPath.getFileSystem(getConf());
if (FileUtils.equalsFileSystem(oldPartPathFS,loadPathFS)) {
newPartPath = oldPartPath;
}
}
} else {
newPartPath = oldPartPath == null
? genPartPathFromTable(tbl, partSpec, tblDataLocationPath) : oldPartPath;
}
{code}
Actually, oldPart is null does not mean oldPartPath is not exists in HDFS, but
it just set oldPartPath is null, and give null value to following method
replaceFiles.
I think we could just give newPartPath value to the oldPartPath when oldPart is
null, may this causes other problems? Or should we check partitions directory
before mr work and throw errors to the end user if there are files under it?
was (Author: bone an):
This issue is caused by method loadPartitionInternal of Hive.java
{code:java}
Path oldPartPath = (oldPart != null) ? oldPart.getDataLocation() : null;
Path newPartPath = null;
if (inheritLocation) {
newPartPath = genPartPathFromTable(tbl, partSpec, tblDataLocationPath);
if(oldPart != null) {
/*
* If we are moving the partition across filesystem boundaries
* inherit from the table properties. Otherwise (same filesystem) use the
* original partition location.
*
* See: HIVE-1707 and HIVE-2117 for background
*/
FileSystem oldPartPathFS = oldPartPath.getFileSystem(getConf());
FileSystem loadPathFS = loadPath.getFileSystem(getConf());
if (FileUtils.equalsFileSystem(oldPartPathFS,loadPathFS)) {
newPartPath = oldPartPath;
}
}
} else {
newPartPath = oldPartPath == null
? genPartPathFromTable(tbl, partSpec, tblDataLocationPath) : oldPartPath;
}
{code}
Actually, oldPart is null does not mean oldPartPath is not exists in HDFS, but
it just set oldPartPath is null, and give null value to following method
replaceFiles.
I think we could just give newPartPath value to the oldPartPath when oldPart is
null, but may this causes other problems, or should we check partitions
directory before mr work and throw errors to the end user if there are files
under it?
> Inserting overwrite partitions clause does not clean directories while
> partitions' info is not stored in metadata
> -
>
> Key: HIVE-22077
> URL: https://issues.apache.org/jira/browse/HIVE-22077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1, 4.0.0, 2.3.4
>Reporter: Hui An
>Assignee: Hui An
>Priority: Major
>
> Inserting overwrite static partitions may not clean related HDFS location if
> partitions' info is not stored in metadata.
> Steps to Reproduce this issue :
>
> 1. Create a managed table :
>
> {code:sql}
> CREATE TABLE `test`(
>`id` string)
> PARTITIONED BY (
>`dayno` string)
> ROW FORMAT SERDE
>'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
>'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION |
>'hdfs://test-dev-hdfs/user/hive/warehouse/test.db/test'
> TBLPROPERTIES (
>'transient_lastDdlTime'='1564731656')
> {code}
>
> 2. Create partition's directory and put some data under it
> ----
> {code:java}
> hdfs dfs -mkdir
> hdfs://test-dev-hdfs/user/hive/warehouse/test.db
[jira] [Reopened] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[ https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar reopened HIVE-21828: -- > Tez: Use a pre-parsed TezConfiguration from DagUtils > > > Key: HIVE-21828 > URL: https://issues.apache.org/jira/browse/HIVE-21828 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21828.1.patch, HIVE-21828.2.patch, > HIVE-21828.5.patch, HIVE-21828.6.patch, HIVE-21828.7.patch > > > The HS2 tez-site.xml does not change dynamically - the XML parsed components > of the config can be obtained statically and kept across sessions. > This allows for the replacing of "new TezConfiguration()" with a HS2 local > version instead. > The configuration object however has to reference the right resource file > (i.e location of tez-site.xml) without reparsing it for each query. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[ https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-21828: - Attachment: HIVE-21828.7.patch > Tez: Use a pre-parsed TezConfiguration from DagUtils > > > Key: HIVE-21828 > URL: https://issues.apache.org/jira/browse/HIVE-21828 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21828.1.patch, HIVE-21828.2.patch, > HIVE-21828.5.patch, HIVE-21828.6.patch, HIVE-21828.7.patch > > > The HS2 tez-site.xml does not change dynamically - the XML parsed components > of the config can be obtained statically and kept across sessions. > This allows for the replacing of "new TezConfiguration()" with a HS2 local > version instead. > The configuration object however has to reference the right resource file > (i.e location of tez-site.xml) without reparsing it for each query. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[ https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-21828: - Status: Patch Available (was: Reopened) > Tez: Use a pre-parsed TezConfiguration from DagUtils > > > Key: HIVE-21828 > URL: https://issues.apache.org/jira/browse/HIVE-21828 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21828.1.patch, HIVE-21828.2.patch, > HIVE-21828.5.patch, HIVE-21828.6.patch, HIVE-21828.7.patch > > > The HS2 tez-site.xml does not change dynamically - the XML parsed components > of the config can be obtained statically and kept across sessions. > This allows for the replacing of "new TezConfiguration()" with a HS2 local > version instead. > The configuration object however has to reference the right resource file > (i.e location of tez-site.xml) without reparsing it for each query. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (HIVE-22076) JDK11: Remove ParallelGC in debug.sh
[
https://issues.apache.org/jira/browse/HIVE-22076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898847#comment-16898847
]
Hive QA commented on HIVE-22076:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12976493/HIVE-22076.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16723 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18238/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18238/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18238/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12976493 - PreCommit-HIVE-Build
> JDK11: Remove ParallelGC in debug.sh
>
>
> Key: HIVE-22076
> URL: https://issues.apache.org/jira/browse/HIVE-22076
> Project: Hive
> Issue Type: Bug
> Components: Diagnosability
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Minor
> Attachments: HIVE-22076.1.patch
>
>
> The JDK debug mode no longer depends on ParallelGC
> This was a workaround for JDK6 bug -
> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6862295
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (HIVE-22075) Fix the max-reducers=1 regression from HIVE-14200
[
https://issues.apache.org/jira/browse/HIVE-22075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898869#comment-16898869
]
Hive QA commented on HIVE-22075:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 2250 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18239/dev-support/hive-personality.sh
|
| git revision | master / d7475aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18239/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix the max-reducers=1 regression from HIVE-14200
> -
>
> Key: HIVE-22075
> URL: https://issues.apache.org/jira/browse/HIVE-22075
> Project: Hive
> Issue Type: Bug
> Components: llap, Physical Optimizer
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Jesus Camacho Rodriguez
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22075.1.patch
>
>
> The condition does not kick in when
> {code}
> minPartition=1, maxPartition=1, nReducers=1, maxReducers=1
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
