[jira] [Updated] (HIVE-24563) Check if we can interchange client and server sides for umbilical for external client flow
[
https://issues.apache.org/jira/browse/HIVE-24563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Chaurasia updated HIVE-24563:
-
Description:
Currently we open three tcp connections when llap external client communicates
to llap.
{noformat}
llap-ext-client ... llap
connection1: client ...>>... server
(RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
connection)
connection2: client ...>>... server
(for reading the output of t1, t2, t3. llap-ext-client initiates connection)
connection3: umbilical server ...<<... client
(RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
connection)
{noformat}
connection3 starts a umbilical(RPC) server at the client side to which llap
daemon keeps sending the task statuses / heartbeats and node heartbeats.
*The Problem*
In cloud based deployment, we need to open tcp traffic.
1. For connection1 and connection2, we need to open incoming tcp traffic on the
machines running llap from client.
2. For connection3, we need to open incoming tcp traffic on the machines where
llap-ext-client is running, from llap daemon.
Here clients also need to worry about opening traffic(from llap) at their end.
*Possible Solution*
This jira is to evaluate the possibility of interchanging Umbilical server and
client sides i.e. umbilical server will run in llap only and llap-ext-client
will act as client and initiate the connection.
We can have umbilical address in llap splits (when get_splits is called by
external client) which the client can later connect to.
cc [~prasanth_j] [~harishjp]
was:
Currently we open three tcp connections when llap external client communicates
to llap.
{noformat}
llap-ext-client ... llap
connection1: client ...>>... server
(RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
connection)
connection2: client ...>>... server
(for reading the output of t1, t2, t3. llap-ext-client initiates connection)
connection3: umbilical server ...<<... client
(RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
connection)
{noformat}
connection3 starts a umbilical(RPC) server at the client side to which llap
daemon keeps sending the task statuses / heartbeats and node heartbeats.
*The Problem*
In cloud based deployment, we need to open tcp traffic.
1. For connection1 and connection2, we need to open incoming tcp traffic on the
machines running llap from client.
2. For connection3, we need to open incoming tcp traffic on the machines where
llap-ext-client is running, from llap daemon.
Here clients also need to worry about opening traffic(from llap) at their end.
*Possible Solution*
This jira is to evaluate the possibility of interchanging Umbilical server and
client sides i.e. umbilical server will run in llap only and llap-ext-client
will act as client and initiate the connection.
We can have umbilical address in llap splits (when get_splits is called by
external client) which the client can later connect to.
cc [~prasanth_j]
> Check if we can interchange client and server sides for umbilical for
> external client flow
> --
>
> Key: HIVE-24563
> URL: https://issues.apache.org/jira/browse/HIVE-24563
> Project: Hive
> Issue Type: Sub-task
> Components: Hive, llap
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Major
>
> Currently we open three tcp connections when llap external client
> communicates to llap.
> {noformat}
>llap-ext-client ... llap
> connection1: client ...>>... server
> (RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
> connection)
> connection2: client ...>>... server
> (for reading the output of t1, t2, t3. llap-ext-client initiates connection)
> connection3: umbilical server ...<<... client
> (RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
> connection)
> {noformat}
> connection3 starts a umbilical(RPC) server at the client side to which llap
> daemon keeps sending the task statuses / heartbeats and node heartbeats.
> *The Problem*
> In cloud based deployment, we need to open tcp traffic.
> 1. For connection1 and connection2, we need to open incoming tcp traffic on
> the machines running llap from client.
> 2. For connection3, we need to open incoming tcp traffic on the machines
> where llap-ext-client is running, from llap daemon.
> Here clients also need to worry about opening traffic(from llap) at their
> end.
> *Possible Solution*
>
[jira] [Updated] (HIVE-24563) Check if we can interchange client and server sides for umbilical for external client flow
[
https://issues.apache.org/jira/browse/HIVE-24563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Chaurasia updated HIVE-24563:
-
Description:
Currently we open three tcp connections when llap external client communicates
to llap.
{noformat}
llap-ext-client ... llap
connection1: client ...>>... server
(RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
connection)
connection2: client ...>>... server
(for reading the output of t1, t2, t3. llap-ext-client initiates connection)
connection3: umbilical server ...<<... client
(RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
connection)
{noformat}
connection3 starts a umbilical(RPC) server at the client side to which llap
daemon keeps sending the task statuses / heartbeats and node heartbeats.
*The Problem*
In cloud based deployment, we need to open tcp traffic.
1. For connection1 and connection2, we need to open incoming tcp traffic on the
machines running llap from client.
2. For connection3, we need to open incoming tcp traffic on the machines where
llap-ext-client is running, from llap daemon.
Here clients also need to worry about opening traffic(from llap) at their end.
*Possible Solution*
This jira is to evaluate the possibility of interchanging Umbilical server and
client sides i.e. umbilical server will run in llap only and llap-ext-client
will act as client and initiate the connection.
We can have umbilical address in llap splits (when get_splits is called by
external client) which the client can later connect to.
cc [~prasanth_j]
was:
Currently we open three tcp connections when llap external client communicates
to llap.
{noformat}
llap-ext-client ... llap
connection1: client ...>>... server
(RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
connection)
connection2: client ...>>... server
(for reading the output of t1, t2, t3. llap-ext-client initiates connection)
connection3: umbilical server ...<<... client
(RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
connection)
{noformat}
connection3 starts a umbilical(RPC) server at the client side to which llap
daemon keeps sending the task statuses / heartbeats and node heartbeats.
*The Problem*
In cloud based deployment, we need to open tcp traffic.
1. For connection1 and connection2, we need to open incoming tcp traffic on the
machines running llap from client.
2. For connection3, we need to open incoming tcp traffic on the machines where
llap-ext-client is running, from llap daemon.
Here clients also need to worry about opening traffic(from llap) at their end.
*Possible Solution*
This jira is to evaluate the possibility of interchanging Umbilical server and
client sides i.e. umbilical server will run in llap only and llap-ext-client
will act as client and initiate the connection.
We can have umbilical address in llap splits (when get_splits is called by
external client) which the client can later connect to.
> Check if we can interchange client and server sides for umbilical for
> external client flow
> --
>
> Key: HIVE-24563
> URL: https://issues.apache.org/jira/browse/HIVE-24563
> Project: Hive
> Issue Type: Sub-task
> Components: Hive, llap
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Major
>
> Currently we open three tcp connections when llap external client
> communicates to llap.
> {noformat}
>llap-ext-client ... llap
> connection1: client ...>>... server
> (RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
> connection)
> connection2: client ...>>... server
> (for reading the output of t1, t2, t3. llap-ext-client initiates connection)
> connection3: umbilical server ...<<... client
> (RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
> connection)
> {noformat}
> connection3 starts a umbilical(RPC) server at the client side to which llap
> daemon keeps sending the task statuses / heartbeats and node heartbeats.
> *The Problem*
> In cloud based deployment, we need to open tcp traffic.
> 1. For connection1 and connection2, we need to open incoming tcp traffic on
> the machines running llap from client.
> 2. For connection3, we need to open incoming tcp traffic on the machines
> where llap-ext-client is running, from llap daemon.
> Here clients also need to worry about opening traffic(from llap) at their
> end.
> *Possible Solution*
> This jira is to evaluate th
[jira] [Updated] (HIVE-24563) Check if we can interchange client and server sides for umbilical for external client flow
[
https://issues.apache.org/jira/browse/HIVE-24563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Chaurasia updated HIVE-24563:
-
Description:
Currently we open three tcp connections when llap external client communicates
to llap.
{noformat}
llap-ext-client ... llap
connection1: client ...>>... server
(RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
connection)
connection2: client ...>>... server
(for reading the output of t1, t2, t3. llap-ext-client initiates connection)
connection3: umbilical server ...<<... client
(RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
connection)
{noformat}
connection3 starts a umbilical(RPC) server at the client side to which llap
daemon keeps sending the task statuses / heartbeats and node heartbeats.
*The Problem*
In cloud based deployment, we need to open tcp traffic.
1. For connection1 and connection2, we need to open incoming tcp traffic on the
machines running llap from client.
2. For connection3, we need to open incoming tcp traffic on the machines where
llap-ext-client is running, from llap daemon.
Here clients also need to worry about opening traffic(from llap) at their end.
*Possible Solution*
This jira is to evaluate the possibility of interchanging Umbilical server and
client sides i.e. umbilical server will run in llap only and llap-ext-client
will act as client and initiate the connection.
We can have umbilical address in llap splits (when get_splits is called by
external client) which the client can later connect to.
was:
Currently we open three tcp connections when llap external client communicates
to llap.
{noformat}
llap-ext-client ... llap
connection1: client ...>>... server
(RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
connection)
connection2: client ...>>... server
(for reading the output of t1, t2, t3. llap-ext-client initiates connection)
connection3: umbilical server ...<<... client
(RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
connection)
{noformat}
connection3 starts a umbilical(RPC) server at the client side to which llap
daemon keeps sending the task statuses / heartbeats and node heartbeats.
*The Problem*
In cloud based deployment, we need to open tcp traffic.
1. For connection1 and connection2, we need to open incoming tcp traffic on the
machines running llap from client.
2. For connection3, we need to open incoming tcp traffic on the machines where
llap-ext-client is running, from llap daemon.
Here clients also need to worry about opening traffic(from llap) at their end.
This jira is to evaluate the possibility of interchanging Umbilical server and
client sides i.e. umbilical server will run in llap only and llap-ext-client
will act as client and initiate the connection.
We can have umbilical address in llap splits (when get_splits is called by
external client) which the client can later connect to.
> Check if we can interchange client and server sides for umbilical for
> external client flow
> --
>
> Key: HIVE-24563
> URL: https://issues.apache.org/jira/browse/HIVE-24563
> Project: Hive
> Issue Type: Sub-task
> Components: Hive, llap
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Major
>
> Currently we open three tcp connections when llap external client
> communicates to llap.
> {noformat}
>llap-ext-client ... llap
> connection1: client ...>>... server
> (RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
> connection)
> connection2: client ...>>... server
> (for reading the output of t1, t2, t3. llap-ext-client initiates connection)
> connection3: umbilical server ...<<... client
> (RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
> connection)
> {noformat}
> connection3 starts a umbilical(RPC) server at the client side to which llap
> daemon keeps sending the task statuses / heartbeats and node heartbeats.
> *The Problem*
> In cloud based deployment, we need to open tcp traffic.
> 1. For connection1 and connection2, we need to open incoming tcp traffic on
> the machines running llap from client.
> 2. For connection3, we need to open incoming tcp traffic on the machines
> where llap-ext-client is running, from llap daemon.
> Here clients also need to worry about opening traffic(from llap) at their
> end.
> *Possible Solution*
> This jira is to evaluate the possibility of interchanging Umbil
[jira] [Assigned] (HIVE-24563) Check if we can interchange client and server sides for umbilical for external client flow
[
https://issues.apache.org/jira/browse/HIVE-24563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shubham Chaurasia reassigned HIVE-24563:
> Check if we can interchange client and server sides for umbilical for
> external client flow
> --
>
> Key: HIVE-24563
> URL: https://issues.apache.org/jira/browse/HIVE-24563
> Project: Hive
> Issue Type: Sub-task
> Components: Hive, llap
>Reporter: Shubham Chaurasia
>Assignee: Shubham Chaurasia
>Priority: Major
>
> Currently we open three tcp connections when llap external client
> communicates to llap.
> {noformat}
>llap-ext-client ... llap
> connection1: client ...>>... server
> (RPC for submitting fragments - say t1, t2, t3. llap-ext-client initiates
> connection)
> connection2: client ...>>... server
> (for reading the output of t1, t2, t3. llap-ext-client initiates connection)
> connection3: umbilical server ...<<... client
> (RPC for status updates/heartbeat of t1, t2, t3. llap Daemon initiates
> connection)
> {noformat}
> connection3 starts a umbilical(RPC) server at the client side to which llap
> daemon keeps sending the task statuses / heartbeats and node heartbeats.
> *The Problem*
> In cloud based deployment, we need to open tcp traffic.
> 1. For connection1 and connection2, we need to open incoming tcp traffic on
> the machines running llap from client.
> 2. For connection3, we need to open incoming tcp traffic on the machines
> where llap-ext-client is running, from llap daemon.
> Here clients also need to worry about opening traffic(from llap) at their
> end.
> This jira is to evaluate the possibility of interchanging Umbilical server
> and client sides i.e. umbilical server will run in llap only and
> llap-ext-client will act as client and initiate the connection.
> We can have umbilical address in llap splits (when get_splits is called by
> external client) which the client can later connect to.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24559) Fix some spelling issues
[ https://issues.apache.org/jira/browse/HIVE-24559?focusedWorklogId=527521&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527521 ] ASF GitHub Bot logged work on HIVE-24559: - Author: ASF GitHub Bot Created on: 23/Dec/20 06:57 Start Date: 23/Dec/20 06:57 Worklog Time Spent: 10m Work Description: rickyma commented on pull request #1805: URL: https://github.com/apache/hive/pull/1805#issuecomment-749974134 Just some typo fixes in some comments, there is no need to pass all the checks. Can anyone please merge this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527521) Time Spent: 0.5h (was: 20m) > Fix some spelling issues > > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > There are some minor typos: > [https://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858|https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24526) Get grouped locations of external table data using metatool.
[ https://issues.apache.org/jira/browse/HIVE-24526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arko Sharma updated HIVE-24526: --- Attachment: HIVE-24526.03.patch HIVE-24526.02.patch HIVE-24526.01.patch > Get grouped locations of external table data using metatool. > > > Key: HIVE-24526 > URL: https://issues.apache.org/jira/browse/HIVE-24526 > Project: Hive > Issue Type: Task >Reporter: Arko Sharma >Assignee: Arko Sharma >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24526.01.patch, HIVE-24526.02.patch, > HIVE-24526.03.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Add a functionality to metatool to get a list of locations which cover all > external-table data-locations for a database specified by user. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24309) Simplify ConvertJoinMapJoin logic
[ https://issues.apache.org/jira/browse/HIVE-24309?focusedWorklogId=527443&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527443 ] ASF GitHub Bot logged work on HIVE-24309: - Author: ASF GitHub Bot Created on: 23/Dec/20 01:00 Start Date: 23/Dec/20 01:00 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1606: URL: https://github.com/apache/hive/pull/1606#issuecomment-749857921 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527443) Time Spent: 20m (was: 10m) > Simplify ConvertJoinMapJoin logic > -- > > Key: HIVE-24309 > URL: https://issues.apache.org/jira/browse/HIVE-24309 > Project: Hive > Issue Type: Improvement >Reporter: Panagiotis Garefalakis >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > ConvertMapJoin logic can be further simplified: > [https://github.com/pgaref/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java#L92] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-23583) Upgrade to ant 1.10.9 due to CVEs
[ https://issues.apache.org/jira/browse/HIVE-23583?focusedWorklogId=527442&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527442 ] ASF GitHub Bot logged work on HIVE-23583: - Author: ASF GitHub Bot Created on: 23/Dec/20 01:00 Start Date: 23/Dec/20 01:00 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1599: URL: https://github.com/apache/hive/pull/1599#issuecomment-749857938 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527442) Time Spent: 20m (was: 10m) > Upgrade to ant 1.10.9 due to CVEs > - > > Key: HIVE-23583 > URL: https://issues.apache.org/jira/browse/HIVE-23583 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.2 >Reporter: Renukaprasad C >Assignee: Kevin Risden >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-23583.01.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Update ANT to fix: > CVE-2020-1945: Apache Ant insecure temporary file vulnerability > Severity: Medium > Vendor: > The Apache Software Foundation > Versions Affected: > Apache Ant 1.1 to 1.9.14 and 1.10.0 to 1.10.7 > Description: > Apache Ant uses the default temporary directory identified by the Java > system property java.io.tmpdir for several tasks and may thus leak > sensitive information. The fixcrlf and replaceregexp tasks also copy > files from the temporary directory back into the build tree allowing an > attacker to inject modified source files into the build process. > Mitigation: > Ant users of versions 1.1 to 1.9.14 and 1.10.0 to 1.10.7 should set the > java.io.tmpdir system property to point to a directory only readable and > writable by the current user prior to running Ant. > Users of versions 1.9.15 and 1.10.8 can use the Ant property ant.tmpfile > instead. Users of Ant 1.10.8 can rely on Ant protecting the temporary > files if the underlying filesystem allows it, but we still recommend > using a private temporary directory instead. > References: > https://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-1945 > https://nvd.nist.gov/vuln/detail/CVE-2020-1945 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24307) Beeline with property-file and -e parameter is failing
[
https://issues.apache.org/jira/browse/HIVE-24307?focusedWorklogId=527441&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527441
]
ASF GitHub Bot logged work on HIVE-24307:
-
Author: ASF GitHub Bot
Created on: 23/Dec/20 00:59
Start Date: 23/Dec/20 00:59
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #1603:
URL: https://github.com/apache/hive/pull/1603#issuecomment-749857931
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527441)
Time Spent: 20m (was: 10m)
> Beeline with property-file and -e parameter is failing
> --
>
> Key: HIVE-24307
> URL: https://issues.apache.org/jira/browse/HIVE-24307
> Project: Hive
> Issue Type: Bug
>Reporter: Ayush Saxena
>Assignee: Ayush Saxena
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-24307-01.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Beeline query with property file specified with -e parameter fails with :
> {noformat}
> Cannot run commands specified using -e. No current connection
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24562) Deflake TestHivePrivilegeObjectOwnerNameAndType
[
https://issues.apache.org/jira/browse/HIVE-24562?focusedWorklogId=527402&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527402
]
ASF GitHub Bot logged work on HIVE-24562:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 22:55
Start Date: 22/Dec/20 22:55
Worklog Time Spent: 10m
Work Description: vihangk1 merged pull request #1809:
URL: https://github.com/apache/hive/pull/1809
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527402)
Time Spent: 0.5h (was: 20m)
> Deflake TestHivePrivilegeObjectOwnerNameAndType
> ---
>
> Key: HIVE-24562
> URL: https://issues.apache.org/jira/browse/HIVE-24562
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> One of my unrelated PRs fails this test
> {{TestHivePrivilegeObjectOwnerNameAndType}}. The exception which I see in the
> logs is below:
> {noformat}
> Caused by: ERROR 42X05: Table/View 'TXN_LOCK_TBL' does not exist.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.sql.compile.LockTableNode.bindStatement(Unknown Source)
> at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown
> Source)
> at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
> at
> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
> Source)
> ... 73 more
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.openTxns(TxnHandler.java:651)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.open_txns(HiveMetaStore.java:8301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy46.open_txns(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxnsIntr(HiveMetaStoreClient.java:3634)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxn(HiveMetaStoreClient.java:3595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:218)
> at com.sun.proxy.$Proxy47.openTxn(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:243)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:227)
> at
> org.apache.hadoop.hive.ql.Compiler.openTransaction(Compiler.java:268)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:215)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:492)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:445)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:178)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:150)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:137)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.runCmd(TestHivePrivilegeObjectOwnerNameAndType.java:86)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.beforeTest(TestHivePrivilegeObjectOwnerNameAndType.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.ref
[jira] [Work logged] (HIVE-24562) Deflake TestHivePrivilegeObjectOwnerNameAndType
[
https://issues.apache.org/jira/browse/HIVE-24562?focusedWorklogId=527379&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527379
]
ASF GitHub Bot logged work on HIVE-24562:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 21:49
Start Date: 22/Dec/20 21:49
Worklog Time Spent: 10m
Work Description: vihangk1 commented on pull request #1809:
URL: https://github.com/apache/hive/pull/1809#issuecomment-749792041
> Why the test sometimes pass without the fix?
Yeah, I am not sure about this. The exception trace that I see suggests that
this should fail every time since datanucleus should automatically create the
tables.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527379)
Time Spent: 20m (was: 10m)
> Deflake TestHivePrivilegeObjectOwnerNameAndType
> ---
>
> Key: HIVE-24562
> URL: https://issues.apache.org/jira/browse/HIVE-24562
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> One of my unrelated PRs fails this test
> {{TestHivePrivilegeObjectOwnerNameAndType}}. The exception which I see in the
> logs is below:
> {noformat}
> Caused by: ERROR 42X05: Table/View 'TXN_LOCK_TBL' does not exist.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.sql.compile.LockTableNode.bindStatement(Unknown Source)
> at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown
> Source)
> at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
> at
> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
> Source)
> ... 73 more
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.openTxns(TxnHandler.java:651)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.open_txns(HiveMetaStore.java:8301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy46.open_txns(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxnsIntr(HiveMetaStoreClient.java:3634)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxn(HiveMetaStoreClient.java:3595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:218)
> at com.sun.proxy.$Proxy47.openTxn(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:243)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:227)
> at
> org.apache.hadoop.hive.ql.Compiler.openTransaction(Compiler.java:268)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:215)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:492)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:445)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:178)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:150)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:137)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.runCmd(TestHivePrivilegeObjectOwnerNameAndType.java:86)
> at
> org.ap
[jira] [Work logged] (HIVE-24561) Deflake TestCachedStoreUpdateUsingEvents
[
https://issues.apache.org/jira/browse/HIVE-24561?focusedWorklogId=527374&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527374
]
ASF GitHub Bot logged work on HIVE-24561:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 21:24
Start Date: 22/Dec/20 21:24
Worklog Time Spent: 10m
Work Description: vihangk1 commented on a change in pull request #1808:
URL: https://github.com/apache/hive/pull/1808#discussion_r547514156
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStoreUpdateUsingEvents.java
##
@@ -94,8 +96,9 @@ private Database createTestDb(String dbName, String dbOwner) {
}
private Table createTestTblParam(String dbName, String tblName, String
tblOwner,
- List cols, List
ptnCols, Map tblParams) {
-String serdeLocation = "file:/tmp";
+ List cols, List
ptnCols, Map tblParams)
+ throws IOException {
+String serdeLocation = Files.createTempDirectory(dbName + "_" +
tblName).toString();
Review comment:
same comment as above. Based on the documentation of
Files.createTempDirectory it returns "the path to the newly created directory
that did not exist before this method was invoked"
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527374)
Time Spent: 1h (was: 50m)
> Deflake TestCachedStoreUpdateUsingEvents
>
>
> Key: HIVE-24561
> URL: https://issues.apache.org/jira/browse/HIVE-24561
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> TestCachedStoreUpdateUsingEvents seems to use "file:/tmp" as the table and
> database directory. The cleanUp method will clean all the sub-directories
> directories in /tmp which can be error prone.
> Also noticed that I see a lot NPEs from
> {{SharedCache#getMemorySizeEstimator}} because the {{sizeEstimators}} field
> is null. We should add a null check for that field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24561) Deflake TestCachedStoreUpdateUsingEvents
[
https://issues.apache.org/jira/browse/HIVE-24561?focusedWorklogId=527373&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527373
]
ASF GitHub Bot logged work on HIVE-24561:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 21:23
Start Date: 22/Dec/20 21:23
Worklog Time Spent: 10m
Work Description: vihangk1 commented on a change in pull request #1808:
URL: https://github.com/apache/hive/pull/1808#discussion_r547513624
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStoreUpdateUsingEvents.java
##
@@ -82,9 +84,9 @@ public void setUp() throws Exception {
HiveMetaStore.HMSHandler.createDefaultCatalog(rawStore, new
Warehouse(conf));
}
- private Database createTestDb(String dbName, String dbOwner) {
+ private Database createTestDb(String dbName, String dbOwner) throws
IOException {
String dbDescription = dbName;
-String dbLocation = "file:/tmp";
+String dbLocation = Files.createTempDirectory(dbName).toString();
Review comment:
yeah, the Files.createTempDirectory creates a temp directory by
appending the timestamp. DbName is just used as a prefix for the directory name.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527373)
Time Spent: 50m (was: 40m)
> Deflake TestCachedStoreUpdateUsingEvents
>
>
> Key: HIVE-24561
> URL: https://issues.apache.org/jira/browse/HIVE-24561
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> TestCachedStoreUpdateUsingEvents seems to use "file:/tmp" as the table and
> database directory. The cleanUp method will clean all the sub-directories
> directories in /tmp which can be error prone.
> Also noticed that I see a lot NPEs from
> {{SharedCache#getMemorySizeEstimator}} because the {{sizeEstimators}} field
> is null. We should add a null check for that field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24561) Deflake TestCachedStoreUpdateUsingEvents
[
https://issues.apache.org/jira/browse/HIVE-24561?focusedWorklogId=527363&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527363
]
ASF GitHub Bot logged work on HIVE-24561:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 20:53
Start Date: 22/Dec/20 20:53
Worklog Time Spent: 10m
Work Description: kishendas commented on a change in pull request #1808:
URL: https://github.com/apache/hive/pull/1808#discussion_r547501638
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStoreUpdateUsingEvents.java
##
@@ -82,9 +84,9 @@ public void setUp() throws Exception {
HiveMetaStore.HMSHandler.createDefaultCatalog(rawStore, new
Warehouse(conf));
}
- private Database createTestDb(String dbName, String dbOwner) {
+ private Database createTestDb(String dbName, String dbOwner) throws
IOException {
String dbDescription = dbName;
-String dbLocation = "file:/tmp";
+String dbLocation = Files.createTempDirectory(dbName).toString();
Review comment:
Even the dbName can be same across multiple tests, which can get cleaned
up. Would it be a good idea to use the current timestamp or generate a random
number and prepend that to dbName and use that for the temporary directory ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527363)
Time Spent: 40m (was: 0.5h)
> Deflake TestCachedStoreUpdateUsingEvents
>
>
> Key: HIVE-24561
> URL: https://issues.apache.org/jira/browse/HIVE-24561
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> TestCachedStoreUpdateUsingEvents seems to use "file:/tmp" as the table and
> database directory. The cleanUp method will clean all the sub-directories
> directories in /tmp which can be error prone.
> Also noticed that I see a lot NPEs from
> {{SharedCache#getMemorySizeEstimator}} because the {{sizeEstimators}} field
> is null. We should add a null check for that field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24561) Deflake TestCachedStoreUpdateUsingEvents
[
https://issues.apache.org/jira/browse/HIVE-24561?focusedWorklogId=527362&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527362
]
ASF GitHub Bot logged work on HIVE-24561:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 20:53
Start Date: 22/Dec/20 20:53
Worklog Time Spent: 10m
Work Description: kishendas commented on a change in pull request #1808:
URL: https://github.com/apache/hive/pull/1808#discussion_r547501798
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStoreUpdateUsingEvents.java
##
@@ -94,8 +96,9 @@ private Database createTestDb(String dbName, String dbOwner) {
}
private Table createTestTblParam(String dbName, String tblName, String
tblOwner,
- List cols, List
ptnCols, Map tblParams) {
-String serdeLocation = "file:/tmp";
+ List cols, List
ptnCols, Map tblParams)
+ throws IOException {
+String serdeLocation = Files.createTempDirectory(dbName + "_" +
tblName).toString();
Review comment:
Even the dbName and tblName can be same across multiple tests, which can
get cleaned up. Would it be a good idea to use the current timestamp or
generate a random number and prepend that to dbName and use that for the
temporary directory ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527362)
Time Spent: 0.5h (was: 20m)
> Deflake TestCachedStoreUpdateUsingEvents
>
>
> Key: HIVE-24561
> URL: https://issues.apache.org/jira/browse/HIVE-24561
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> TestCachedStoreUpdateUsingEvents seems to use "file:/tmp" as the table and
> database directory. The cleanUp method will clean all the sub-directories
> directories in /tmp which can be error prone.
> Also noticed that I see a lot NPEs from
> {{SharedCache#getMemorySizeEstimator}} because the {{sizeEstimators}} field
> is null. We should add a null check for that field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24561) Deflake TestCachedStoreUpdateUsingEvents
[
https://issues.apache.org/jira/browse/HIVE-24561?focusedWorklogId=527361&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527361
]
ASF GitHub Bot logged work on HIVE-24561:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 20:52
Start Date: 22/Dec/20 20:52
Worklog Time Spent: 10m
Work Description: kishendas commented on a change in pull request #1808:
URL: https://github.com/apache/hive/pull/1808#discussion_r547501638
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStoreUpdateUsingEvents.java
##
@@ -82,9 +84,9 @@ public void setUp() throws Exception {
HiveMetaStore.HMSHandler.createDefaultCatalog(rawStore, new
Warehouse(conf));
}
- private Database createTestDb(String dbName, String dbOwner) {
+ private Database createTestDb(String dbName, String dbOwner) throws
IOException {
String dbDescription = dbName;
-String dbLocation = "file:/tmp";
+String dbLocation = Files.createTempDirectory(dbName).toString();
Review comment:
Even the dbName can be same across multiple tests, which can get cleaned
up. Would it be a good idea to use the current timestamp or generate a random
number and use that for the temporary directory ?
##
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/cache/TestCachedStoreUpdateUsingEvents.java
##
@@ -94,8 +96,9 @@ private Database createTestDb(String dbName, String dbOwner) {
}
private Table createTestTblParam(String dbName, String tblName, String
tblOwner,
- List cols, List
ptnCols, Map tblParams) {
-String serdeLocation = "file:/tmp";
+ List cols, List
ptnCols, Map tblParams)
+ throws IOException {
+String serdeLocation = Files.createTempDirectory(dbName + "_" +
tblName).toString();
Review comment:
Even the dbName and tblName can be same across multiple tests, which can
get cleaned up. Would it be a good idea to use the current timestamp or
generate a random number and use that for the temporary directory ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527361)
Time Spent: 20m (was: 10m)
> Deflake TestCachedStoreUpdateUsingEvents
>
>
> Key: HIVE-24561
> URL: https://issues.apache.org/jira/browse/HIVE-24561
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> TestCachedStoreUpdateUsingEvents seems to use "file:/tmp" as the table and
> database directory. The cleanUp method will clean all the sub-directories
> directories in /tmp which can be error prone.
> Also noticed that I see a lot NPEs from
> {{SharedCache#getMemorySizeEstimator}} because the {{sizeEstimators}} field
> is null. We should add a null check for that field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24562) Deflake TestHivePrivilegeObjectOwnerNameAndType
[
https://issues.apache.org/jira/browse/HIVE-24562?focusedWorklogId=527351&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527351
]
ASF GitHub Bot logged work on HIVE-24562:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 20:27
Start Date: 22/Dec/20 20:27
Worklog Time Spent: 10m
Work Description: vihangk1 opened a new pull request #1809:
URL: https://github.com/apache/hive/pull/1809
### What changes were proposed in this pull request?
This change modifies the test TestHivePrivilegeObjectOwnerNameAndType in the
hope that it will reduce the intermittent failures that I saw on my unrelated
PR. The full trace of the failure is available in the JIRA.
### Why are the changes needed?
Deflake the test.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Ran the modified test locally.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527351)
Remaining Estimate: 0h
Time Spent: 10m
> Deflake TestHivePrivilegeObjectOwnerNameAndType
> ---
>
> Key: HIVE-24562
> URL: https://issues.apache.org/jira/browse/HIVE-24562
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> One of my unrelated PRs fails this test
> {{TestHivePrivilegeObjectOwnerNameAndType}}. The exception which I see in the
> logs is below:
> {noformat}
> Caused by: ERROR 42X05: Table/View 'TXN_LOCK_TBL' does not exist.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.sql.compile.LockTableNode.bindStatement(Unknown Source)
> at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown
> Source)
> at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
> at
> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
> Source)
> ... 73 more
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.openTxns(TxnHandler.java:651)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.open_txns(HiveMetaStore.java:8301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy46.open_txns(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxnsIntr(HiveMetaStoreClient.java:3634)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxn(HiveMetaStoreClient.java:3595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:218)
> at com.sun.proxy.$Proxy47.openTxn(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:243)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:227)
> at
> org.apache.hadoop.hive.ql.Compiler.openTransaction(Compiler.java:268)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:215)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:492)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:445)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:178)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:150)
> at org.apache.hadoop.hive.
[jira] [Updated] (HIVE-24562) Deflake TestHivePrivilegeObjectOwnerNameAndType
[
https://issues.apache.org/jira/browse/HIVE-24562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-24562:
--
Labels: pull-request-available (was: )
> Deflake TestHivePrivilegeObjectOwnerNameAndType
> ---
>
> Key: HIVE-24562
> URL: https://issues.apache.org/jira/browse/HIVE-24562
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> One of my unrelated PRs fails this test
> {{TestHivePrivilegeObjectOwnerNameAndType}}. The exception which I see in the
> logs is below:
> {noformat}
> Caused by: ERROR 42X05: Table/View 'TXN_LOCK_TBL' does not exist.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.sql.compile.LockTableNode.bindStatement(Unknown Source)
> at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown
> Source)
> at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
> at
> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
> Source)
> ... 73 more
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.openTxns(TxnHandler.java:651)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.open_txns(HiveMetaStore.java:8301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy46.open_txns(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxnsIntr(HiveMetaStoreClient.java:3634)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxn(HiveMetaStoreClient.java:3595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:218)
> at com.sun.proxy.$Proxy47.openTxn(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:243)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:227)
> at
> org.apache.hadoop.hive.ql.Compiler.openTransaction(Compiler.java:268)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:215)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:492)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:445)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:178)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:150)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:137)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.runCmd(TestHivePrivilegeObjectOwnerNameAndType.java:86)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.beforeTest(TestHivePrivilegeObjectOwnerNameAndType.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
> at
> org.junit.internal.runners.statements.RunBefores.evalua
[jira] [Commented] (HIVE-24562) Deflake TestHivePrivilegeObjectOwnerNameAndType
[
https://issues.apache.org/jira/browse/HIVE-24562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253738#comment-17253738
]
Vihang Karajgaonkar commented on HIVE-24562:
I believe we should run {{TestTxnDbUtil.prepDb(conf);}} in the test setup
method since it creates the transactional schema. I have seen this to be used
in other tests like {{TestHiveAuthorizerCheckInvocation}} as well.
> Deflake TestHivePrivilegeObjectOwnerNameAndType
> ---
>
> Key: HIVE-24562
> URL: https://issues.apache.org/jira/browse/HIVE-24562
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
>
> One of my unrelated PRs fails this test
> {{TestHivePrivilegeObjectOwnerNameAndType}}. The exception which I see in the
> logs is below:
> {noformat}
> Caused by: ERROR 42X05: Table/View 'TXN_LOCK_TBL' does not exist.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.sql.compile.LockTableNode.bindStatement(Unknown Source)
> at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown
> Source)
> at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
> at
> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
> Source)
> ... 73 more
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.openTxns(TxnHandler.java:651)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.open_txns(HiveMetaStore.java:8301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy46.open_txns(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxnsIntr(HiveMetaStoreClient.java:3634)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxn(HiveMetaStoreClient.java:3595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:218)
> at com.sun.proxy.$Proxy47.openTxn(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:243)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:227)
> at
> org.apache.hadoop.hive.ql.Compiler.openTransaction(Compiler.java:268)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:215)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:492)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:445)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:178)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:150)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:137)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.runCmd(TestHivePrivilegeObjectOwnerNameAndType.java:86)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.beforeTest(TestHivePrivilegeObjectOwnerNameAndType.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.jun
[jira] [Assigned] (HIVE-24562) Deflake TestHivePrivilegeObjectOwnerNameAndType
[
https://issues.apache.org/jira/browse/HIVE-24562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar reassigned HIVE-24562:
--
> Deflake TestHivePrivilegeObjectOwnerNameAndType
> ---
>
> Key: HIVE-24562
> URL: https://issues.apache.org/jira/browse/HIVE-24562
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
>
> One of my unrelated PRs fails this test
> {{TestHivePrivilegeObjectOwnerNameAndType}}. The exception which I see in the
> logs is below:
> {noformat}
> Caused by: ERROR 42X05: Table/View 'TXN_LOCK_TBL' does not exist.
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at org.apache.derby.iapi.error.StandardException.newException(Unknown
> Source)
> at
> org.apache.derby.impl.sql.compile.LockTableNode.bindStatement(Unknown Source)
> at org.apache.derby.impl.sql.GenericStatement.prepMinion(Unknown
> Source)
> at org.apache.derby.impl.sql.GenericStatement.prepare(Unknown Source)
> at
> org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.prepareInternalStatement(Unknown
> Source)
> ... 73 more
> )
> at
> org.apache.hadoop.hive.metastore.txn.TxnHandler.openTxns(TxnHandler.java:651)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.open_txns(HiveMetaStore.java:8301)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:147)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:108)
> at com.sun.proxy.$Proxy46.open_txns(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxnsIntr(HiveMetaStoreClient.java:3634)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.openTxn(HiveMetaStoreClient.java:3595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:218)
> at com.sun.proxy.$Proxy47.openTxn(Unknown Source)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:243)
> at
> org.apache.hadoop.hive.ql.lockmgr.DbTxnManager.openTxn(DbTxnManager.java:227)
> at
> org.apache.hadoop.hive.ql.Compiler.openTransaction(Compiler.java:268)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:215)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:492)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:445)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:178)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:150)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:137)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.runCmd(TestHivePrivilegeObjectOwnerNameAndType.java:86)
> at
> org.apache.hadoop.hive.ql.security.authorization.plugin.TestHivePrivilegeObjectOwnerNameAndType.beforeTest(TestHivePrivilegeObjectOwnerNameAndType.java:82)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.RunBefores.invokeMethod(RunBefores.java:33)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> a
[jira] [Updated] (HIVE-24561) Deflake TestCachedStoreUpdateUsingEvents
[
https://issues.apache.org/jira/browse/HIVE-24561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-24561:
--
Labels: pull-request-available (was: )
> Deflake TestCachedStoreUpdateUsingEvents
>
>
> Key: HIVE-24561
> URL: https://issues.apache.org/jira/browse/HIVE-24561
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TestCachedStoreUpdateUsingEvents seems to use "file:/tmp" as the table and
> database directory. The cleanUp method will clean all the sub-directories
> directories in /tmp which can be error prone.
> Also noticed that I see a lot NPEs from
> {{SharedCache#getMemorySizeEstimator}} because the {{sizeEstimators}} field
> is null. We should add a null check for that field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24561) Deflake TestCachedStoreUpdateUsingEvents
[
https://issues.apache.org/jira/browse/HIVE-24561?focusedWorklogId=527339&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527339
]
ASF GitHub Bot logged work on HIVE-24561:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 20:02
Start Date: 22/Dec/20 20:02
Worklog Time Spent: 10m
Work Description: vihangk1 opened a new pull request #1808:
URL: https://github.com/apache/hive/pull/1808
### What changes were proposed in this pull request?
This PR modifies the test TestCachedStoreUpdateUsingEvents to not use
"file:/tmp" which is error prone since clean up method can delete all the files
in /tmp. Also fixes a harmless NPE which unnecessarily spams the log file.
### Why are the changes needed?
I observed this test to be failing on a unrelated PR. I am hoping that these
changes will help remove any flakiness if present.
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Ran the test locally.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527339)
Remaining Estimate: 0h
Time Spent: 10m
> Deflake TestCachedStoreUpdateUsingEvents
>
>
> Key: HIVE-24561
> URL: https://issues.apache.org/jira/browse/HIVE-24561
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> TestCachedStoreUpdateUsingEvents seems to use "file:/tmp" as the table and
> database directory. The cleanUp method will clean all the sub-directories
> directories in /tmp which can be error prone.
> Also noticed that I see a lot NPEs from
> {{SharedCache#getMemorySizeEstimator}} because the {{sizeEstimators}} field
> is null. We should add a null check for that field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-24561) Deflake TestCachedStoreUpdateUsingEvents
[
https://issues.apache.org/jira/browse/HIVE-24561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar reassigned HIVE-24561:
--
> Deflake TestCachedStoreUpdateUsingEvents
>
>
> Key: HIVE-24561
> URL: https://issues.apache.org/jira/browse/HIVE-24561
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
>
> TestCachedStoreUpdateUsingEvents seems to use "file:/tmp" as the table and
> database directory. The cleanUp method will clean all the sub-directories
> directories in /tmp which can be error prone.
> Also noticed that I see a lot NPEs from
> {{SharedCache#getMemorySizeEstimator}} because the {{sizeEstimators}} field
> is null. We should add a null check for that field.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-24556) Optimize DefaultGraphWalker for case when node has no grandchildren
[ https://issues.apache.org/jira/browse/HIVE-24556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved HIVE-24556. Fix Version/s: 4.0.0 Resolution: Fixed Pushed to master, thanks for your contribution [~jfs]! > Optimize DefaultGraphWalker for case when node has no grandchildren > --- > > Key: HIVE-24556 > URL: https://issues.apache.org/jira/browse/HIVE-24556 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Investigating query with large IN clause with constant strings (100k+) taking > significant time during compilation revealed a possible optimization within > DefaultGraphWalker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24556) Optimize DefaultGraphWalker for case when node has no grandchildren
[ https://issues.apache.org/jira/browse/HIVE-24556?focusedWorklogId=527327&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527327 ] ASF GitHub Bot logged work on HIVE-24556: - Author: ASF GitHub Bot Created on: 22/Dec/20 19:35 Start Date: 22/Dec/20 19:35 Worklog Time Spent: 10m Work Description: jcamachor merged pull request #1803: URL: https://github.com/apache/hive/pull/1803 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527327) Time Spent: 20m (was: 10m) > Optimize DefaultGraphWalker for case when node has no grandchildren > --- > > Key: HIVE-24556 > URL: https://issues.apache.org/jira/browse/HIVE-24556 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Investigating query with large IN clause with constant strings (100k+) taking > significant time during compilation revealed a possible optimization within > DefaultGraphWalker. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527317&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527317 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 19:03 Start Date: 22/Dec/20 19:03 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749721228 You can try to reproduce this via: ``` mvn clean install -DskipTests cd itests mvn clean install -DskipTests mvn test -Dtest=TestMiniDruidKafkaCliDriver -Dqfile=druidkafkamini_avro.q -Dtest.output.overwrite=true ``` although I got a different error: ``` FAILED: Execution Error, return code 4 from org.apache.hadoop.hive.ql.ddl.DDLTask. org.skife.jdbi.v2.exceptions.UnableToObtainConnectionException: java.sql.SQLException: Cannot create PoolableConnectionFactory (java.net.ConnectException : Error connecting to server localhost on port 8,084 with message Connection refused (Connection refused).) ``` It seems it's caused by this: ``` Exception in thread "main" java.lang.NoSuchMethodError: com.google.inject.util.Types.collectionOf(Ljava/lang/reflect/Type;)Ljava/lang/reflect/ParameterizedType; » at com.google.inject.multibindings.Multibinder.collectionOfProvidersOf(Multibinder.java:202) » at com.google.inject.multibindings.Multibinder$RealMultibinder.(Multibinder.java:283) » at com.google.inject.multibindings.Multibinder$RealMultibinder.(Multibinder.java:258) » at com.google.inject.multibindings.Multibinder.newRealSetBinder(Multibinder.java:178) » at com.google.inject.multibindings.Multibinder.newSetBinder(Multibinder.java:150) » at org.apache.druid.guice.LifecycleModule.getEagerBinder(LifecycleModule.java:115) » at org.apache.druid.guice.LifecycleModule.configure(LifecycleModule.java:121) » at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340) » at com.google.inject.spi.Elements.getElements(Elements.java:110) » at com.google.inject.util.Modules$OverrideModule.configure(Modules.java:177) » at com.google.inject.AbstractModule.configure(AbstractModule.java:62) » at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340) » at com.google.inject.spi.Elements.getElements(Elements.java:110) » at com.google.inject.util.Modules$OverrideModule.configure(Modules.java:177) » at com.google.inject.AbstractModule.configure(AbstractModule.java:62) » at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340) » at com.google.inject.spi.Elements.getElements(Elements.java:110) » at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138) » at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104) » at com.google.inject.Guice.createInjector(Guice.java:96) » at com.google.inject.Guice.createInjector(Guice.java:73) » at com.google.inject.Guice.createInjector(Guice.java:62) » at org.apache.druid.initialization.Initialization.makeInjectorWithModules(Initialization.java:431) » at org.apache.druid.cli.GuiceRunnable.makeInjector(GuiceRunnable.java:69) » at org.apache.druid.cli.ServerRunnable.run(ServerRunnable.java:58) » at org.apache.druid.cli.Main.main(Main.java:113) ``` which seems did not happen in the Jenkins CI. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527317) Time Spent: 6h 40m (was: 6.5h) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 6h 40m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527309&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527309 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 18:37 Start Date: 22/Dec/20 18:37 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749710426 BTW @iemejia I'm seeing this test consistently failing in all the test runs: ``` https://user-images.githubusercontent.com/506679/102921647-90edd200-4441-11eb-87e2-f9afc3dbcbb6.png";> Can you take a look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527309) Time Spent: 6h 20m (was: 6h 10m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 6h 20m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527310&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527310 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 18:37 Start Date: 22/Dec/20 18:37 Worklog Time Spent: 10m Work Description: sunchao edited a comment on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749710426 BTW @iemejia I'm seeing this test consistently failing in all the test runs: https://user-images.githubusercontent.com/506679/102921647-90edd200-4441-11eb-87e2-f9afc3dbcbb6.png";> Can you take a look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527310) Time Spent: 6.5h (was: 6h 20m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 6.5h > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527267&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527267 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 16:42 Start Date: 22/Dec/20 16:42 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #1806: URL: https://github.com/apache/hive/pull/1806#issuecomment-749646060 Thanks @iemejia . Can you fix the compilation error? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527267) Time Spent: 6h 10m (was: 6h) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 6h 10m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24560) Move Logging and Column Name,Type Parsing to AbstractSerde Class
[ https://issues.apache.org/jira/browse/HIVE-24560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-24560: -- Labels: pull-request-available (was: ) > Move Logging and Column Name,Type Parsing to AbstractSerde Class > > > Key: HIVE-24560 > URL: https://issues.apache.org/jira/browse/HIVE-24560 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24560) Move Logging and Column Name,Type Parsing to AbstractSerde Class
[ https://issues.apache.org/jira/browse/HIVE-24560?focusedWorklogId=527252&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527252 ] ASF GitHub Bot logged work on HIVE-24560: - Author: ASF GitHub Bot Created on: 22/Dec/20 15:50 Start Date: 22/Dec/20 15:50 Worklog Time Spent: 10m Work Description: belugabehr opened a new pull request #1807: URL: https://github.com/apache/hive/pull/1807 …e Class ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527252) Remaining Estimate: 0h Time Spent: 10m > Move Logging and Column Name,Type Parsing to AbstractSerde Class > > > Key: HIVE-24560 > URL: https://issues.apache.org/jira/browse/HIVE-24560 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24470) Separate HiveMetastore Thrift and Driver logic
[ https://issues.apache.org/jira/browse/HIVE-24470?focusedWorklogId=527245&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527245 ] ASF GitHub Bot logged work on HIVE-24470: - Author: ASF GitHub Bot Created on: 22/Dec/20 15:32 Start Date: 22/Dec/20 15:32 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #1787: URL: https://github.com/apache/hive/pull/1787#issuecomment-749600969 Thanks for the fixes, approved. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527245) Time Spent: 4h 50m (was: 4h 40m) > Separate HiveMetastore Thrift and Driver logic > -- > > Key: HIVE-24470 > URL: https://issues.apache.org/jira/browse/HIVE-24470 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Cameron Moberg >Assignee: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Time Spent: 4h 50m > Remaining Estimate: 0h > > In the file HiveMetastore.java the majority of the code is a thrift interface > rather than the actual logic behind starting hive metastore, this should be > moved out into a separate file to clean up the file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24470) Separate HiveMetastore Thrift and Driver logic
[ https://issues.apache.org/jira/browse/HIVE-24470?focusedWorklogId=527244&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527244 ] ASF GitHub Bot logged work on HIVE-24470: - Author: ASF GitHub Bot Created on: 22/Dec/20 15:31 Start Date: 22/Dec/20 15:31 Worklog Time Spent: 10m Work Description: miklosgergely commented on a change in pull request #1787: URL: https://github.com/apache/hive/pull/1787#discussion_r547343191 ## File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java ## @@ -0,0 +1,10189 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hive.metastore; + +import com.codahale.metrics.Counter; +import com.codahale.metrics.Timer; +import com.facebook.fb303.FacebookBase; +import com.facebook.fb303.fb_status; +import com.google.common.annotations.VisibleForTesting; +import com.google.common.base.Preconditions; +import com.google.common.base.Splitter; +import com.google.common.base.Supplier; +import com.google.common.base.Suppliers; +import com.google.common.collect.Lists; +import com.google.common.util.concurrent.ThreadFactoryBuilder; +import org.apache.commons.collections.CollectionUtils; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.FileStatus; +import org.apache.hadoop.fs.FileSystem; +import org.apache.hadoop.fs.Path; +import org.apache.hadoop.hive.common.AcidConstants; +import org.apache.hadoop.hive.common.AcidMetaDataFile; +import org.apache.hadoop.hive.common.StatsSetupConst; +import org.apache.hadoop.hive.common.TableName; +import org.apache.hadoop.hive.common.ValidReaderWriteIdList; +import org.apache.hadoop.hive.common.ValidWriteIdList; +import org.apache.hadoop.hive.common.repl.ReplConst; +import org.apache.hadoop.hive.metastore.api.*; +import org.apache.hadoop.hive.metastore.conf.MetastoreConf; +import org.apache.hadoop.hive.metastore.conf.MetastoreConf.ConfVars; +import org.apache.hadoop.hive.metastore.events.AbortTxnEvent; +import org.apache.hadoop.hive.metastore.events.AcidWriteEvent; +import org.apache.hadoop.hive.metastore.events.AddCheckConstraintEvent; +import org.apache.hadoop.hive.metastore.events.AddDefaultConstraintEvent; +import org.apache.hadoop.hive.metastore.events.AddForeignKeyEvent; +import org.apache.hadoop.hive.metastore.events.AddNotNullConstraintEvent; +import org.apache.hadoop.hive.metastore.events.AddPartitionEvent; +import org.apache.hadoop.hive.metastore.events.AddPrimaryKeyEvent; +import org.apache.hadoop.hive.metastore.events.AddSchemaVersionEvent; +import org.apache.hadoop.hive.metastore.events.AddUniqueConstraintEvent; +import org.apache.hadoop.hive.metastore.events.AllocWriteIdEvent; +import org.apache.hadoop.hive.metastore.events.AlterCatalogEvent; +import org.apache.hadoop.hive.metastore.events.AlterDatabaseEvent; +import org.apache.hadoop.hive.metastore.events.AlterISchemaEvent; +import org.apache.hadoop.hive.metastore.events.AlterPartitionEvent; +import org.apache.hadoop.hive.metastore.events.AlterSchemaVersionEvent; +import org.apache.hadoop.hive.metastore.events.AlterTableEvent; +import org.apache.hadoop.hive.metastore.events.CommitTxnEvent; +import org.apache.hadoop.hive.metastore.events.CommitCompactionEvent; +import org.apache.hadoop.hive.metastore.events.ConfigChangeEvent; +import org.apache.hadoop.hive.metastore.events.CreateCatalogEvent; +import org.apache.hadoop.hive.metastore.events.CreateDatabaseEvent; +import org.apache.hadoop.hive.metastore.events.CreateFunctionEvent; +import org.apache.hadoop.hive.metastore.events.CreateISchemaEvent; +import org.apache.hadoop.hive.metastore.events.CreateTableEvent; +import org.apache.hadoop.hive.metastore.events.DeletePartitionColumnStatEvent; +import org.apache.hadoop.hive.metastore.events.DeleteTableColumnStatEvent; +import org.apache.hadoop.hive.metastore.events.DropCatalogEvent; +import org.apache.hadoop.hive.metastore.events.DropConstraintEvent; +import org.apache.hadoop.hive.metastore.events.DropDatabaseEvent; +import org.apache.hadoop.hive.metastore.events.DropFunctionEvent; +import
[jira] [Updated] (HIVE-24560) Move Logging and Column Name,Type Parsing to AbstractSerde Class
[ https://issues.apache.org/jira/browse/HIVE-24560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-24560: -- Summary: Move Logging and Column Name,Type Parsing to AbstractSerde Class (was: Move Column Name and Type Parsing to AbstractSerde Class) > Move Logging and Column Name,Type Parsing to AbstractSerde Class > > > Key: HIVE-24560 > URL: https://issues.apache.org/jira/browse/HIVE-24560 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24560) Move Column Name and Type Parsing to AbstractSerde Class
[ https://issues.apache.org/jira/browse/HIVE-24560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253576#comment-17253576 ] David Mollitor commented on HIVE-24560: --- Builds on my work here: [HIVE-24332] > Move Column Name and Type Parsing to AbstractSerde Class > > > Key: HIVE-24560 > URL: https://issues.apache.org/jira/browse/HIVE-24560 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-24560) Move Column Name and Type Parsing to AbstractSerde Class
[ https://issues.apache.org/jira/browse/HIVE-24560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-24560: - > Move Column Name and Type Parsing to AbstractSerde Class > > > Key: HIVE-24560 > URL: https://issues.apache.org/jira/browse/HIVE-24560 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24535) Cleanup AcidUtils.Directory and remove unnecessary filesystem listings
[
https://issues.apache.org/jira/browse/HIVE-24535?focusedWorklogId=527239&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527239
]
ASF GitHub Bot logged work on HIVE-24535:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 15:25
Start Date: 22/Dec/20 15:25
Worklog Time Spent: 10m
Work Description: pvargacl commented on a change in pull request #1779:
URL: https://github.com/apache/hive/pull/1779#discussion_r547339344
##
File path: ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java
##
@@ -1316,7 +1282,7 @@ private Directory getAcidState() throws IOException {
}
-private AcidDirInfo callInternal() throws IOException {
+private AcidDirectory callInternal() throws IOException {
if (context.acidOperationalProperties != null
Review comment:
I played around this a bit and I think will do a follow up ticket for
mm, because this again is very messy.
For mm tables HiveInputformat will take the partition/table directory, find
the delta directories in it and validate them against the validwriteid list.
This is neccessary to handle every other file format. But the consequence of
this, that the OrcInputFormat#getSplits will be called with the delta
directories directly, not with the partition/table dir as it is done normally.
So this bypass actually will do a listing in the delta dir itself. And
"findOriginals" not only finds the original for MM tables it finds all of the
files in the table, and it is necessary otherwise we would not generate split
for normal bucketfiles. So the sort answer is that these originals is not the
same originals as the context of mm tables ...
The most disgusting part is, when you create an orc table that is not
transactional, this method will be called, and we will create a full
AcidDirectory just to list all the files in the table as originals.
OrcInputformat is very much interweaved with acid code.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527239)
Time Spent: 2h (was: 1h 50m)
> Cleanup AcidUtils.Directory and remove unnecessary filesystem listings
> --
>
> Key: HIVE-24535
> URL: https://issues.apache.org/jira/browse/HIVE-24535
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Varga
>Assignee: Peter Varga
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> * AcidUtils.getAcidState is doing a recursive listing on S3 FileSystem, it
> already knows the content of each delta and base directory, this could be
> returned to OrcInputFormat, to avoid listing each delta directory again there.
> * AcidUtils.getAcidstate submethods are collecting more and more infos about
> the state of the data directory. This could be done directly to the final
> Directory object to avoid 10+ parameters in methods.
> * AcidUtils.Directory, OrcInputFormat.AcidDirInfo and AcidUtils.TxnBase can
> be merged to one class, to clean up duplications.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527212&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527212 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 14:27 Start Date: 22/Dec/20 14:27 Worklog Time Spent: 10m Work Description: iemejia edited a comment on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749565575 @sunchao Done #1806 :crossed_fingers: This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527212) Time Spent: 6h (was: 5h 50m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 6h > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527211&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527211 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 14:26 Start Date: 22/Dec/20 14:26 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749565575 @sunchao Done #1806 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527211) Time Spent: 5h 50m (was: 5h 40m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 5h 50m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527210&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527210 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 14:26 Start Date: 22/Dec/20 14:26 Worklog Time Spent: 10m Work Description: iemejia opened a new pull request #1806: URL: https://github.com/apache/hive/pull/1806 R: @sunchao This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527210) Time Spent: 5h 40m (was: 5.5h) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 5h 40m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24559) Fix some spelling issues
[ https://issues.apache.org/jira/browse/HIVE-24559?focusedWorklogId=527206&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527206 ] ASF GitHub Bot logged work on HIVE-24559: - Author: ASF GitHub Bot Created on: 22/Dec/20 14:15 Start Date: 22/Dec/20 14:15 Worklog Time Spent: 10m Work Description: rickyma commented on pull request #1805: URL: https://github.com/apache/hive/pull/1805#issuecomment-749560289 I force pushed to fix more spelling mistakes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527206) Time Spent: 20m (was: 10m) > Fix some spelling issues > > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > There are some minor typos: > [https://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858|https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24559) Fix some spelling issues
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RickyMa updated HIVE-24559: --- Description: There are some minor typos: [https://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858|https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] was: There is a minor typo in TxnHandler: [https://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858|https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] > Fix some spelling issues > > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > There are some minor typos: > [https://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858|https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24559) Fix some spelling issues
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RickyMa updated HIVE-24559: --- Description: There is a minor typo in TxnHandler: [https://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858|https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] was: There is a minor typo in TxnHandler: [https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] fron -> from > Fix some spelling issues > > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > There is a minor typo in TxnHandler: > [https://github.com/apache/hive/pull/1805/fileshttps://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858|https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24559) Fix some spelling issues
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RickyMa updated HIVE-24559: --- Summary: Fix some spelling issues (was: Fix a spelling issue in TxnHandler) > Fix some spelling issues > > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > There is a minor typo in TxnHandler: > [https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] > > fron -> from -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24535) Cleanup AcidUtils.Directory and remove unnecessary filesystem listings
[ https://issues.apache.org/jira/browse/HIVE-24535?focusedWorklogId=527192&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527192 ] ASF GitHub Bot logged work on HIVE-24535: - Author: ASF GitHub Bot Created on: 22/Dec/20 13:36 Start Date: 22/Dec/20 13:36 Worklog Time Spent: 10m Work Description: klcopp commented on pull request #1779: URL: https://github.com/apache/hive/pull/1779#issuecomment-749543306 Got it, thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527192) Time Spent: 1h 50m (was: 1h 40m) > Cleanup AcidUtils.Directory and remove unnecessary filesystem listings > -- > > Key: HIVE-24535 > URL: https://issues.apache.org/jira/browse/HIVE-24535 > Project: Hive > Issue Type: Improvement >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Major > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > * AcidUtils.getAcidState is doing a recursive listing on S3 FileSystem, it > already knows the content of each delta and base directory, this could be > returned to OrcInputFormat, to avoid listing each delta directory again there. > * AcidUtils.getAcidstate submethods are collecting more and more infos about > the state of the data directory. This could be done directly to the final > Directory object to avoid 10+ parameters in methods. > * AcidUtils.Directory, OrcInputFormat.AcidDirInfo and AcidUtils.TxnBase can > be merged to one class, to clean up duplications. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24559) Fix a spelling issue in TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RickyMa updated HIVE-24559: --- Description: There is a minor typo in TxnHandler: [https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] fron -> from was: There is a minor typo in TxnHandler: https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858 > Fix a spelling issue in TxnHandler > -- > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > There is a minor typo in TxnHandler: > [https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] > > fron -> from -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-24559) Fix a spelling issue in TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17253496#comment-17253496 ] RickyMa commented on HIVE-24559: I've already created a pull request to fix this issue. > Fix a spelling issue in TxnHandler > -- > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > There is a minor typo in TxnHandler: > [https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858] > > fron -> from -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24559) Fix a spelling issue in TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RickyMa updated HIVE-24559: --- Description: There is a minor typo in TxnHandler: https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858 was:There is a minor typo in TxnHandler. > Fix a spelling issue in TxnHandler > -- > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > There is a minor typo in TxnHandler: > https://github.com/apache/hive/blob/branch-2.3/metastore/src/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L858 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24559) Fix a spelling issue in TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RickyMa updated HIVE-24559: --- Description: There is a minor typo in TxnHandler. > Fix a spelling issue in TxnHandler > -- > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > There is a minor typo in TxnHandler. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24559) Fix a spelling issue in TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-24559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-24559: -- Labels: pull-request-available (was: ) > Fix a spelling issue in TxnHandler > -- > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24559) Fix a spelling issue in TxnHandler
[ https://issues.apache.org/jira/browse/HIVE-24559?focusedWorklogId=527186&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527186 ] ASF GitHub Bot logged work on HIVE-24559: - Author: ASF GitHub Bot Created on: 22/Dec/20 13:26 Start Date: 22/Dec/20 13:26 Worklog Time Spent: 10m Work Description: rickyma opened a new pull request #1805: URL: https://github.com/apache/hive/pull/1805 ### What changes were proposed in this pull request? This pull request fixes a minor typo in TxnHandler. ### Why are the changes needed? This pull request fixes a minor typo in TxnHandler. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No need to be tested. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527186) Remaining Estimate: 0h Time Spent: 10m > Fix a spelling issue in TxnHandler > -- > > Key: HIVE-24559 > URL: https://issues.apache.org/jira/browse/HIVE-24559 > Project: Hive > Issue Type: Improvement >Reporter: RickyMa >Priority: Trivial > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24470) Separate HiveMetastore Thrift and Driver logic
[ https://issues.apache.org/jira/browse/HIVE-24470?focusedWorklogId=527140&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527140 ] ASF GitHub Bot logged work on HIVE-24470: - Author: ASF GitHub Bot Created on: 22/Dec/20 10:20 Start Date: 22/Dec/20 10:20 Worklog Time Spent: 10m Work Description: dataproc-metastore commented on a change in pull request #1787: URL: https://github.com/apache/hive/pull/1787#discussion_r547192325 ## File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HMSHandler.java ## @@ -0,0 +1,10189 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.hadoop.hive.metastore; + +import com.codahale.metrics.Counter; +import com.codahale.metrics.Timer; +import com.facebook.fb303.FacebookBase; +import com.facebook.fb303.fb_status; +import com.google.common.annotations.VisibleForTesting; +import com.google.common.base.Preconditions; +import com.google.common.base.Splitter; +import com.google.common.base.Supplier; +import com.google.common.base.Suppliers; +import com.google.common.collect.Lists; +import com.google.common.util.concurrent.ThreadFactoryBuilder; +import org.apache.commons.collections.CollectionUtils; +import org.apache.hadoop.conf.Configuration; +import org.apache.hadoop.fs.FileStatus; +import org.apache.hadoop.fs.FileSystem; +import org.apache.hadoop.fs.Path; +import org.apache.hadoop.hive.common.AcidConstants; +import org.apache.hadoop.hive.common.AcidMetaDataFile; +import org.apache.hadoop.hive.common.StatsSetupConst; +import org.apache.hadoop.hive.common.TableName; +import org.apache.hadoop.hive.common.ValidReaderWriteIdList; +import org.apache.hadoop.hive.common.ValidWriteIdList; +import org.apache.hadoop.hive.common.repl.ReplConst; +import org.apache.hadoop.hive.metastore.api.*; +import org.apache.hadoop.hive.metastore.conf.MetastoreConf; +import org.apache.hadoop.hive.metastore.conf.MetastoreConf.ConfVars; +import org.apache.hadoop.hive.metastore.events.AbortTxnEvent; +import org.apache.hadoop.hive.metastore.events.AcidWriteEvent; +import org.apache.hadoop.hive.metastore.events.AddCheckConstraintEvent; +import org.apache.hadoop.hive.metastore.events.AddDefaultConstraintEvent; +import org.apache.hadoop.hive.metastore.events.AddForeignKeyEvent; +import org.apache.hadoop.hive.metastore.events.AddNotNullConstraintEvent; +import org.apache.hadoop.hive.metastore.events.AddPartitionEvent; +import org.apache.hadoop.hive.metastore.events.AddPrimaryKeyEvent; +import org.apache.hadoop.hive.metastore.events.AddSchemaVersionEvent; +import org.apache.hadoop.hive.metastore.events.AddUniqueConstraintEvent; +import org.apache.hadoop.hive.metastore.events.AllocWriteIdEvent; +import org.apache.hadoop.hive.metastore.events.AlterCatalogEvent; +import org.apache.hadoop.hive.metastore.events.AlterDatabaseEvent; +import org.apache.hadoop.hive.metastore.events.AlterISchemaEvent; +import org.apache.hadoop.hive.metastore.events.AlterPartitionEvent; +import org.apache.hadoop.hive.metastore.events.AlterSchemaVersionEvent; +import org.apache.hadoop.hive.metastore.events.AlterTableEvent; +import org.apache.hadoop.hive.metastore.events.CommitTxnEvent; +import org.apache.hadoop.hive.metastore.events.CommitCompactionEvent; +import org.apache.hadoop.hive.metastore.events.ConfigChangeEvent; +import org.apache.hadoop.hive.metastore.events.CreateCatalogEvent; +import org.apache.hadoop.hive.metastore.events.CreateDatabaseEvent; +import org.apache.hadoop.hive.metastore.events.CreateFunctionEvent; +import org.apache.hadoop.hive.metastore.events.CreateISchemaEvent; +import org.apache.hadoop.hive.metastore.events.CreateTableEvent; +import org.apache.hadoop.hive.metastore.events.DeletePartitionColumnStatEvent; +import org.apache.hadoop.hive.metastore.events.DeleteTableColumnStatEvent; +import org.apache.hadoop.hive.metastore.events.DropCatalogEvent; +import org.apache.hadoop.hive.metastore.events.DropConstraintEvent; +import org.apache.hadoop.hive.metastore.events.DropDatabaseEvent; +import org.apache.hadoop.hive.metastore.events.DropFunctionEvent; +i
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527136&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527136 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 09:50 Start Date: 22/Dec/20 09:50 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749453686 @iemejia thanks, can you open a PR against branch-2.3? let's see how the tests go first. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527136) Time Spent: 5.5h (was: 5h 20m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 5.5h > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24548) CompactionHeartbeater leaks metastore connections
[ https://issues.apache.org/jira/browse/HIVE-24548?focusedWorklogId=527135&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527135 ] ASF GitHub Bot logged work on HIVE-24548: - Author: ASF GitHub Bot Created on: 22/Dec/20 09:49 Start Date: 22/Dec/20 09:49 Worklog Time Spent: 10m Work Description: klcopp merged pull request #1790: URL: https://github.com/apache/hive/pull/1790 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527135) Time Spent: 0.5h (was: 20m) > CompactionHeartbeater leaks metastore connections > - > > Key: HIVE-24548 > URL: https://issues.apache.org/jira/browse/HIVE-24548 > Project: Hive > Issue Type: Bug >Reporter: Peter Varga >Assignee: Peter Varga >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Every Heartbeater thread creates a new metastore client, that is never closed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24535) Cleanup AcidUtils.Directory and remove unnecessary filesystem listings
[
https://issues.apache.org/jira/browse/HIVE-24535?focusedWorklogId=527134&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527134
]
ASF GitHub Bot logged work on HIVE-24535:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 09:43
Start Date: 22/Dec/20 09:43
Worklog Time Spent: 10m
Work Description: pvargacl commented on a change in pull request #1779:
URL: https://github.com/apache/hive/pull/1779#discussion_r547173740
##
File path: ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java
##
@@ -1490,49 +1432,31 @@ else if (prev != null && next.maxWriteId ==
prev.maxWriteId
prev = next;
}
else {
-obsolete.add(next.path);
+directory.getObsolete().add(next.path);
}
}
+directory.getCurrentDirectories().clear();
+directory.getCurrentDirectories().addAll(deltas);
+ }
-if(bestBase.oldestBase != null && bestBase.basePath == null &&
-isCompactedBase(ParsedBase.parseBase(bestBase.oldestBase), fs,
dirSnapshots)) {
+ private static ValidTxnList getValidTxnList(Configuration conf) {
+ValidTxnList validTxnList = null;
+String s = conf.get(ValidTxnList.VALID_TXNS_KEY);
+if(!Strings.isNullOrEmpty(s)) {
/*
- * If here, it means there was a base_x (> 1 perhaps) but none were
suitable for given
- * {@link writeIdList}. Note that 'original' files are logically a
base_Long.MIN_VALUE and thus
- * cannot have any data for an open txn. We could check {@link deltas}
has files to cover
- * [1,n] w/o gaps but this would almost never happen...
+ * getAcidState() is sometimes called on non-transactional tables, e.g.
+ * OrcInputFileFormat.FileGenerator.callInternal(). e.g. orc_merge3.q
In that case
+ * writeIdList is bogus - doesn't even have a table name.
+ * see https://issues.apache.org/jira/browse/HIVE-20856.
*
- * We only throw for base_x produced by Compactor since that base erases
all history and
- * cannot be used for a client that has a snapshot in which something
inside this base is
- * open. (Nor can we ignore this base of course) But base_x which is a
result of IOW,
- * contains all history so we treat it just like delta wrt visibility.
Imagine, IOW which
- * aborts. It creates a base_x, which can and should just be ignored.*/
- long[] exceptions = writeIdList.getInvalidWriteIds();
- String minOpenWriteId = exceptions != null && exceptions.length > 0 ?
-Long.toString(exceptions[0]) : "x";
- throw new IOException(ErrorMsg.ACID_NOT_ENOUGH_HISTORY.format(
-Long.toString(writeIdList.getHighWatermark()),
- minOpenWriteId, bestBase.oldestBase.toString()));
-}
-
-Path base = null;
-boolean isBaseInRawFormat = false;
-if (bestBase.basePath != null) {
- base = bestBase.basePath;
- isBaseInRawFormat = MetaDataFile.isRawFormat(base, fs, dirSnapshots !=
null ? dirSnapshots.get(base) : null);
+ * For now, assert that ValidTxnList.VALID_TXNS_KEY is set only if this
is really a read
+ * of a transactional table.
+ * see {@link #getChildState(FileStatus, HdfsFileStatusWithId,
ValidWriteIdList, List, List, List, List, TxnBase, boolean, List, Map,
FileSystem, ValidTxnList)}
Review comment:
Removed this link, because it is outdated, i don't see anything related
to this in getChildState.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527134)
Time Spent: 1h 40m (was: 1.5h)
> Cleanup AcidUtils.Directory and remove unnecessary filesystem listings
> --
>
> Key: HIVE-24535
> URL: https://issues.apache.org/jira/browse/HIVE-24535
> Project: Hive
> Issue Type: Improvement
>Reporter: Peter Varga
>Assignee: Peter Varga
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> * AcidUtils.getAcidState is doing a recursive listing on S3 FileSystem, it
> already knows the content of each delta and base directory, this could be
> returned to OrcInputFormat, to avoid listing each delta directory again there.
> * AcidUtils.getAcidstate submethods are collecting more and more infos about
> the state of the data directory. This could be done directly to the final
> Directory object to avoid 10+ parameters in methods.
> * AcidUtils.Directory, OrcInputFormat.Acid
[jira] [Work started] (HIVE-24558) Handle update in table level regular expression.
[ https://issues.apache.org/jira/browse/HIVE-24558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-24558 started by Aasha Medhi. -- > Handle update in table level regular expression. > > > Key: HIVE-24558 > URL: https://issues.apache.org/jira/browse/HIVE-24558 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24558.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work stopped] (HIVE-24558) Handle update in table level regular expression.
[ https://issues.apache.org/jira/browse/HIVE-24558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-24558 stopped by Aasha Medhi. -- > Handle update in table level regular expression. > > > Key: HIVE-24558 > URL: https://issues.apache.org/jira/browse/HIVE-24558 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24558.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-24558) Handle update in table level regular expression.
[ https://issues.apache.org/jira/browse/HIVE-24558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-24558 started by Aasha Medhi. -- > Handle update in table level regular expression. > > > Key: HIVE-24558 > URL: https://issues.apache.org/jira/browse/HIVE-24558 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24558.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24558) Handle update in table level regular expression.
[ https://issues.apache.org/jira/browse/HIVE-24558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-24558: -- Labels: pull-request-available (was: ) > Handle update in table level regular expression. > > > Key: HIVE-24558 > URL: https://issues.apache.org/jira/browse/HIVE-24558 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24558.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24558) Handle update in table level regular expression.
[ https://issues.apache.org/jira/browse/HIVE-24558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-24558: --- Attachment: HIVE-24558.01.patch Status: Patch Available (was: In Progress) > Handle update in table level regular expression. > > > Key: HIVE-24558 > URL: https://issues.apache.org/jira/browse/HIVE-24558 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Labels: pull-request-available > Attachments: HIVE-24558.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24558) Handle update in table level regular expression.
[ https://issues.apache.org/jira/browse/HIVE-24558?focusedWorklogId=527131&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527131 ] ASF GitHub Bot logged work on HIVE-24558: - Author: ASF GitHub Bot Created on: 22/Dec/20 09:34 Start Date: 22/Dec/20 09:34 Worklog Time Spent: 10m Work Description: aasha opened a new pull request #1804: URL: https://github.com/apache/hive/pull/1804 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527131) Remaining Estimate: 0h Time Spent: 10m > Handle update in table level regular expression. > > > Key: HIVE-24558 > URL: https://issues.apache.org/jira/browse/HIVE-24558 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: HIVE-24558.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24554) TestCompactor fails to compile since HIVE-24477
[
https://issues.apache.org/jira/browse/HIVE-24554?focusedWorklogId=527124&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527124
]
ASF GitHub Bot logged work on HIVE-24554:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 09:17
Start Date: 22/Dec/20 09:17
Worklog Time Spent: 10m
Work Description: klcopp commented on pull request #1800:
URL: https://github.com/apache/hive/pull/1800#issuecomment-749438357
Thanks for taking care of this @abstractdog !!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 527124)
Time Spent: 40m (was: 0.5h)
> TestCompactor fails to compile since HIVE-24477
> ---
>
> Key: HIVE-24554
> URL: https://issues.apache.org/jira/browse/HIVE-24554
> Project: Hive
> Issue Type: Bug
>Reporter: László Bodor
>Assignee: László Bodor
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> http://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/PR-1798/1/pipeline
> {code}
> [2020-12-21T17:27:41.296Z] [INFO]
>
> [2020-12-21T17:27:41.296Z] [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile
> (default-testCompile) on project hive-it-unit: Compilation failure:
> Compilation failure:
> [2020-12-21T17:27:41.296Z] [ERROR]
> /home/jenkins/agent/workspace/hive-precommit_PR-1798/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java:[1257,17]
> cannot find symbol
> [2020-12-21T17:27:41.296Z] [ERROR] symbol: variable TxnDbUtil
> [2020-12-21T17:27:41.296Z] [ERROR] location: class
> org.apache.hadoop.hive.ql.txn.compactor.TestCompactor
> [2020-12-21T17:27:41.296Z] [ERROR]
> /home/jenkins/agent/workspace/hive-precommit_PR-1798/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCompactor.java:[1266,13]
> cannot find symbol
> [2020-12-21T17:27:41.296Z] [ERROR] symbol: variable TxnDbUtil
> [2020-12-21T17:27:41.296Z] [ERROR] location: class
> org.apache.hadoop.hive.ql.txn.compactor.TestCompactor
> [2020-12-21T17:27:41.296Z] [ERROR] -> [Help 1]
> [2020-12-21T17:27:41.296Z] [ERROR]
> [2020-12-21T17:27:41.296Z] [ERROR] To see the full stack trace of the errors,
> re-run Maven with the -e switch.
> [2020-12-21T17:27:41.296Z] [ERROR] Re-run Maven using the -X switch to enable
> full debug logging.
> [2020-12-21T17:27:41.296Z] [ERROR]
> [2020-12-21T17:27:41.296Z] [ERROR] For more information about the errors and
> possible solutions, please read the following articles:
> [2020-12-21T17:27:41.296Z] [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [2020-12-21T17:27:41.296Z] [ERROR]
> [2020-12-21T17:27:41.296Z] [ERROR] After correcting the problems, you can
> resume the build with the command
> [2020-12-21T17:27:41.296Z] [ERROR] mvn -rf :hive-it-unit
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24535) Cleanup AcidUtils.Directory and remove unnecessary filesystem listings
[
https://issues.apache.org/jira/browse/HIVE-24535?focusedWorklogId=527122&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527122
]
ASF GitHub Bot logged work on HIVE-24535:
-
Author: ASF GitHub Bot
Created on: 22/Dec/20 09:15
Start Date: 22/Dec/20 09:15
Worklog Time Spent: 10m
Work Description: pvargacl commented on a change in pull request #1779:
URL: https://github.com/apache/hive/pull/1779#discussion_r547159709
##
File path: ql/src/java/org/apache/hadoop/hive/ql/io/AcidDirectory.java
##
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.io;
+
+import org.apache.hadoop.fs.FileSystem;
+import org.apache.hadoop.fs.Path;
+import org.apache.hadoop.hdfs.DistributedFileSystem;
+import org.apache.hadoop.hive.shims.HadoopShims;
+import org.apache.hive.common.util.Ref;
+
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static
org.apache.hadoop.hive.ql.io.AcidUtils.AcidBaseFileType.ORIGINAL_BASE;
+
+/**
+ * AcidDirectory used to provide ACID directory layout information, which
directories and files to read.
+ * This representation only valid in a context of a ValidWriteIdList and
ValidTxnList.
+ */
+public final class AcidDirectory {
+
+ private final Path path;
+ private final FileSystem fs;
+ private final Ref useFileId;
+
+ private AcidUtils.ParsedBase base;
+ private AcidUtils.ParsedBaseLight oldestBase;
+
+ private final List abortedDirectories = new ArrayList<>();
+ private final Set abortedWriteIds = new HashSet<>();
+ private boolean unCompactedAborts;
+ private final List originalFiles = new
ArrayList<>();
+ private final List originalDirectories = new ArrayList<>();
+ private final List obsolete = new ArrayList<>();
+ private final List currentDirectories = new
ArrayList<>();
+
+ public AcidDirectory(Path path, FileSystem fs, Ref useFileId) {
+this.path = path;
+this.fs = fs;
+this.useFileId = useFileId;
+if (!(this.fs instanceof DistributedFileSystem) && this.useFileId != null)
{
+ this.useFileId.value = false;
+}
+ }
+
+ public Path getPath() {
+return path;
+ }
+
+ /**
+ * Get the base directory path.
+ * @return the base directory to read
+ */
+ public Path getBaseDirectory() {
+return base == null ? null : base.getBaseDirPath();
+ }
+
+ /**
+ * Get the base directory.
+ * @return the base directory to read
+ */
+ public AcidUtils.ParsedBase getBase() {
+return base;
+ }
+
+ /**
+ * Oldest base directory in the filesystem, may be shadowed by newer base
+ */
+ public AcidUtils.ParsedBaseLight getOldestBase() {
+return oldestBase;
+ }
+
+ public void setBase(AcidUtils.ParsedBase base) {
+this.base = base;
+ }
+
+ public void setOldestBase(AcidUtils.ParsedBaseLight oldestBase) {
+this.oldestBase = oldestBase;
+ }
+
+ public void setUnCompactedAborts(boolean unCompactedAborts) {
+this.unCompactedAborts = unCompactedAborts;
+ }
+
+ /**
+ * Is Base directory in raw format or in Acid format
+ */
+ public boolean isBaseInRawFormat() {
+return base != null && base.isRawFormat();
+ }
+
+ /**
+ * Get the list of original files. Not {@code null}. Must be sorted.
+ * @return the list of original files (eg. 00_0)
+ */
+ public List getOriginalFiles() {
+return originalFiles;
+ }
+
+ /**
+ * List of original directories containing files in not ACID format
+ */
+ public List getOriginalDirectories() {
+return originalDirectories;
+ }
+
+ /**
+ * Get the list of delta directories that are valid and not
+ * obsolete. Not {@code null}. List must be sorted in a specific way.
+ * See {@link
org.apache.hadoop.hive.ql.io.AcidUtils.ParsedDeltaLight#compareTo(org.apache.hadoop.hive.ql.io.AcidUtils.ParsedDeltaLight)}
+ * for details.
+ * @return the minimal list of current directories
+ */
+ public List getCurrentDirectories() {
+return currentDirectories;
+ }
+
+ /**
+
[jira] [Assigned] (HIVE-24558) Handle update in table level regular expression.
[ https://issues.apache.org/jira/browse/HIVE-24558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi reassigned HIVE-24558: -- > Handle update in table level regular expression. > > > Key: HIVE-24558 > URL: https://issues.apache.org/jira/browse/HIVE-24558 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527112&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527112 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 09:04 Start Date: 22/Dec/20 09:04 Worklog Time Spent: 10m Work Description: iemejia edited a comment on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749430903 @sunchao Great to know the cut is happening soon! If this is absolutely needed is a question of tradeoffs. Avro binary format has not changed since version 1.8.x when it introduced Logical Types, but more recent versions have removed dependencies from the public API (Jackson, Guava, JodaTime, etc) so I suppose catching up to the latest version can have the same risks as catching up to 1.8.x (which Hive already did) for the binary part. I suppose Hive users rarely use Avro directly from the transitive dependency and mostly rely on the Hive APIs (which I hope don't leak Avro) so this might diminish the risk, but of course there is a risk in that particular case. I am probably biased towards the upgrade because I come from the Avro side and I expect the full Big Data ecosystem to be updated and avoid issues because Hive contributors may introduce changes that are API incompatible with more recent versions of Avro and break downstream projects because of this (like the current parallel work on Spark). As usual in software it is all about tradeoffs. This decision is up to you guys as the maintainers and I might miss some other side effects in my analysis because I don't know Hive deeply. In any case if you guys decide to jump to the latest Avro version e.g. 1.10.1 and any issue happens I engage myself from the Avro side to do any fix and get out a release if required. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527112) Time Spent: 5h 20m (was: 5h 10m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 5h 20m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527111&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527111 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 09:02 Start Date: 22/Dec/20 09:02 Worklog Time Spent: 10m Work Description: iemejia edited a comment on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749430903 @sunchao Great to know the cut is happening soon! If this is absolutely needed is a question of tradeoffs. Avro binary format has not changed since version 1.8.x when it introduced Logical Types, but more recent versions have removed dependencies from the public API (Jackson, Guava, JodaTime, etc) so I suppose catching up to the latest version can have the same risks as catching up to 1.8.x which Hive already did for the binary part. I suppose Hive users rarely use Avro directly from the transitive dependency and mostly rely on the Hive APIs (which I hope don't leak Avro) so this might diminish the risk, but of course there is a risk in that particular case. I am probably biased towards the upgrade because I come from the Avro side and I expect the full Big Data ecosystem to be updated and avoid issues because Hive contributors may introduce changes that are API incompatible with more recent versions of Avro and break downstream projects because of this (like the current parallel work on Spark). As usual in software it is all about tradeoffs. This decision is up to you guys as the maintainers and I might miss some other side effects in my analysis because I don't know Hive deeply. In any case if you guys decide to jump to the latest Avro version e.g. 1.10.1 and any issue happens I engage myself from the Avro side to do any fix and get out a release if required. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527111) Time Spent: 5h 10m (was: 5h) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 5h 10m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527110&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527110 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 09:01 Start Date: 22/Dec/20 09:01 Worklog Time Spent: 10m Work Description: iemejia commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749430903 @sunchao Great to know the cut is happening soon! If this is absolutely needed is a question of tradeoffs. Avro binary format has not changed since version 1.8.x when it introduced Logical Types, but APIs have removed dependencies from the public API (Jackson, Guava, JodaTime, etc) so I suppose catching up to the latest version can have the same risks as catching up to 1.8.x which Hive already did for the binary part. I suppose Hive users rarely use Avro directly from the transitive dependency and mostly rely on the Hive APIs (which I hope don't leak Avro) so this might diminish the risk, but of course there is a risk in that particular case. I am probably biased towards the upgrade because I come from the Avro side and I expect the full Big Data ecosystem to be updated and avoid issues because Hive contributors may introduce changes that are API incompatible with more recent versions of Avro and break downstream projects because of this (like the current parallel work on Spark). As usual in software it is all about tradeoffs. This decision is up to you guys as the maintainers and I might miss some other side effects in my analysis because I don't know Hive deeply. In any case if you guys decide to jump to the latest Avro version e.g. 1.10.1 and any issue happens I engage myself from the Avro side to do any fix and get out a release if required. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527110) Time Spent: 5h (was: 4h 50m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 5h > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527108&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527108 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 08:57 Start Date: 22/Dec/20 08:57 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749428659 Yes I'm aware of that PR. The Avro upgrade is in Spark though, and the testing is done with previous Hive release candidates which do not include the Avro upgrade. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527108) Time Spent: 4h 50m (was: 4h 40m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 4h 50m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527107&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527107 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 08:50 Start Date: 22/Dec/20 08:50 Worklog Time Spent: 10m Work Description: h-vetinari commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749425677 I'm just guessing myself, but based on the [changes](https://github.com/apache/spark/pull/30517/files#r547146694) in https://github.com/apache/spark/pull/30517, I think a bumped Avro dependency in Hive 2.3.8 is what people were planning with. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527107) Time Spent: 4h 40m (was: 4.5h) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
\- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]\\w*");
+ public static final Pattern DDL_ID =
Pattern.compile("[a-zA-Z0-9]\\w*");
{quote}
was:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
\- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]\\\w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]\\\w*");
{quote}
> Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> \\{"error":"Invalid DDL identifier :db"\}
> {quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}—
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> \- public static final Pattern DDL_ID =
> Pattern.compile("[a-zA-Z]\\w*");
> + public static final Pattern DDL_ID =
> Pattern.compile("[a-zA-Z0-9]\\w*");
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
\- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]\\\w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]\\\w*");
{quote}
was:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
\- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]\\w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]\\w*");
{quote}
> Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> \\{"error":"Invalid DDL identifier :db"\}
> {quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}—
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> \- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]\\\w*");
> + public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]\\\w*");
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
\- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]\\w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]\\w*");
{quote}
was:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
\- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]w*");
{quote}
> Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> \\{"error":"Invalid DDL identifier :db"\}
> {quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}—
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> \- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]\\w*");
> + public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]\\w*");
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
\- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]w*");
{quote}
was:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}---
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
\- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");
{quote}
> Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> \\{"error":"Invalid DDL identifier :db"\}
> {quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}—
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> \- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]w*");
> + public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]w*");
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24470) Separate HiveMetastore Thrift and Driver logic
[ https://issues.apache.org/jira/browse/HIVE-24470?focusedWorklogId=527105&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527105 ] ASF GitHub Bot logged work on HIVE-24470: - Author: ASF GitHub Bot Created on: 22/Dec/20 08:38 Start Date: 22/Dec/20 08:38 Worklog Time Spent: 10m Work Description: dataproc-metastore commented on a change in pull request #1787: URL: https://github.com/apache/hive/pull/1787#discussion_r547141260 ## File path: standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/TSetIpAddressProcessor.java ## @@ -21,7 +21,7 @@ import java.lang.reflect.InvocationTargetException; import java.net.Socket; -import org.apache.hadoop.hive.metastore.HiveMetaStore.HMSHandler; +import org.apache.hadoop.hive.metastore.HMSHandler; Review comment: Fixed, thanks! ## File path: standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/IpAddressListener.java ## @@ -22,7 +22,7 @@ import java.net.UnknownHostException; import org.apache.hadoop.conf.Configuration; -import org.apache.hadoop.hive.metastore.HiveMetaStore.HMSHandler; +import org.apache.hadoop.hive.metastore.HMSHandler; Review comment: Fixed! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527105) Time Spent: 4h 10m (was: 4h) > Separate HiveMetastore Thrift and Driver logic > -- > > Key: HIVE-24470 > URL: https://issues.apache.org/jira/browse/HIVE-24470 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Cameron Moberg >Assignee: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Time Spent: 4h 10m > Remaining Estimate: 0h > > In the file HiveMetastore.java the majority of the code is a thrift interface > rather than the actual logic behind starting hive metastore, this should be > moved out into a separate file to clean up the file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24470) Separate HiveMetastore Thrift and Driver logic
[ https://issues.apache.org/jira/browse/HIVE-24470?focusedWorklogId=527106&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527106 ] ASF GitHub Bot logged work on HIVE-24470: - Author: ASF GitHub Bot Created on: 22/Dec/20 08:38 Start Date: 22/Dec/20 08:38 Worklog Time Spent: 10m Work Description: dataproc-metastore commented on a change in pull request #1787: URL: https://github.com/apache/hive/pull/1787#discussion_r547141450 ## File path: ql/src/java/org/apache/hadoop/hive/ql/security/authorization/StorageBasedAuthorizationProvider.java ## @@ -28,7 +28,7 @@ import javax.security.auth.login.LoginException; import org.apache.hadoop.hive.conf.Constants; Review comment: Removed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527106) Time Spent: 4h 20m (was: 4h 10m) > Separate HiveMetastore Thrift and Driver logic > -- > > Key: HIVE-24470 > URL: https://issues.apache.org/jira/browse/HIVE-24470 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Cameron Moberg >Assignee: Cameron Moberg >Priority: Minor > Labels: pull-request-available > Time Spent: 4h 20m > Remaining Estimate: 0h > > In the file HiveMetastore.java the majority of the code is a thrift interface > rather than the actual logic behind starting hive metastore, this should be > moved out into a separate file to clean up the file. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}---
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
\- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");
{quote}
was:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");{quote}
> Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> \\{"error":"Invalid DDL identifier :db"\}
> {quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}---
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> \- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
> w*");
> + public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
> w*");
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");{quote}
was:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
{"error":"Invalid DDL identifier :db"}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");{quote}
> Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> \\{"error":"Invalid DDL identifier :db"\}{quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}—
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> - public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
> w*");
> + public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
> w*");{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
{"error":"Invalid DDL identifier :db"}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");{quote}
was:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");
public static final Pattern PROPERTY_ID =
Pattern.compile("[a-zA-Z0-9][\\w\\.\\-|file:/-]*(? Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> {"error":"Invalid DDL identifier :db"}
> {quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}—
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> - public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
> w*");
> + public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
> w*");{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
{\"error\":"Invalid DDL identifier :db"}{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");
public static final Pattern PROPERTY_ID =
Pattern.compile("[a-zA-Z0-9][\\w\\.\\-|file:/-]*(? create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
{"error":"Invalid DDL identifier db"}{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");
public static final Pattern PROPERTY_ID =
Pattern.compile("[a-zA-Z0-9][\\w\\.\\-|file:/-]*(? Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> {\"error\":"Invalid DDL identifier :db"}{quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}—
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> - public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
> w*");
> + public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
> w*");
> public static final Pattern PROPERTY_ID =
> Pattern.compile("[a-zA-Z0-9][\\w\\.\\-|file:/-]*(? .)(? _)$");{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527097&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527097 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 08:29 Start Date: 22/Dec/20 08:29 Worklog Time Spent: 10m Work Description: sunchao commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749416248 Yes, but my impression is that [HIVE-24324](https://issues.apache.org/jira/browse/HIVE-24324) and [HIVE-24436](https://issues.apache.org/jira/browse/HIVE-24436) should be sufficient to solve the issue. But let me know if otherwise. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527097) Time Spent: 4.5h (was: 4h 20m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 4.5h > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
\\{"error":"Invalid DDL identifier :db"\}{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");
public static final Pattern PROPERTY_ID =
Pattern.compile("[a-zA-Z0-9][\\w\\.\\-|file:/-]*(? create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
{\"error\":"Invalid DDL identifier :db"}{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");
public static final Pattern PROPERTY_ID =
Pattern.compile("[a-zA-Z0-9][\\w\\.\\-|file:/-]*(? Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> \\{"error":"Invalid DDL identifier :db"\}{quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}—
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> - public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
> w*");
> + public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
> w*");
> public static final Pattern PROPERTY_ID =
> Pattern.compile("[a-zA-Z0-9][\\w\\.\\-|file:/-]*(? -)(? .)(? _)$");{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-24557) Misalignment in matching db name from Hive and Webhcat
[
https://issues.apache.org/jira/browse/HIVE-24557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Davide Vergari updated HIVE-24557:
---
Description:
I found a small misalignment on how Hive and WebHCat parse database name
starting with numbers.
If you execute:
{quote}hive> create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
{"error":"Invalid DDL identifier db"}{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}—
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
w*");
public static final Pattern PROPERTY_ID =
Pattern.compile("[a-zA-Z0-9][\\w\\.\\-|file:/-]*(? create database 2020db;
OK
Time taken: 0.44 seconds
hive>
{quote}
it works, but if you do the same on webhcatalog it doesn't:
{quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
"Test", "location":
"hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
HTTP/1.1 400 Bad Request
Set-Cookie:
hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
Path=/; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(7.6.0.v20120127)
{"error":"Invalid DDL identifier :db"}
{quote}
I suggest to harmonize the behavior, maybe applying a patch like the following
one:
{quote}---
a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
+++
b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
@@ -1068,7 +1068,7 @@ public void verifyParam(List param, String name)
}
}
- public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]\\w*");
+ public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]\\w*");
public static final Pattern PROPERTY_ID =
Pattern.compile("[a-zA-Z0-9][\\w\\.\\-]*(? Misalignment in matching db name from Hive and Webhcat
> --
>
> Key: HIVE-24557
> URL: https://issues.apache.org/jira/browse/HIVE-24557
> Project: Hive
> Issue Type: Bug
> Components: WebHCat
>Affects Versions: 2.3.6
>Reporter: Davide Vergari
>Priority: Minor
>
> I found a small misalignment on how Hive and WebHCat parse database name
> starting with numbers.
> If you execute:
> {quote}hive> create database 2020db;
> OK
> Time taken: 0.44 seconds
> hive>
> {quote}
> it works, but if you do the same on webhcatalog it doesn't:
> {quote}curl -i -s -X PUT -HContent-type:application/json -d '\{"comment":
> "Test", "location":
> "hdfs://singlenode.localdomain:8020/apps/hive/warehouse/2020db"}'
> 'http://singlenode.localdomain:50111/templeton/v1/ddl/database/2020db?user.name=hdfs'
> HTTP/1.1 400 Bad Request
> Set-Cookie:
> hadoop.auth="u=hdfs&p=hdfs&t=simple&e=1608581595777&s=JzsCUqvyyAYHykAHdteybh1tI4jFLLVVJC5D9FaUu0A=";
> Path=/; HttpOnly
> Content-Type: application/json
> Transfer-Encoding: chunked
> Server: Jetty(7.6.0.v20120127)
> {"error":"Invalid DDL identifier db"}{quote}
>
> I suggest to harmonize the behavior, maybe applying a patch like the
> following one:
>
> {quote}—
> a/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> +++
> b/hcatalog/webhcat/svr/src/main/java/org/apache/hive/hcatalog/templeton/Server.java
> @@ -1068,7 +1068,7 @@ public void verifyParam(List param, String
> name)
> }
> }
> - public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z]
> w*");
> + public static final Pattern DDL_ID = Pattern.compile("[a-zA-Z0-9]
> w*");
> public static final Pattern PROPERTY_ID =
> Pattern.compile("[a-zA-Z0-9][\\w\\.\\-|file:/-]*(? _)$");{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-21737) Upgrade Avro to version 1.10.1
[ https://issues.apache.org/jira/browse/HIVE-21737?focusedWorklogId=527094&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-527094 ] ASF GitHub Bot logged work on HIVE-21737: - Author: ASF GitHub Bot Created on: 22/Dec/20 08:25 Start Date: 22/Dec/20 08:25 Worklog Time Spent: 10m Work Description: h-vetinari commented on pull request #1635: URL: https://github.com/apache/hive/pull/1635#issuecomment-749414690 I'm not speaking for @iemejia of course, but my impression was that this patch was necessary for Hive 2.3.8 in order to (among other things) unblock spark from using a newer parquet (and thus avro) version. See linked issues in [HIVE-21737](https://issues.apache.org/jira/browse/HIVE-21737), [SPARK-27733](https://issues.apache.org/jira/browse/SPARK-27733), etc. The interlocking dependencies on this have been a very thorny problem that people have tried working on for a long time (e.g. #674, #785), so it would be really cool if this didn't miss the release. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 527094) Time Spent: 4h 20m (was: 4h 10m) > Upgrade Avro to version 1.10.1 > -- > > Key: HIVE-21737 > URL: https://issues.apache.org/jira/browse/HIVE-21737 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ismaël Mejía >Assignee: Fokko Driesprong >Priority: Major > Labels: pull-request-available > Attachments: > 0001-HIVE-21737-Make-Avro-use-in-Hive-compatible-with-Avr.patch > > Time Spent: 4h 20m > Remaining Estimate: 0h > > Avro >= 1.9.x bring a lot of fixes including a leaner version of Avro without > Jackson in the public API and Guava as a dependency. Worth the update. -- This message was sent by Atlassian Jira (v8.3.4#803005)
