[jira] [Created] (HIVE-26966) Hive is unable to delete Azure storage objects
László Végh created HIVE-26966:
--
Summary: Hive is unable to delete Azure storage objects
Key: HIVE-26966
URL: https://issues.apache.org/jira/browse/HIVE-26966
Project: Hive
Issue Type: Improvement
Reporter: László Végh
While writing data on cloud hive uses the expected RAZ authenticated way (using
the access by Managed Identity), HiveProtoEventsCleanerTask is following a
different approach, and tries to delete the data using the directory owner,
which may not available in Ranger.
To solve this issue either
* investigate how authentication works for data writing and implement it for
deletion as well (preferred solution)
* or introduce a new configuration value holding the name of the user who
needs to be used for deleting the data.
related hadoop logs:
{code:java}
2022-12-07 11:30:07,163 WARN
org.apache.hadoop.security.ShellBasedUnixGroupsMapping: [pool-310888-thread-7]:
unable to return groups for user 9ffea8fa-dec1-49ea-bb45-72bcb43951e8
org.apache.hadoop.security.ShellBasedUnixGroupsMapping$PartialGroupNameException:
The user name '9ffea8fa-dec1-49ea-bb45-72bcb43951e8' is not found. id:
9ffea8fa-dec1-49ea-bb45-72bcb43951e8: no such user
id: 9ffea8fa-dec1-49ea-bb45-72bcb43951e8: no such user
2022-12-07 11:30:07,164 ERROR
org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore: [pool-310888-thread-7]:
Failed to get primary group for 9ffea8fa-dec1-49ea-bb45-72bcb43951e8, using
user name as primary group name
2022-12-07 11:30:07,231 INFO
org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolManager:
[TezSessionPool-expiration]: Created new tez session for queue: default with
session id: df027903-43dd-46a8-b654-a25834f2b90d
{code}
ranger logs:
{code:java}
2022-12-07 11:29:20,693 INFO
org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager:
Token cancellation requested for identifier: (ABFS delegation owner=hive,
renewer=yarn, realUser=, issueDate=1670411627352, maxDate=1671016427352,
sequenceNumber=24065, masterKeyId=95)
2022-12-07 11:30:07,316 WARN
org.apache.hadoop.security.ShellBasedUnixGroupsMapping: unable to return groups
for user 9ffea8fa-dec1-49ea-bb45-72bcb43951e8
PartialGroupNameException The user name '9ffea8fa-dec1-49ea-bb45-72bcb43951e8'
is not found. id: 9ffea8fa-dec1-49ea-bb45-72bcb43951e8: no such user
id: 9ffea8fa-dec1-49ea-bb45-72bcb43951e8: no such user
2022-12-07 11:30:07,317 ERROR org.apache.ranger.raz.rest.AuthzREST:
AuthzREST.authorizeAccess()
org.apache.ranger.raz.intg.RangerRazException: not authorized to perform
delete-recursive on path
abfs://d...@s05p1appcdp001.dfs.core.windows.net/warehouse/tablespace/external/hive/sys.db/query_data/date=2021-08-20
at
org.apache.ranger.raz.processor.adls.AdlsGen2RazProcessor.generateDSASToken(AdlsGen2RazProcessor.java:216)
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (HIVE-26965) Docker image for Apache Hive
Zhihua Deng created HIVE-26965: -- Summary: Docker image for Apache Hive Key: HIVE-26965 URL: https://issues.apache.org/jira/browse/HIVE-26965 Project: Hive Issue Type: Improvement Reporter: Zhihua Deng This feature work is to provide docker image for Hive and track further improvements. -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: [ANNOUNCE] New PMC Member: Stamatis Zampetakis
Congrats Stamatis! Happy to see that Hive PMC is growing with active members, keep on the good work! Stamatis Zampetakis ezt írta (időpont: 2023. jan. 16., H, 14:28): > Thanks everyone! I am very glad and honoured to join the PMC. > > I really enjoy being part of this community and It is great interacting > with all of you on a daily basis; thank you for being part of this! > > Best, > Stamatis > > On Mon, Jan 16, 2023 at 2:12 PM Jiajun Xie > wrote: > > > Congratulations Stamatis :) > > Very well deserved!!! > > > > On Mon, 16 Jan 2023 at 13:51, Krisztian Kasa > > > wrote: > > > > > Congratulations Stamatis :) > > > > > > On Mon, Jan 16, 2023 at 6:27 AM S T wrote: > > > > > > > Congrats Stamatis. > > > > > > > > Thanks > > > > > > > > On Sat, 14 Jan 2023 at 00:03, Naveen Gangam > > > wrote: > > > > > > > >> Hello Hive Community, > > > >> Apache Hive PMC is pleased to announce that Stamatis Zampetakis has > > > >> accepted the Apache Hive PMC's invitation to become PMC Member, and > is > > > now > > > >> our newest PMC member. Please join me in congratulating Stamatis !!! > > > >> > > > >> He has been an active member in the hive community across many > aspects > > > of > > > >> the project. Many thanks to Stamatis for all the contributions he > has > > > made > > > >> and looking forward to many more future contributions in the > expanded > > > role. > > > >> > > > >> Cheers, > > > >> Naveen (on behalf of Hive PMC) > > > >> > > > > > > > > > >
Re: Moderators for Hive mailing lists
I will join as well -Ayush > On 18-Jan-2023, at 4:16 PM, Stamatis Zampetakis wrote: > > Hi all, > > It appears that most of the current moderators of the Hive mailing lists > are not very active in the project thus messages and subscriptions may take > a while to be approved. > > I am planning to request myself to be added as moderator to all lists but > it would be could nice if two more people could join this effort. Due to > the nature of the mailing lists, some of which are private, these people > must be in the PMC. > > Mailing lists of the project: > * comm...@hive.apache.org > * dev@hive.apache.org > * git...@hive.apache.org > * iss...@hive.apache.org > * priv...@hive.apache.org > * jira-reque...@hive.apache.org > * secur...@hive.apache.org > * u...@hive.apache.org > > Best, > Stamatis
Moderators for Hive mailing lists
Hi all, It appears that most of the current moderators of the Hive mailing lists are not very active in the project thus messages and subscriptions may take a while to be approved. I am planning to request myself to be added as moderator to all lists but it would be could nice if two more people could join this effort. Due to the nature of the mailing lists, some of which are private, these people must be in the PMC. Mailing lists of the project: * comm...@hive.apache.org * dev@hive.apache.org * git...@hive.apache.org * iss...@hive.apache.org * priv...@hive.apache.org * jira-reque...@hive.apache.org * secur...@hive.apache.org * u...@hive.apache.org Best, Stamatis
[jira] [Created] (HIVE-26963) Unset repl.faliover.endpoint during second cycle of optimized bootstrap.
Rakshith C created HIVE-26963: - Summary: Unset repl.faliover.endpoint during second cycle of optimized bootstrap. Key: HIVE-26963 URL: https://issues.apache.org/jira/browse/HIVE-26963 Project: Hive Issue Type: Bug Reporter: Rakshith C Assignee: Rakshith C Scenario: When second cycle of optimized bootstrap is initiated from DR to PROD. repl.failover.endpoint is not unset on DR. Due to this background threads like statsUpdater, partitionManagement etc. ignore the replicated db on DR. -- This message was sent by Atlassian Jira (v8.20.10#820010)
