[jira] [Created] (HIVE-26522) Metastore DelegationToken renewal is ineffective

[Pavan Lanka (Jira)]

Pavan Lanka created HIVE-26522:
--

 Summary: Metastore DelegationToken renewal is ineffective
 Key: HIVE-26522
 URL: https://issues.apache.org/jira/browse/HIVE-26522
 Project: Hive
  Issue Type: Bug
  Components: Standalone Metastore
Affects Versions: 3.1.3, 2.3.8
Reporter: Pavan Lanka


The HMS currently exposes method to renew an obtained delegation token
{code:java}
@Override
public long renewDelegationToken(String tokenStrForm) throws MetaException, 
TException {
  if (localMetaStore) {
return 0;
  }
  return client.renew_delegation_token(tokenStrForm);

}{code}
However on the server side, the renewal of the delegation token does not result 
in the update of the token information with the updated expiry
{code:java}
@Override
public long renewToken(Token token, String renewer) 
throws IOException {
  // since renewal is KERBEROS authenticated token may not be cached
  final DelegationTokenIdentifier id = getTokenIdentifier(token);
  DelegationTokenInformation tokenInfo = this.tokenStore.getToken(id);
  if (tokenInfo == null) {
  throw new InvalidToken("token does not exist: " + id); // no token found
  }
  // ensure associated master key is available
  if (!super.allKeys.containsKey(id.getMasterKeyId())) {
LOGGER.info("Unknown master key (id={}), (re)loading keys from token 
store.",
  id.getMasterKeyId());
reloadKeys();
  }
  // reuse super renewal logic
  synchronized (this) {
--> super.currentTokens.put(id,  tokenInfo);
try {
-->   return super.renewToken(token, renewer);
} finally {
-->   super.currentTokens.remove(id);
}
  }
} {code}
Here you can see that we populate the `super.currentTokens` perform the renewal 
and then remove the token without updating the `tokenStore`

 

As a result of this even though the call for renewal is successful the renewal 
time is not updated for the token and the token is invalidated based on the 
initial expiry time i.e based on when the token was created.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [VOTE] Should we release Hive Storage API 2.7.3-rc2?

[Pavan Lanka]

+1 (non-binding)

* Built Hive Storage API (did not get the failure that Alan called out) using 
the tag
* Ran the individual test TestBytesColumnVector successfully
* Built ORC using 2.7.3 successfully

Regards,
Pavan

> On Aug 2, 2021, at 4:03 PM, Alan Gates  wrote:
> 
> +1.  Checked the keys and the sha checksum.  Did a build and ran the
> tests.  Ran rat.  I did get one failing test, but it doesn't look crucial:
> 
> TestBytesColumnVector.testConcatAndPadding:109 row 0 expected:<[?]> but
> was:<[???]>
> 
> Alan.
> 
> On Thu, Jul 29, 2021 at 3:14 PM Owen O'Malley 
> wrote:
> 
>> Hello all,
>> 
>> Following on previous discussions, I would like to propose a new
>> storage-api release including HIVE-25400.
>> 
>> Shall we release the following artifacts as Hive Storage API 2.7.3?
>> 
>> tar: http://home.apache.org/~omalley/hive-storage-2.7.3/
>> tag: https://github.com/apache/hive/releases/tag/storage-release-2.7.3-rc2
>> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350287
>> 
>> .. Owen
>> 

Re: [VOTE] Should we release Hive Storage API 2.8.1 rc2?

[Pavan Lanka]

+1 (non-binding)

* Built the release
* Built ORC using 2.8.1
* Ran the benchmarks compactExpression on the SArg

Regards,
Pavan

> On Aug 2, 2021, at 4:09 PM, Alan Gates  wrote:
> 
> +1.  Built it, ran rat, checked the signature and the hash.
> 
> Alan.
> 
> On Thu, Jul 29, 2021 at 3:13 PM Owen O'Malley 
> wrote:
> 
>> Hello all,
>> 
>> I would like to propose a new storage-api release including HIVE-25400.
>> 
>> Shall we release the following artifacts as Hive Storage API 2.8.1?
>> 
>> tar: http://home.apache.org/~omalley/hive-storage-2.8.1/
>> tag: https://github.com/apache/hive/releases/tag/storage-release-2.8.1-rc2
>> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350456
>> 
>> .. Owen
>> 

Re: [VOTE] Should we release Hive Storage API 2.8.0-rc0 ?

[Pavan Lanka]

+1 (non-binding)

I have done the following:
* Built and Tested storage-release-2.8.0-rc0 using OpenJDK8
* Built and Tested ORC with updated storage api version 
  - Had to fix a test class that implements PredicateLeaf which has a new 
method. This is a breaking change but I think this should be ok
* Verified the performance gains of HIVE-24458

Regards,
Pavan


> On Jun 21, 2021, at 8:07 AM, Panos Garefalakis  wrote:
> 
> Hello all,
> 
> Following on previous discussions, I would like to propose a new
> storage-api release including HIVE-24458
> .
> 
> Shall we release the following artifacts as Hive Storage API 2.8.0?
> 
> tar: http://home.apache.org/~pgaref/hive-storage-2.8.0/
> tag: https://github.com/apache/hive/releases/tag/storage-release-2.8.0-rc0
> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350287
> 
> Cheers,
> Panagiotis

Re: [VOTE] Should we release Hive Storage API 2.7.3-rc0 ?

[Pavan Lanka]

+1 (non binding)

Built and verified unit tests
Built and tested ORC
Verified HIVE-24458 functionality and performance gains

Regards,
Pavan


> On Jun 8, 2021, at 2:14 AM, Panos Garefalakis  wrote:
> 
> Hello all,
> 
> I would like to propose a new storage-api release with HIVE-24458
>  in it (as discussed in a
> previous thead).
> 
> Shall we release the following artifacts as Hive Storage API 2.7.3?
> 
> tar: http://home.apache.org/~pgaref/hive-storage-2.7.3/
> tag: https://github.com/apache/hive/releases/tag/storage-release-2.7.3-rc0
> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350287
> 
> Cheers,
> Panagiotis

Requesting a new storage release

[Pavan Lanka]

Hi,

I wanted to request a release for the storage-api. We are interested in 
HIVE-24458 . 
Please let me know if there is a process that I need to follow to request for 
the same.

Thanks,
Pavan

[jira] [Updated] (HIVE-8732) ORC string statistics are not merged correctly

[Pavan Lanka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavan Lanka updated HIVE-8732:
--
Affects Version/s: 0.11.0
   0.12.0
   0.13.0

 ORC string statistics are not merged correctly
 --

 Key: HIVE-8732
 URL: https://issues.apache.org/jira/browse/HIVE-8732
 Project: Hive
  Issue Type: Bug
  Components: File Formats
Affects Versions: 0.11.0, 0.12.0, 0.13.0
Reporter: Owen O'Malley
Assignee: Owen O'Malley
Priority: Blocker
 Fix For: 0.14.0

 Attachments: HIVE-8732.patch, HIVE-8732.patch, HIVE-8732.patch


 Currently ORC's string statistics do not merge correctly causing incorrect 
 maximum values.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Edit access to the Hive Wiki

[Pavan Lanka]

Hi,

I have registered myself as pavibhai on the https://cwiki.apache.org and
would like change privilege so that I can contribute to the content.


Regards,
Pavan