[jira] [Created] (HIVE-25417) Null bit vector is not handled while getting the stats for Postgres backend

2021-08-02 Thread mahesh kumar behera (Jira)
mahesh kumar behera created HIVE-25417:
--

 Summary: Null bit vector is not handled while getting the stats 
for Postgres backend
 Key: HIVE-25417
 URL: https://issues.apache.org/jira/browse/HIVE-25417
 Project: Hive
  Issue Type: Sub-task
  Components: HiveServer2
Reporter: mahesh kumar behera
Assignee: mahesh kumar behera


While adding stats with null bit vector, a special string "HL" is added as 
Postgres does not support null value for byte columns. But while getting the 
stats, the conversion to null is not done. This is causing failure during 
deserialisation of bit vector field if the existing stats is used for merge.

 
{code:java}
 The input stream is not a HyperLogLog stream.  7276-1 instead of 727676 or 
7077^Mat 
org.apache.hadoop.hive.common.ndv.hll.HyperLogLogUtils.checkMagicString(HyperLogLogUtils.java:349)^M
 at 
org.apache.hadoop.hive.common.ndv.hll.HyperLogLogUtils.deserializeHLL(HyperLogLogUtils.java:139)^M
   at 
org.apache.hadoop.hive.common.ndv.hll.HyperLogLogUtils.deserializeHLL(HyperLogLogUtils.java:213)^M
   at 
org.apache.hadoop.hive.common.ndv.hll.HyperLogLogUtils.deserializeHLL(HyperLogLogUtils.java:227)^M
   at 
org.apache.hadoop.hive.common.ndv.NumDistinctValueEstimatorFactory.getNumDistinctValueEstimator(NumDistinctValueEstimatorFactory.java:53)^M
  at 
org.apache.hadoop.hive.metastore.columnstats.cache.LongColumnStatsDataInspector.updateNdvEstimator(LongColumnStatsDataInspector.java:124)^M
  at 
org.apache.hadoop.hive.metastore.columnstats.cache.LongColumnStatsDataInspector.getNdvEstimator(LongColumnStatsDataInspector.java:107)^M
 at 
org.apache.hadoop.hive.metastore.columnstats.merge.LongColumnStatsMerger.merge(LongColumnStatsMerger.java:36)^M
  at 
org.apache.hadoop.hive.metastore.utils.MetaStoreUtils.mergeColStats(MetaStoreUtils.java:1174)^M
  at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.updateTableColumnStatsWithMerge(HiveMetaStore.java:8934)^M
 at 
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.set_aggr_stats_for(HiveMetaStore.java:8800)^M
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)^M
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)^M 
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)^M
  at java.lang.reflect.Method.invoke(Method.java:498)^M   at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:160)^M
at 
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:121)^M
at com.sun.proxy.$Proxy35.set_aggr_stats_for(Unknown Source)^M  at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$set_aggr_stats_for.getResult(ThriftHiveMetastore.java:20489)^M
at 
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Processor$set_aggr_stats_for.getResult(ThriftHiveMetastore.java:20473)^M
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)^M at 
org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)^M   at 
org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:643)^M
   at 
org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor$1.run(HadoopThriftAuthBridge.java:638)^M
   at java.security.AccessController.doPrivileged(Native Method)^M at 
javax.security.auth.Subject.doAs(Subject.java:422)^M at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)^M
   at 
org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:638)^M
 at 
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)^M
   at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)^M
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)^M
at java.lang.Thread.run(Thread.java:748) {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-25416) Hive metastore leak memory cause datanucleus-api-jdo bug

2021-08-02 Thread shezm (Jira)
shezm created HIVE-25416:


 Summary: Hive metastore leak memory cause datanucleus-api-jdo bug
 Key: HIVE-25416
 URL: https://issues.apache.org/jira/browse/HIVE-25416
 Project: Hive
  Issue Type: Bug
 Environment:  

 

 

 

 

 

 
Reporter: shezm
Assignee: shezm
 Attachments: leak.jpg

I encountered a memory leak case. The MAT info :

!leak.jpg!

Full error message is :
{code:java}
Cannot get Long result for param = 8 for column "`FUNCS`.`FUNC_ID`" : Operation 
not allowed after ResultSet closed{code}
This is because there is a bug in the JDOPersistenceManager.retrieveAll code.
{code:java}
// code placeholder
JDOPersistenceManager{
public void retrieveAll(Collection pcs, boolean useFetchPlan) {
this.assertIsOpen();
ArrayList failures = new ArrayList();
Iterator i = pcs.iterator();

while(i.hasNext()) {
try {
this.jdoRetrieve(i.next(), useFetchPlan);
} catch (RuntimeException var6) {
failures.add(var6);
}
}

if (!failures.isEmpty()) {
throw new JDOUserException(Localiser.msg("010038"), 
(Exception[])((Exception[])failures.toArray(new Exception[failures.size()])));
}
}
}
{code}
In some extreme cases   the function of next() does not work . This will result 
in a very large list like as shown above.

 

The bug detail can see this : 
[https://github.com/datanucleus/datanucleus-api-jdo/issues/106]

This problem is fixed in datanucleus-api-jdo version 5.2.6. So we should 
upgrade it .

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Welcome Marta to Hive PMC

2021-08-02 Thread Ashutosh Chauhan
Hi all,

It's an honor to announce that Apache Hive PMC has recently voted to invite
Marta Kuczora as a new Hive PMC member. Marta is a long time Hive
contributor and committer, and has made significant contributions in Hive.
Please join me in congratulating her and looking forward to a bigger role
that she will play in the Apache Hive project.

Thanks,
Ashutosh


Re: [VOTE] Should we release Hive Storage API 2.7.3-rc2?

2021-08-02 Thread Chao Sun
+1 (binding)

- build the release and ran all the tests successfully (the tests was done
in Mac OS so I didn't see the test failure Alan reported)
- verified checksum and signatures

Chao

On Mon, Aug 2, 2021 at 5:29 PM Alan Gates  wrote:

> Centos, java-1.8.0-openjdk-devel
>
> Alan.
>
> On Mon, Aug 2, 2021 at 5:20 PM Owen O'Malley 
> wrote:
>
> > Ok, I was able to reproduce it in ORC's Ubuntu 18 docker image. I'll
> track
> > it down.
> >
> > On Tue, Aug 3, 2021 at 12:06 AM Owen O'Malley 
> > wrote:
> >
> > > Hmm, I just reran the build both from git and the tar ball and
> > > TestBytesColumnVector passed both times. Which jvm were you using?
> > >
> > > On Tue, Aug 3, 2021 at 12:02 AM Pavan Lanka 
> > > wrote:
> > >
> > >> +1 (non-binding)
> > >>
> > >> * Built Hive Storage API (did not get the failure that Alan called
> out)
> > >> using the tag
> > >> * Ran the individual test TestBytesColumnVector successfully
> > >> * Built ORC using 2.7.3 successfully
> > >>
> > >> Regards,
> > >> Pavan
> > >>
> > >> > On Aug 2, 2021, at 4:03 PM, Alan Gates 
> wrote:
> > >> >
> > >> > +1.  Checked the keys and the sha checksum.  Did a build and ran the
> > >> > tests.  Ran rat.  I did get one failing test, but it doesn't look
> > >> crucial:
> > >> >
> > >> > TestBytesColumnVector.testConcatAndPadding:109 row 0 expected:<[?]>
> > but
> > >> > was:<[???]>
> > >> >
> > >> > Alan.
> > >> >
> > >> > On Thu, Jul 29, 2021 at 3:14 PM Owen O'Malley <
> owen.omal...@gmail.com
> > >
> > >> > wrote:
> > >> >
> > >> >> Hello all,
> > >> >>
> > >> >> Following on previous discussions, I would like to propose a new
> > >> >> storage-api release including HIVE-25400.
> > >> >>
> > >> >> Shall we release the following artifacts as Hive Storage API 2.7.3?
> > >> >>
> > >> >> tar: http://home.apache.org/~omalley/hive-storage-2.7.3/
> > >> >> tag:
> > >> https://github.com/apache/hive/releases/tag/storage-release-2.7.3-rc2
> > >> >> jiras:
> > https://issues.apache.org/jira/projects/HIVE/versions/12350287
> > >> >>
> > >> >> .. Owen
> > >> >>
> > >>
> > >>
> >
>


Re: [VOTE] Should we release Hive Storage API 2.8.1 rc2?

2021-08-02 Thread Chao Sun
+1 (binding)

- build the release and ran all the tests successfully
- verified checksum and signatures

Chao

On Mon, Aug 2, 2021 at 4:49 PM Pavan Lanka  wrote:

> +1 (non-binding)
>
> * Built the release
> * Built ORC using 2.8.1
> * Ran the benchmarks compactExpression on the SArg
>
> Regards,
> Pavan
>
> > On Aug 2, 2021, at 4:09 PM, Alan Gates  wrote:
> >
> > +1.  Built it, ran rat, checked the signature and the hash.
> >
> > Alan.
> >
> > On Thu, Jul 29, 2021 at 3:13 PM Owen O'Malley 
> > wrote:
> >
> >> Hello all,
> >>
> >> I would like to propose a new storage-api release including HIVE-25400.
> >>
> >> Shall we release the following artifacts as Hive Storage API 2.8.1?
> >>
> >> tar: http://home.apache.org/~omalley/hive-storage-2.8.1/
> >> tag:
> https://github.com/apache/hive/releases/tag/storage-release-2.8.1-rc2
> >> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350456
> >>
> >> .. Owen
> >>
>
>


[jira] [Created] (HIVE-25415) auto-assign breaks on forks

2021-08-02 Thread Josh Soref (Jira)
Josh Soref created HIVE-25415:
-

 Summary: auto-assign breaks on forks
 Key: HIVE-25415
 URL: https://issues.apache.org/jira/browse/HIVE-25415
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Reporter: Josh Soref


{code:java}
Run shufo/auto-assign-reviewer-by-files@v1.1.1
5{
6  '**/*.thrift': [ 'kgyrtkirk', 'klcopp' ],
7  '**/*.g': [ 'kgyrtkirk' ],
8  '**/package.jdo': [ 'kgyrtkirk' ],
9  '**/schq/**': [ 'kgyrtkirk' ],
10  '**/*Scheduled*': [ 'kgyrtkirk' ],
11  '**/*[sS]ketches*': [ 'kgyrtkirk' ],
12  Jenkinsfile: [ 'kgyrtkirk' ],
13  '.github/**': [ 'kgyrtkirk' ],
14  '**/ddl/**': [ 'miklosgergely' ],
15  '**/ql/*@(Driver|Compiler|Executor)*.java': [ 'miklosgergely' ],
16  '**/schematool/**': [ 'miklosgergely' ],
17  '**/metatool/**': [ 'miklosgergely' ],
18  '**/tez/**/*.java': [ 'abstractdog' ],
19  '**/*Tez*java': [ 'abstractdog' ],
20  '**/*TopNKey*java': [ 'kasakrisz' ],
21  '**/*CardinalityPreserving*java': [ 'kasakrisz' ],
22  '**/*Llap*java': [ 'pgaref' ]
23}
24  beeline/src/test/org/apache/hive/beeline/schematool/TestHiveSchemaTool.java 
matches **/schematool/**
25finished!
26(node:1453) UnhandledPromiseRejectionWarning: HttpError: Reviews may only be 
requested from collaborators. One or more of the users or teams you specified 
is not a collaborator of the check-spelling/hive repository.
27at 
/home/runner/work/_actions/shufo/auto-assign-reviewer-by-files/v1.1.1/dist/index.js:1:301912
28at processTicksAndRejections (internal/process/task_queues.js:93:5)
29at async assignReviewers 
(/home/runner/work/_actions/shufo/auto-assign-reviewer-by-files/v1.1.1/dist/index.js:1:39056)
30(node:1453) UnhandledPromiseRejectionWarning: Unhandled promise rejection. 
This error originated either by throwing inside of an async function without a 
catch block, or by rejecting a promise which was not handled with .catch(). 
(rejection id: 1)
31(node:1453) [DEP0018] DeprecationWarning: Unhandled promise rejections are 
deprecated. In the future, promise rejections that are not handled will 
terminate the Node.js process with a non-zero exit code.
Complete job0s {code}
 
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


Re: [VOTE] Should we release Hive Storage API 2.7.3-rc2?

2021-08-02 Thread Alan Gates
Centos, java-1.8.0-openjdk-devel

Alan.

On Mon, Aug 2, 2021 at 5:20 PM Owen O'Malley  wrote:

> Ok, I was able to reproduce it in ORC's Ubuntu 18 docker image. I'll track
> it down.
>
> On Tue, Aug 3, 2021 at 12:06 AM Owen O'Malley 
> wrote:
>
> > Hmm, I just reran the build both from git and the tar ball and
> > TestBytesColumnVector passed both times. Which jvm were you using?
> >
> > On Tue, Aug 3, 2021 at 12:02 AM Pavan Lanka 
> > wrote:
> >
> >> +1 (non-binding)
> >>
> >> * Built Hive Storage API (did not get the failure that Alan called out)
> >> using the tag
> >> * Ran the individual test TestBytesColumnVector successfully
> >> * Built ORC using 2.7.3 successfully
> >>
> >> Regards,
> >> Pavan
> >>
> >> > On Aug 2, 2021, at 4:03 PM, Alan Gates  wrote:
> >> >
> >> > +1.  Checked the keys and the sha checksum.  Did a build and ran the
> >> > tests.  Ran rat.  I did get one failing test, but it doesn't look
> >> crucial:
> >> >
> >> > TestBytesColumnVector.testConcatAndPadding:109 row 0 expected:<[?]>
> but
> >> > was:<[???]>
> >> >
> >> > Alan.
> >> >
> >> > On Thu, Jul 29, 2021 at 3:14 PM Owen O'Malley  >
> >> > wrote:
> >> >
> >> >> Hello all,
> >> >>
> >> >> Following on previous discussions, I would like to propose a new
> >> >> storage-api release including HIVE-25400.
> >> >>
> >> >> Shall we release the following artifacts as Hive Storage API 2.7.3?
> >> >>
> >> >> tar: http://home.apache.org/~omalley/hive-storage-2.7.3/
> >> >> tag:
> >> https://github.com/apache/hive/releases/tag/storage-release-2.7.3-rc2
> >> >> jiras:
> https://issues.apache.org/jira/projects/HIVE/versions/12350287
> >> >>
> >> >> .. Owen
> >> >>
> >>
> >>
>


Re: [VOTE] Should we release Hive Storage API 2.7.3-rc2?

2021-08-02 Thread Owen O'Malley
Ok, I was able to reproduce it in ORC's Ubuntu 18 docker image. I'll track
it down.

On Tue, Aug 3, 2021 at 12:06 AM Owen O'Malley 
wrote:

> Hmm, I just reran the build both from git and the tar ball and
> TestBytesColumnVector passed both times. Which jvm were you using?
>
> On Tue, Aug 3, 2021 at 12:02 AM Pavan Lanka 
> wrote:
>
>> +1 (non-binding)
>>
>> * Built Hive Storage API (did not get the failure that Alan called out)
>> using the tag
>> * Ran the individual test TestBytesColumnVector successfully
>> * Built ORC using 2.7.3 successfully
>>
>> Regards,
>> Pavan
>>
>> > On Aug 2, 2021, at 4:03 PM, Alan Gates  wrote:
>> >
>> > +1.  Checked the keys and the sha checksum.  Did a build and ran the
>> > tests.  Ran rat.  I did get one failing test, but it doesn't look
>> crucial:
>> >
>> > TestBytesColumnVector.testConcatAndPadding:109 row 0 expected:<[?]> but
>> > was:<[???]>
>> >
>> > Alan.
>> >
>> > On Thu, Jul 29, 2021 at 3:14 PM Owen O'Malley 
>> > wrote:
>> >
>> >> Hello all,
>> >>
>> >> Following on previous discussions, I would like to propose a new
>> >> storage-api release including HIVE-25400.
>> >>
>> >> Shall we release the following artifacts as Hive Storage API 2.7.3?
>> >>
>> >> tar: http://home.apache.org/~omalley/hive-storage-2.7.3/
>> >> tag:
>> https://github.com/apache/hive/releases/tag/storage-release-2.7.3-rc2
>> >> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350287
>> >>
>> >> .. Owen
>> >>
>>
>>


Re: [VOTE] Should we release Hive Storage API 2.7.3-rc2?

2021-08-02 Thread Owen O'Malley
Hmm, I just reran the build both from git and the tar ball and
TestBytesColumnVector passed both times. Which jvm were you using?

On Tue, Aug 3, 2021 at 12:02 AM Pavan Lanka 
wrote:

> +1 (non-binding)
>
> * Built Hive Storage API (did not get the failure that Alan called out)
> using the tag
> * Ran the individual test TestBytesColumnVector successfully
> * Built ORC using 2.7.3 successfully
>
> Regards,
> Pavan
>
> > On Aug 2, 2021, at 4:03 PM, Alan Gates  wrote:
> >
> > +1.  Checked the keys and the sha checksum.  Did a build and ran the
> > tests.  Ran rat.  I did get one failing test, but it doesn't look
> crucial:
> >
> > TestBytesColumnVector.testConcatAndPadding:109 row 0 expected:<[?]> but
> > was:<[???]>
> >
> > Alan.
> >
> > On Thu, Jul 29, 2021 at 3:14 PM Owen O'Malley 
> > wrote:
> >
> >> Hello all,
> >>
> >> Following on previous discussions, I would like to propose a new
> >> storage-api release including HIVE-25400.
> >>
> >> Shall we release the following artifacts as Hive Storage API 2.7.3?
> >>
> >> tar: http://home.apache.org/~omalley/hive-storage-2.7.3/
> >> tag:
> https://github.com/apache/hive/releases/tag/storage-release-2.7.3-rc2
> >> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350287
> >>
> >> .. Owen
> >>
>
>


Re: [VOTE] Should we release Hive Storage API 2.7.3-rc2?

2021-08-02 Thread Pavan Lanka
+1 (non-binding)

* Built Hive Storage API (did not get the failure that Alan called out) using 
the tag
* Ran the individual test TestBytesColumnVector successfully
* Built ORC using 2.7.3 successfully

Regards,
Pavan

> On Aug 2, 2021, at 4:03 PM, Alan Gates  wrote:
> 
> +1.  Checked the keys and the sha checksum.  Did a build and ran the
> tests.  Ran rat.  I did get one failing test, but it doesn't look crucial:
> 
> TestBytesColumnVector.testConcatAndPadding:109 row 0 expected:<[?]> but
> was:<[???]>
> 
> Alan.
> 
> On Thu, Jul 29, 2021 at 3:14 PM Owen O'Malley 
> wrote:
> 
>> Hello all,
>> 
>> Following on previous discussions, I would like to propose a new
>> storage-api release including HIVE-25400.
>> 
>> Shall we release the following artifacts as Hive Storage API 2.7.3?
>> 
>> tar: http://home.apache.org/~omalley/hive-storage-2.7.3/
>> tag: https://github.com/apache/hive/releases/tag/storage-release-2.7.3-rc2
>> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350287
>> 
>> .. Owen
>> 



Re: [VOTE] Should we release Hive Storage API 2.8.1 rc2?

2021-08-02 Thread Pavan Lanka
+1 (non-binding)

* Built the release
* Built ORC using 2.8.1
* Ran the benchmarks compactExpression on the SArg

Regards,
Pavan

> On Aug 2, 2021, at 4:09 PM, Alan Gates  wrote:
> 
> +1.  Built it, ran rat, checked the signature and the hash.
> 
> Alan.
> 
> On Thu, Jul 29, 2021 at 3:13 PM Owen O'Malley 
> wrote:
> 
>> Hello all,
>> 
>> I would like to propose a new storage-api release including HIVE-25400.
>> 
>> Shall we release the following artifacts as Hive Storage API 2.8.1?
>> 
>> tar: http://home.apache.org/~omalley/hive-storage-2.8.1/
>> tag: https://github.com/apache/hive/releases/tag/storage-release-2.8.1-rc2
>> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350456
>> 
>> .. Owen
>> 



Re: [VOTE] Should we release Hive Storage API 2.8.1 rc2?

2021-08-02 Thread Alan Gates
+1.  Built it, ran rat, checked the signature and the hash.

Alan.

On Thu, Jul 29, 2021 at 3:13 PM Owen O'Malley 
wrote:

> Hello all,
>
> I would like to propose a new storage-api release including HIVE-25400.
>
> Shall we release the following artifacts as Hive Storage API 2.8.1?
>
> tar: http://home.apache.org/~omalley/hive-storage-2.8.1/
> tag: https://github.com/apache/hive/releases/tag/storage-release-2.8.1-rc2
> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350456
>
> .. Owen
>


Re: [VOTE] Should we release Hive Storage API 2.7.3-rc2?

2021-08-02 Thread Alan Gates
+1.  Checked the keys and the sha checksum.  Did a build and ran the
tests.  Ran rat.  I did get one failing test, but it doesn't look crucial:

TestBytesColumnVector.testConcatAndPadding:109 row 0 expected:<[?]> but
was:<[???]>

Alan.

On Thu, Jul 29, 2021 at 3:14 PM Owen O'Malley 
wrote:

> Hello all,
>
> Following on previous discussions, I would like to propose a new
> storage-api release including HIVE-25400.
>
> Shall we release the following artifacts as Hive Storage API 2.7.3?
>
> tar: http://home.apache.org/~omalley/hive-storage-2.7.3/
> tag: https://github.com/apache/hive/releases/tag/storage-release-2.7.3-rc2
> jiras: https://issues.apache.org/jira/projects/HIVE/versions/12350287
>
> .. Owen
>


[jira] [Created] (HIVE-25414) Optimise Hive::addWriteNotificationLog: Reduce FS call per notification

2021-08-02 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HIVE-25414:
---

 Summary: Optimise Hive::addWriteNotificationLog: Reduce FS call 
per notification
 Key: HIVE-25414
 URL: https://issues.apache.org/jira/browse/HIVE-25414
 Project: Hive
  Issue Type: Sub-task
Reporter: Rajesh Balamohan


AddWriteNotification is slow due to FS interactions (i.e to get the set of 
insert file information). This can be avoided as FileStatus can be passed 
instead of Path from parent methods.

[https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L3572]
 

[https://github.com/apache/hive/blob/96b39cd5190f0cfadb677e3f3b7ead1d663921b2/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L3620]

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-25413) Optimise ObjectStore::alterPartitions to reduce DB calls for getPartitions

2021-08-02 Thread Rajesh Balamohan (Jira)
Rajesh Balamohan created HIVE-25413:
---

 Summary: Optimise ObjectStore::alterPartitions to reduce DB calls 
for getPartitions
 Key: HIVE-25413
 URL: https://issues.apache.org/jira/browse/HIVE-25413
 Project: Hive
  Issue Type: Improvement
Reporter: Rajesh Balamohan


[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L5121]

For dynamic load partitions (e.g when 1000s of parts being updated), it takes 
lot of time to fetch one partition at a time. It would be good to have  
"List getMPartitions" to reduce number of interactions with DB.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (HIVE-25412) Fix vulnerabilities in Hive Metastore Standalone

2021-08-02 Thread Brisly Priya Joseph (Jira)
Brisly Priya Joseph created HIVE-25412:
--

 Summary: Fix vulnerabilities in Hive Metastore Standalone
 Key: HIVE-25412
 URL: https://issues.apache.org/jira/browse/HIVE-25412
 Project: Hive
  Issue Type: Task
  Components: Standalone Metastore
Affects Versions: 3.0.0
Reporter: Brisly Priya Joseph


The following vulnerabilities are detected in Standalone Hive Metastore:

CVE-2019-12814 - jackson-databind-2.9.4 - (Fix available in 
jackson-databind-2.9.9.1)

CVE-2020-8840 - jackson-databind-2.9.4 - (Fix available in 
jackson-databind-2.9.10.3,2.8.11.5)

CVE-2019-16943 - jackson-databind-2.9.4 - (Fix available in 
jackson-databind-2.9.10.1,2.10.0)

CVE-2018-10237 - guava-19.0 - (Fix available in guava-25.0,24.1.1,23.6.1)

CVE-2015-1832 - derby-10.10.2.0 - (Fix available in derby-10.14.2.0)

please upgrade the packages to the mentioned versions with fix.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)