Re: [VOTE] Apache Hive 4.0.0-alpha-1 Release Candidate 1

[Stamatis Zampetakis]

Hi Peter,

Many thanks for rolling out the RC and for resolving many of the blocker
issues that were remaining.

In general, it is a good practice to include the commit hash (which tags
the release) and the checksum hashes of the release artifacts [1] in the
vote email to minimize the chances of man-in-the-middle attacks and voting
on wrong packages.
Can you please update this thread with those?

Best,
Stamatis

[1] https://people.apache.org/~pvary/apache-hive-4.0.0-alpha-1-rc1/


On Tue, Mar 22, 2022 at 5:00 PM Naveen Gangam 
wrote:

> I have been able to build and run a quick test. I have NOT verified the
> signature. I was trying to run the HMS Checkin tests and got this. I
> suspect these are not specific to the alpha-1 branch. But it is not a test
> failure (although it appears like it should be)
> *"mvn test
>
> -Dtest.groups=org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest"*
>
> [*INFO*] Running org.apache.hadoop.hive.common.metrics.*TestLegacyMetrics*
>
> [main] WARN org.apache.hadoop.hive.common.metrics.LegacyMetrics - Could not
> find counter value for foo.n, returning null instead.
>
> javax.management.AttributeNotFoundException: Key [foo.n] not found/tracked
>
> at
>
> org.apache.hadoop.hive.common.metrics.MetricsMBeanImpl.getAttribute(MetricsMBeanImpl.java:56)
>
>
> [*WARNING*] *Tests **run: 18*, Failures: 0, Errors: 0, *Skipped: 2*, Time
> elapsed: 4.158 s - in
> org.apache.hadoop.hive.metastore.client.*TestCatalogs*
>
> [*INFO*] Running org.apache.hadoop.hive.metastore.*TestMarkPartition*
>
> [*INFO*] *Tests run: 1*, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
> 15.021 s - in org.apache.hadoop.hive.metastore.*TestMarkPartition*
>
> [*INFO*]
>
> [*INFO*] Results:
>
> [*INFO*]
>
> [*WARNING*] *Tests run: 2182, Failures: 0, Errors: 0, Skipped: 5*
>
>
> So over no test failures.
>
> +1 pending other votes (& non-binding)
>
> Thank you
> Naveen
>
> On Tue, Mar 22, 2022 at 9:32 AM Marton Bod 
> wrote:
>
> > +1 (non-binding)
> > Tested the checksums, signatures and built it successfully
> >
> > On Tue, Mar 22, 2022 at 2:26 PM Peter Vary 
> > wrote:
> >
> > > Hi Team,
> > >
> > > Apache Hive 4.0.0-alpha-1 Release Candidate 1 is available here:
> > >
> > > https://people.apache.org/~pvary/apache-hive-4.0.0-alpha-1-rc1/
> > >
> > > Maven artifacts are available here:
> > >
> > > https://repository.apache.org/content/repositories/orgapachehive-/
> > >
> > > The tag 4.0.0-alpha-1-rc1 has been applied to the source for this
> release
> > > in github, you can see it at
> > > https://github.com/apache/hive/tree/release-4.0.0-alpha-1-rc1
> > >
> > > Voting will conclude in 72 hours.
> > >
> > > All interested parties: Please test.
> > > Hive PMC Members: Please test and vote.
> > >
> > > Thanks.
> >
>

[jira] [Created] (HIVE-26060) Invalidate acid table directory cache on drop table

[Antal Sinkovits (Jira)]

Antal Sinkovits created HIVE-26060:
--

 Summary: Invalidate acid table directory cache on drop table
 Key: HIVE-26060
 URL: https://issues.apache.org/jira/browse/HIVE-26060
 Project: Hive
  Issue Type: Bug
Affects Versions: 4.0.0-alpha-1
Reporter: Antal Sinkovits
Assignee: Antal Sinkovits






--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: [VOTE] Apache Hive 4.0.0-alpha-1 Release Candidate 1

[Naveen Gangam]

I have been able to build and run a quick test. I have NOT verified the
signature. I was trying to run the HMS Checkin tests and got this. I
suspect these are not specific to the alpha-1 branch. But it is not a test
failure (although it appears like it should be)
*"mvn test
-Dtest.groups=org.apache.hadoop.hive.metastore.annotation.MetastoreCheckinTest"*

[*INFO*] Running org.apache.hadoop.hive.common.metrics.*TestLegacyMetrics*

[main] WARN org.apache.hadoop.hive.common.metrics.LegacyMetrics - Could not
find counter value for foo.n, returning null instead.

javax.management.AttributeNotFoundException: Key [foo.n] not found/tracked

at
org.apache.hadoop.hive.common.metrics.MetricsMBeanImpl.getAttribute(MetricsMBeanImpl.java:56)


[*WARNING*] *Tests **run: 18*, Failures: 0, Errors: 0, *Skipped: 2*, Time
elapsed: 4.158 s - in org.apache.hadoop.hive.metastore.client.*TestCatalogs*

[*INFO*] Running org.apache.hadoop.hive.metastore.*TestMarkPartition*

[*INFO*] *Tests run: 1*, Failures: 0, Errors: 0, Skipped: 0, Time elapsed:
15.021 s - in org.apache.hadoop.hive.metastore.*TestMarkPartition*

[*INFO*]

[*INFO*] Results:

[*INFO*]

[*WARNING*] *Tests run: 2182, Failures: 0, Errors: 0, Skipped: 5*


So over no test failures.

+1 pending other votes (& non-binding)

Thank you
Naveen

On Tue, Mar 22, 2022 at 9:32 AM Marton Bod 
wrote:

> +1 (non-binding)
> Tested the checksums, signatures and built it successfully
>
> On Tue, Mar 22, 2022 at 2:26 PM Peter Vary 
> wrote:
>
> > Hi Team,
> >
> > Apache Hive 4.0.0-alpha-1 Release Candidate 1 is available here:
> >
> > https://people.apache.org/~pvary/apache-hive-4.0.0-alpha-1-rc1/
> >
> > Maven artifacts are available here:
> >
> > https://repository.apache.org/content/repositories/orgapachehive-/
> >
> > The tag 4.0.0-alpha-1-rc1 has been applied to the source for this release
> > in github, you can see it at
> > https://github.com/apache/hive/tree/release-4.0.0-alpha-1-rc1
> >
> > Voting will conclude in 72 hours.
> >
> > All interested parties: Please test.
> > Hive PMC Members: Please test and vote.
> >
> > Thanks.
>

Re: [VOTE] Apache Hive 4.0.0-alpha-1 Release Candidate 1

[Marton Bod]

+1 (non-binding)
Tested the checksums, signatures and built it successfully

On Tue, Mar 22, 2022 at 2:26 PM Peter Vary 
wrote:

> Hi Team,
>
> Apache Hive 4.0.0-alpha-1 Release Candidate 1 is available here:
>
> https://people.apache.org/~pvary/apache-hive-4.0.0-alpha-1-rc1/
>
> Maven artifacts are available here:
>
> https://repository.apache.org/content/repositories/orgapachehive-/
>
> The tag 4.0.0-alpha-1-rc1 has been applied to the source for this release
> in github, you can see it at
> https://github.com/apache/hive/tree/release-4.0.0-alpha-1-rc1
>
> Voting will conclude in 72 hours.
>
> All interested parties: Please test.
> Hive PMC Members: Please test and vote.
>
> Thanks.

[VOTE] Apache Hive 4.0.0-alpha-1 Release Candidate 1

[Peter Vary]

Hi Team,

Apache Hive 4.0.0-alpha-1 Release Candidate 1 is available here:

https://people.apache.org/~pvary/apache-hive-4.0.0-alpha-1-rc1/

Maven artifacts are available here:

https://repository.apache.org/content/repositories/orgapachehive-/

The tag 4.0.0-alpha-1-rc1 has been applied to the source for this release in 
github, you can see it at
https://github.com/apache/hive/tree/release-4.0.0-alpha-1-rc1

Voting will conclude in 72 hours.

All interested parties: Please test.
Hive PMC Members: Please test and vote.

Thanks.

Re: Is the web broken?

[Peter Vary]

Hi Stamatis,

Tried to dig up the details, and this is what I have found and should work 
currently:

Committer builds docs locally from a checkout of hive-site.git gh-pages branch
commits the diff to gh-pages branch
reviews the preview at https://apache.github.io/hive-site/ 
 
if all looks good then PR/merge to main, nothing else to do the changes will go 
live.

Disclaimer: I have not tried this out, just copied from the private list.

Thanks for checking this!
Peter

> On 2022. Mar 22., at 13:14, Stamatis Zampetakis  wrote:
> 
> Hi all,
> 
> There have been various problems around the website lately and from the
> exchanges in [1,2] it is still not clear to me what's the process to update
> the website.
> 
> We have more or less the same code and Jekyll builders in two repositories
> [3,4] which doesn't seem normal.
> 
> Moreover, the hive-site repo currently has three branches:
> * main
> * asf-site
> * gh-pages
> and it is not completely clear to me the role of each.
> 
> From my perspective all the source code (.md files and Jekyll generators)
> should be in the main Hive repo [3] and the hive-site [4] should only be
> used for the deployed site (html pages). In the hive-site repo, we
> shouldn't need to maintain and use more than one branch.
> 
> We should convert to some workflow that we want to follow otherwise
> updating the website will be a big pain in the near future.
> 
> Best,
> Stamatis
> 
> [1] https://issues.apache.org/jira/browse/INFRA-20776
> [2] https://issues.apache.org/jira/browse/INFRA-23007
> [3] https://github.com/apache/hive/tree/master/docs
> [4] https://github.com/apache/hive-site
> 
> On Tue, Mar 15, 2022 at 9:11 PM Stamatis Zampetakis 
> wrote:
> 
>> Hi,
>> 
>> From the discussion in INFRA-20776, it seems there has been some kind of
>> communication in the private hive list. If it is possible to share this
>> information please let us know.
>> 
>> Apart from that please stop including the dev@calcite list in this thread.
>> I accidentally started that by cc'ing the wrong list in my previous email;
>> I meant to include dev@hive.
>> 
>> Best,
>> Stamatis
>> 
>> On Tue, Mar 15, 2022 at 8:54 PM Chao Sun  wrote:
>> 
>>> Hi Gavin,
>>> 
>>> Do you have any idea why hive.apache.org is broken after the hive-site
>>> change?
>>> 
>>> Thanks,
>>> Chao
>>> 
>>> On Thu, Mar 10, 2022 at 7:16 PM Chao Sun  wrote:
 
 Sorry, we are in the process of migrating to hive-site.git from CMS.
>>> There are a few issues to be fixed and I’m talking with ASF infra on this.
 
 On Thu, Mar 10, 2022 at 6:05 PM 王道远(健身) 
>>> wrote:
> 
> Hi,
> 
> Could we keep a redirect page at the old place, or a CNAME resolution?
> 
> Best,
> Adrian
> 
> --
> 发件人：Stamatis Zampetakis 
> 发送时间：2022年3月10日(星期四) 18:33
> 收件人：user ; dev 
> 主 题：Re: Is the web broken?
> 
> 
> Hi,
> 
> I am not sure if I am missing something but I get the impression [1]
>>> that the site from now on will be served from here:
> https://apache.github.io/hive-site/
> 
> Best,
> Stamatis
> 
> [1] https://issues.apache.org/jira/browse/INFRA-20776
> 
> On Thu, Mar 10, 2022 at 10:21 AM Ming  wrote:
> I have the same situation
> 
> Ming
> shezhimin...@gmail.com
> 签名由 网易邮箱大师 定制
> 
> On 03/10/2022 17:12，Pau Tallada wrote：
> Is it only me or the https://hive.apache.org/ web is showing a
>>> directory listing?!
> 
> --
> --
> Pau Tallada Crespí
> Departament de Serveis
> Port d'Informació Científica (PIC)
> Tel: +34 93 170 2729
> --
> 
>>> 
>> 

[jira] [Created] (HIVE-26059) Eventually clean compactions in "refused" state from compaction history

[Jira]

László Végh created HIVE-26059:
--

 Summary: Eventually clean compactions in "refused" state from 
compaction history
 Key: HIVE-26059
 URL: https://issues.apache.org/jira/browse/HIVE-26059
 Project: Hive
  Issue Type: Improvement
  Components: Hive
Reporter: László Végh


Compactions in states succeeded, failed, and did not initiate have a retention 
threshold (example: metastore.compactor.history.retention.succeeded) and are 
purged from COMPLETED_COMPACTIONS if the number of compactions in this state 
per partition/unpartitioned table passes the threshold. This keeps the size of 
COMPLETED_COMPACTIONS in check.

We should also purge refused compactions from COMPLETED_COMPACTIONS.

See:
CompactionTxnHandler#purgeCompactionHistory

! Also: REFUSED_RESPONSE should be added to 
org.apache.hadoop.hive.metastore.txn.TxnStore#COMPACTION_STATES so that metrics 
will be collected about it.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Is the web broken?

[Stamatis Zampetakis]

Hi all,

There have been various problems around the website lately and from the
exchanges in [1,2] it is still not clear to me what's the process to update
the website.

We have more or less the same code and Jekyll builders in two repositories
[3,4] which doesn't seem normal.

Moreover, the hive-site repo currently has three branches:
* main
* asf-site
* gh-pages
and it is not completely clear to me the role of each.

>From my perspective all the source code (.md files and Jekyll generators)
should be in the main Hive repo [3] and the hive-site [4] should only be
used for the deployed site (html pages). In the hive-site repo, we
shouldn't need to maintain and use more than one branch.

We should convert to some workflow that we want to follow otherwise
updating the website will be a big pain in the near future.

Best,
Stamatis

[1] https://issues.apache.org/jira/browse/INFRA-20776
[2] https://issues.apache.org/jira/browse/INFRA-23007
[3] https://github.com/apache/hive/tree/master/docs
[4] https://github.com/apache/hive-site

On Tue, Mar 15, 2022 at 9:11 PM Stamatis Zampetakis 
wrote:

> Hi,
>
> From the discussion in INFRA-20776, it seems there has been some kind of
> communication in the private hive list. If it is possible to share this
> information please let us know.
>
> Apart from that please stop including the dev@calcite list in this thread.
> I accidentally started that by cc'ing the wrong list in my previous email;
> I meant to include dev@hive.
>
> Best,
> Stamatis
>
> On Tue, Mar 15, 2022 at 8:54 PM Chao Sun  wrote:
>
>> Hi Gavin,
>>
>> Do you have any idea why hive.apache.org is broken after the hive-site
>> change?
>>
>> Thanks,
>> Chao
>>
>> On Thu, Mar 10, 2022 at 7:16 PM Chao Sun  wrote:
>> >
>> > Sorry, we are in the process of migrating to hive-site.git from CMS.
>> There are a few issues to be fixed and I’m talking with ASF infra on this.
>> >
>> > On Thu, Mar 10, 2022 at 6:05 PM 王道远(健身) 
>> wrote:
>> >>
>> >> Hi,
>> >>
>> >> Could we keep a redirect page at the old place, or a CNAME resolution?
>> >>
>> >> Best,
>> >> Adrian
>> >>
>> >> --
>> >> 发件人：Stamatis Zampetakis 
>> >> 发送时间：2022年3月10日(星期四) 18:33
>> >> 收件人：user ; dev 
>> >> 主 题：Re: Is the web broken?
>> >>
>> >>
>> >> Hi,
>> >>
>> >> I am not sure if I am missing something but I get the impression [1]
>> that the site from now on will be served from here:
>> >> https://apache.github.io/hive-site/
>> >>
>> >> Best,
>> >> Stamatis
>> >>
>> >> [1] https://issues.apache.org/jira/browse/INFRA-20776
>> >>
>> >> On Thu, Mar 10, 2022 at 10:21 AM Ming  wrote:
>> >> I have the same situation
>> >>
>> >> Ming
>> >> shezhimin...@gmail.com
>> >> 签名由 网易邮箱大师 定制
>> >>
>> >> On 03/10/2022 17:12，Pau Tallada wrote：
>> >> Is it only me or the https://hive.apache.org/ web is showing a
>> directory listing?!
>> >>
>> >> --
>> >> --
>> >> Pau Tallada Crespí
>> >> Departament de Serveis
>> >> Port d'Informació Científica (PIC)
>> >> Tel: +34 93 170 2729
>> >> --
>> >>
>>
>

[jira] [Created] (HIVE-26058) Choose meaningful names for the Metastore pool threads

[Zhihua Deng (Jira)]

Zhihua Deng created HIVE-26058:
--

 Summary: Choose meaningful names for the Metastore pool threads
 Key: HIVE-26058
 URL: https://issues.apache.org/jira/browse/HIVE-26058
 Project: Hive
  Issue Type: Improvement
Reporter: Zhihua Deng


Due to TThreadPoolServer#createDefaultExecutorService setting the thread name 
by 

 
{code:java}
thread.setName("TThreadPoolServer WorkerProcess-%d");  {code}
 

The logger output the thread name like:
{noformat}
[TThreadPoolServer WorkerProcess-%d] utils.FileUtils: Renaming 
pfile:/{noformat}
, which makes it hard to identify and debug a thread.

 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)