Re: Planning Hive 4.0.1

[Ayush Saxena]

Hi Zhihua,
Thanks for driving the release. I’ve added labels to a few more
tickets that were already resolved but seem like good candidates for
inclusion in the 4.0.1 release.

-Ayush

On Wed, 18 Sept 2024 at 15:51, Zhihua Deng  wrote:
>
> Hi Team,
>
> As most of the issues in 
> https://issues.apache.org/jira/browse/HIVE-28204?jql=labels%20%3D%20hive-4.0.1-must
>  have been resolved, I will prepare the 4.0.1 RC for vote soon. In case 
> somebody has the ticket which want to be in and it doesn't have the label, 
> please let us know or label it "hive-4.0.1-must" on the Jira.
>
> Thanks in advance!
>
> Best,
> Zhihua

Fwd: IMPORTANT: Hive date parsing issue, input needed

[Ayush Saxena]

Since this is landing in Spam for some reason for most people, I am
explicitly forwarding it rather than replying, so as people can get
it.

I will vote for (2) considering it was returning `null` earlier as
well & the new behaviour came up recently, it would be a behaviour
change for folks migrating from earlier versions of hive to 4.x & the
new behaviour ain't that fancy either...

-Ayush

-- Forwarded message -
From: Zoltán Rátkai 
Date: Wed, 18 Sept 2024 at 12:55
Subject: Re: IMPORTANT: Hive date parsing issue, input needed
To: 
Cc: dev 


Hi Ayush,

thanks for your thoughts!

My answers:

- On the official Hive page about types this is not documented at all:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=82706456#LanguageManualTypes-date

"Dates:  DATE values describe a particular year/month/day, in the form
-­MM-­DD. For example, DATE '2013-­01-­01'. Date types do not have
a time of day component. The range of values supported for the Date
type is -­01-­01 to -­12-­31, dependent on support by the
primitive Java Date type."

- Since the beginning of Hive it worked like returning null, this was
introduced one year ago as a side effect of this: HIVE-27586 "Parse
dates from strings ignoring trailing (potentialy) invalid chars", so
the goal of this ticket was to be able to handle string to date
parsing like : "2024-09-18 GARBAGE" and parse it to 2024-09-18 as
date.
So it is not a long standing behaviour.

I made a comparison of other DB (Postre, MySQL, ORACLE) and none of
them handle it like Hive now. Please check it here:
https://issues.apache.org/jira/browse/HIVE-28483
I agree with you to align with those. This is why I asked this question.

Thanks and regards,

Zoltan Ratkai


On Wed, Sep 18, 2024 at 8:58 AM Ayush Saxena  wrote:
>
> ++ Adding the dev mailing list, as this might be relevant to them as well.
>
> I have a few additional questions:
>
> * Has this behavior been present since the inception of Hive, or did
> it start occurring more recently? If this has been the behavior for
> over a decade, it might be best to leave it unchanged. However, if
> this is a recent development, we should aim to restore the original
> behavior. In my opinion, any change that alters long-standing behavior
> could be considered incompatible unless it was introduced to prevent
> data loss or address a security issue.
> * How do other engines (e.g., Impala, Spark) handle similar scenarios,
> and how do other databases like MySQL and PostgreSQL behave in this
> regard? In the past, we've typically aimed to align with these
> systems, not necessarily exactly, but closely enough—especially in
> cases where there's ambiguity or conflict.
>
>
> -Ayush
>
>
> On Wed, 18 Sept 2024 at 12:00, Zoltán Rátkai  wrote:
> >
> > Dear Hive User!
> >
> > In Hive the standard date format is -MM-DD, like “2024-09-17”.
> > So when casting a string which format is different, like DD-MM- e.g. 
> > “17-09-2024" to date, as a Hive user what would you expect as a result?
> >
> >
> > 1.  “0017-09-20”  => Year of 17 September 20
> > or
> > 2. null
> >
> > It is possible to give a date pattern CAST( AS DATE FORMAT 
> > )and cast with it like "DD-MM-".
> > Earlier it gave back null, but currently Hive gives back the first, which I 
> > think unexpected as a user and customers complain about that. There is a 
> > debate if it is a bug or feature, so I need your help!
> >
> > I would like to ask you to reply with a vote on 1 or 2 what you want as a 
> > user to happen!
> >
> > I really appreciate your input!
> >
> > Thank you,
> >
> > Zoltan Ratkai

Re: IMPORTANT: Hive date parsing issue, input needed

[Ayush Saxena]

++ Adding the dev mailing list, as this might be relevant to them as well.

I have a few additional questions:

* Has this behavior been present since the inception of Hive, or did
it start occurring more recently? If this has been the behavior for
over a decade, it might be best to leave it unchanged. However, if
this is a recent development, we should aim to restore the original
behavior. In my opinion, any change that alters long-standing behavior
could be considered incompatible unless it was introduced to prevent
data loss or address a security issue.
* How do other engines (e.g., Impala, Spark) handle similar scenarios,
and how do other databases like MySQL and PostgreSQL behave in this
regard? In the past, we've typically aimed to align with these
systems, not necessarily exactly, but closely enough—especially in
cases where there's ambiguity or conflict.


-Ayush


On Wed, 18 Sept 2024 at 12:00, Zoltán Rátkai  wrote:
>
> Dear Hive User!
>
> In Hive the standard date format is -MM-DD, like “2024-09-17”.
> So when casting a string which format is different, like DD-MM- e.g. 
> “17-09-2024" to date, as a Hive user what would you expect as a result?
>
>
> 1.  “0017-09-20”  => Year of 17 September 20
> or
> 2. null
>
> It is possible to give a date pattern CAST( AS DATE FORMAT 
> )and cast with it like "DD-MM-".
> Earlier it gave back null, but currently Hive gives back the first, which I 
> think unexpected as a user and customers complain about that. There is a 
> debate if it is a bug or feature, so I need your help!
>
> I would like to ask you to reply with a vote on 1 or 2 what you want as a 
> user to happen!
>
> I really appreciate your input!
>
> Thank you,
>
> Zoltan Ratkai

Re: Hive 4 integration to store table on S3 and ADLS gen2

[Ayush Saxena]

Hi Somesh,

But while trying so we are seeing following exception :
> hadoop fs -ls s3a://somesh.qa.bucket/ -:



This has nothing to do with Hive as such, You have configured Hadoop S3
client wrong, you are missing configs, your hadoop ls command itself is
failing, there is no Hive involved here. You need to setup the FileSystem
correctly...

This is a hadoop problem, maybe you can explore reading this doc in hadoop
[1] & that might help, if you still face issues, you should bug the Hadoop
mailing lists not hive

-Ayush

[1]
https://hadoop.apache.org/docs/stable/hadoop-aws/tools/hadoop-aws/index.html

On Wed, 18 Sept 2024 at 11:12, Awasthi, Somesh
 wrote:

> Hi Team,
>
>
>
> I want to setup hive4 standalone to store table on S3 and Adls gen2 as a
> storage .
>
>
>
> Could you please help me as with proper steps and configurations required
> for this.
>
>
>
> Because we are facing multiple issue on this please help me here ASPA.
>
>
>
> *What we tried.*
>
>
>
> I am trying to configure AWS S3 configuration with the Hadoop and Hive
> setup.
>
> But while trying so we are seeing following exception :
>
> hadoop fs -ls s3a://somesh.qa.bucket/ -:
>
> Fatal internal error java.lang.RuntimeException:
> java.lang.ClassNotFoundException: Class
> org.apache.hadoop.fs.s3a.S3AFileSystem not found
>
> *To resolve this I have added hadoop-aws-3.3.6.jar and
> aws-java-sdk-bundle-1.12.770.jar in Hadoop classpath.*
>
> *i.e is under : /usr/local/hadoop/share/hadoop/common/lib*
>
> *And S3 related configurations in the core-site.xml file: under
> /usr/local/hadoop/etc/hadoop directory.*
>
> fs.default.name s3a://somesh.qa.bucket fs.s3a.impl
> org.apache.hadoop.fs.s3a.S3AFileSystem fs.s3a.endpoint
> s3.us-west-2.amazonaws.com fs.s3a.access.key {Access _Key_Value}
> fs.s3a.secret.key {Secret_Key_Value} fs.s3a.path.style.access false
>
> Now when we try hadoop fs -ls s3a://somesh.qa.bucket/
>
> We are observing following exception :
>
> 2024-08-22 13:50:11,294 INFO impl.MetricsConfig: Loaded properties from
> hadoop-metrics2.properties
> 2024-08-22 13:50:11,376 INFO impl.MetricsSystemImpl: Scheduled Metric
> snapshot period at 10 second(s).
> 2024-08-22 13:50:11,376 INFO impl.MetricsSystemImpl: s3a-file-system
> metrics system started
> 2024-08-22 13:50:11,434 WARN util.VersionInfoUtils: The AWS SDK for Java
> 1.x entered maintenance mode starting July 31, 2024 and will reach end of
> support on December 31, 2025. For more information, see
> https://aws.amazon.com/blogs/developer/the-aws-sdk-for-java-1-x-is-in-maintenance-mode-effective-july-31-2024/
> You can print where on the file system the AWS SDK for Java 1.x core
> runtime is located by setting the AWS_JAVA_V1_PRINT_LOCATION environment
> variable or aws.java.v1.printLocation system property to 'true'.
> This message can be disabled by setting the
> AWS_JAVA_V1_DISABLE_DEPRECATION_ANNOUNCEMENT environment variable or
> aws.java.v1.disableDeprecationAnnouncement system property to 'true'.
> The AWS SDK for Java 1.x is being used here:
> at java.lang.Thread.getStackTrace(Thread.java:1564)
> at
> com.amazonaws.util.VersionInfoUtils.printDeprecationAnnouncement(VersionInfoUtils.java:81)
> at com.amazonaws.util.VersionInfoUtils.(VersionInfoUtils.java:59)
> at com.amazonaws.internal.EC2ResourceFetcher.(EC2ResourceFetcher.java:44)
> at
> com.amazonaws.auth.InstanceMetadataServiceCredentialsFetcher.(InstanceMetadataServiceCredentialsFetcher.java:38)
> at
> com.amazonaws.auth.InstanceProfileCredentialsProvider.(InstanceProfileCredentialsProvider.java:111)
> at
> com.amazonaws.auth.InstanceProfileCredentialsProvider.(InstanceProfileCredentialsProvider.java:91)
> at
> com.amazonaws.auth.InstanceProfileCredentialsProvider.(InstanceProfileCredentialsProvider.java:75)
> at
> com.amazonaws.auth.InstanceProfileCredentialsProvider.(InstanceProfileCredentialsProvider.java:58)
> at
> com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper.initializeProvider(EC2ContainerCredentialsProviderWrapper.java:66)
> at
> com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper.(EC2ContainerCredentialsProviderWrapper.java:55)
> at
> org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider.(IAMInstanceCredentialsProvider.java:53)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.fs.s3a.S3AUtils.createAWSCredentialProvider(S3AUtils.java:727)
> at
> org.apache.hadoop.fs.s3a.S3AUtils.buildAWSProviderList(S3AUtils.java:659)
> at
> org.apache.hadoop.fs.s3a.S3AUtils.createAWSCredentialProviderSet(S3AUtils.java:585)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.bindAWSClient(S3AFileSystem.java:959)
> at
> org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:586)

Re: Review more to become a Hive committer

[Ayush Saxena]

Thanx Stamatis for starting the thread.
I just want to add one more line here that it isn't a number game,
Most of the folks on the PMC go through the reviews as well, what they
were actually, The main stuff is whether the folks on the PMC trust
someone to review the code once they are committer, do they show
adequate understanding of the code from it and are encouraged enough
to review the code properly along with contributing to it.

Just mentioning, So in case anyone thinks opening all the PRs and just
shooting a +1/LGTM on every PR, without actually reviewing will be a
hack, then that ain't gonna work & will just backfire :-)

-Ayush

On Tue, 3 Sept 2024 at 17:21, Stamatis Zampetakis  wrote:
>
> Hi all,
>
> Recently, I have been discussing with some people from the Hive PMC
> about the requirements to invite someone to become a Hive committer.
> For those who are not aware there is a page in the wiki which outlines
> the process [1].
>
> Going over the list of contributors, we have noticed that there are
> various very promising candidates with solid code contributions and
> technical skills. One area that many people fall short and thus don't
> receive an invitation is reviews.
>
> PRs in Hive require at least one +1 from a committer before getting
> merged thus Hive committers should be active in doing reviews. People
> who are interested and motivated to become committers should not
> neglect the reviewing process.
>
> Best,
> Stamatis
>
> [1] https://cwiki.apache.org/confluence/display/Hive/BecomingACommitter

Re: Security health of the Hive project

[Ayush Saxena]

Hi Arnout,
Just letting you know we have heard you, this mail didn't go unnoticed
and we are working on getting things in control. I am sharing the
context of the current situation as well:

AFAIK the issues which were acknowledged by the Hive Security team as
actual bugs in the code are all fixed in the current code & most of
them apart from 1(If I am not mistaken), all were released as part of
the Hive-4.0 release as well, the only reason they are hanging on us
is because we need to release it on all active release lines to
formally announce them. We did work in that area & announced Hive-1.x
& 2.x as EOL. The only other release line remaining is Hive-3.x, Which
I have been talking with folks & we will call it off as well, since we
don't have enough volunteers for that release line and no recognizable
active development is going on there either.

So, as we announce 3.x as EOL mostly post having 4.0.1 release, I
think that is "Majority vote by the PMC", so I am pretty sure we will
have the numbers to do so. I think immediately after that we would be
announcing most of the CVEs. I don't know if there is any way to
announce without having it in 3.x (or mark a release line active but
not secure or something like that). I think there would be some noise
against it, since 3.x does have a good user base, but it doesn't stop
them from using it,, just conveys it ain't safe & the best line and
the active set of developers aren't focusing on that. I will take that
bullet if there is one :-)

Regarding responding to issues reported: I skimmed over the security@,
As of now I couldn't find any unanswered one. I think there was one,
which got answered. Well most of us try to acknowledge ASAP but there
are challenges in that area as well, There is so much of SPAM on
security@ list, that sometimes some genuine issues get missed in that
flood of irrelevant emails, Folks in the community have taken an
initiative in that area as well, & discussing on having a secured JIRA
+ moderating the security@ list & we are exploring that area as well
to improve this aspect. So, that should be improved as well, I can't
say it would be the best but better for sure.

Hope that clarifies the state of the project in terms of security &
why the issues aren't getting announced and the work done by the PMC
in that area. Open to suggestions if any. Thanks for highlighting
this.

++ HIVE private@ in case anyone has any other feedback or shares some
different opinions

-Ayush

On Thu, 15 Aug 2024 at 14:51, Apache Security Team  wrote:
>
> Hello Hive community,
>
> The Hive project is struggling to perform its security duties[0]: there are 
> unusually old security reports that the Hive Security Team / PMC has not 
> disclosed yet, and triaging new incoming security reports also takes longer 
> than responsible. This not only falls short of what is expected as an Apache 
> project, but longer-term could have legal consequences for the ASF and 
> individual contributors, with legislation such as the CRA coming into force 
> in Europe and similar measures being expected around the world.
>
> The ASF Security Team has expressed its concern before, leading to the first 
> formal escalation step [1] of issuing a call for help on your public 
> mailinglist [2] back in March, after an earlier call by your PMC in September 
> [3]. This yielded one volunteer. As far as I can tell the PMC has not 
> enlisted this volunteer yet.
>
> If the Hive project cannot return to a healthy cadence of dealing with 
> security issues, the only responsible decision for the PMC (which is 
> collectively responsible for the oversight of the project) would be to 
> initiate the move to the Attic. Of course we hope this can be prevented.
>
>
> Kind regards,
>
> Arnout Engelen
> ASF Security Team
>
> [0] https://apache.org/security/committers.html
> [1] 
> https://cwiki.apache.org/confluence/display/SECURITY/Project+Security+Response+Formal+Escalation
> [2] https://lists.apache.org/thread/8wghsxdlj8bfygf2ptcdb8pojlvxwjx8
> [3] https://lists.apache.org/thread/j0ztt61wjz9gc46dj6fpor30xh437h9n

Re: Requesting Reviews for HIVE-28118

[Ayush Saxena]

Hi Eshwitha,
That PR has a "continuously" failing test in the CI:
TestEncryptedHDFSCliDriver and you have modified the code around HDFS
& Encryption only. I saw you had a comment on the ticket that it
passes for you locally, but thats not how we can push it forward, you
need to get a green build.

Just had a quick pass, with my limited HDFS knowledge, I think that
code is conceptually wrong as well though it is fixing one of your use
cases, this seems to be treating if the path doesn't exist, it ain't
encrypted...

Unfortunately, I don't have enough bandwidth to fix the logic or help
debug the test failure anytime soon. AFAIK we don't officially support
ViewFs either in Hive now, there are issues with that atleast in
Hive-4.x, when you will be operating cross mounts and maybe some more
if someone tests that thoroughly!!!

-Ayush

On Mon, 12 Aug 2024 at 01:10, Eshwitha Sai  wrote:
>
> Hi all,
>
> This is a review request for the patch Github Pull Request #5137 related to 
> HIVE-28118.
> The patch has an LGTM (from deniskuzZ) and is waiting for further review 
> (from ayushtkn)
>
> Kindly review the same.
>
> Thanks & Regards,
> Eshwitha Sai.

Re: Access to confluence and updating configuration properties for Hive

[Ayush Saxena]

Hi Yan Liu,
Thanx for volunteering, It would require a cwiki user account, I am
creating an INFRA ticket for you,
https://issues.apache.org/jira/browse/INFRA-26016

Can you drop your preferred userid & email id over there, once your account
is created let me know & I will help with setting up the permissions for
you to update the page

-Ayush

On Mon, 5 Aug 2024 at 11:35, yan liu  wrote:

> to whom it may concern
>
>  I am Yan Liu and currently work at Cloudera,  I have noticed that many
> configs that exists in the HiveConf.java
> https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
>   did
> not show up on the hive configuration page
> https://cwiki.apache.org/confluence/display/Hive/Configuration+Properties
>
> For example: hive.repl.cm.enabled is not documented on the confluent page
>
> I volunteer to constantly updating the hive configuration wiki page to
> make sure properties are in sync with the actual HiveConf.java
>
> 
> hive/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java at master ·
> apache/hive
> 
> Apache Hive. Contribute to apache/hive development by creating an account
> on GitHub.
> github.com
>
> Please let me know how to proceed.
>
> Thank you very much
>
> Best Regards
>
> Yan Liu
>

Re: [DISCUSS] Put security list under moderation

[Ayush Saxena]

+1

-Ayush

On Wed, 17 Jul 2024 at 18:53, Stamatis Zampetakis  wrote:
>
> Hi all,
>
> Currently the Hive security list is not moderated allowing anyone to
> send messages to it. As a result it has a huge amount of spam messages
> that reach every subscriber and pollute the archive.
>
> Security lists are by default non-moderated with the argument of not
> accidentally missing some security report. I feel that people
> monitoring the security list are more likely to miss a security report
> due to spam rather than moderation.
>
> I propose to raise an INFRA ticket and put the security list under
> moderation. This would increase the burden for existing moderators
> (myself, and Ayush) but reduce the noise for the rest of the
> subscribers and lead to a more readable/searchable archive.
>
> Let me know your thoughts.
>
> Best,
> Stamatis

Re: Hive 4.0 interview/podcast

[Ayush Saxena]

Hey Guys,
I just watched the interview - It's awesome. Big thanx to Rich &
Stamatis for putting this together!!!

-Ayush

On Tue, 11 Jun 2024 at 18:38, Rich Bowen  wrote:
>
> Hi, folks. Congratulations on the release of Hive 4.0. I've just published an 
> interview with Stamatis Zampetakis, and it's live at 
> https://youtu.be/7HX2MieyzW4 (video) and at https://wp.me/p8gHED-41k (just 
> the audio).
>
> Thanks, Stamatis! It was good talking with you.
>
> --Rich

Re: Inquiry About Release Date for Hive-exec-core: 4.0.1

[Ayush Saxena]

Should be there in 4.0.1 release, expected in about 2 months-AyushOn 29 May 2024, at 4:29 PM, Mergu Ravi  wrote:Sorry, I mentioned the wrong version; the version is 4.1.0.On Wed, May 29, 2024 at 4:27 PM Mergu Ravi  wrote:Hi Hive Team,I hope this message finds you well.I am writing to inquire about the release date for Hive-exec-core 4.0.1. Specifically, I am interested in the timeline for the release of the fix detailed in the Jira story HIVE-28211.Could you please provide an update on when we can expect this release to be available?Thank you for your assistance.--Thanks & Regards,Ravi MerguSMTS-2o:  +91 40 64535355m: +91 9959618687w: www.gaiansolutions.com Please consider the environment before printing this email. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail
-- Thanks & Regards,Ravi MerguSMTS-2o:  +91 40 64535355m: +91 9959618687w: www.gaiansolutions.com Please consider the environment before printing this email. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail

[ANNOUNCE] Hive 2.x EOL

[Ayush Saxena]

Hi All,
The Apache Hive Community has voted to declare the 2.x release line as End of 
Life. This means no further updates or releases will be made for this release 
line.

We urge all Hive 2.x users to upgrade to the latest versions promptly to 
benefit from new features and ongoing support.

-Ayush Saxena
(On Behalf of Apache Hive PMC)

[VOTE][RESULT] Mark Hive 2.x EOL

[Ayush Saxena]

Thanx Everyone for the participation, We have 15 +1s (5 Binding) in
favour with no -1s casted:

* Naveen Gangam(Binding)
* Zoltan Ratkai
* Sourabh Badhya
* Krisztian Kasa(Binding)
* Okumin
* Aman Sinha
* Mahesh Raju Somaraju
* Chao Sun(Binding)
* Kokila Narayanan
* Simhadri G
* Stamatis Zampetakis(Binding)
* Denys Kuzmenko
* Attila Turoczy
* Butao Zhang
* Ayush Saxena(Binding)

The vote passes, I will make the announcements and take care of other
requirements.

-Ayush

On Wed, 15 May 2024 at 20:31, Naveen Gangam
 wrote:
>
> +1 Totally
>
> On Tue, May 14, 2024 at 3:30 AM Zoltán Rátkai  
> wrote:
>>
>> +1 (non-binding)
>>
>> Regards,
>>
>> Zoltan Ratkai
>>
>> On Tue, May 14, 2024 at 5:42 AM Sourabh Badhya 
>>  wrote:
>>>
>>> +1 (non-binding)
>>>
>>> Regards,
>>> Sourabh Badhya
>>>
>>> On Mon, May 13, 2024 at 10:31 PM Krisztian Kasa 
>>>  wrote:
>>>>
>>>> +1 (binding)
>>>>
>>>> On Mon, May 13, 2024 at 4:55 PM Okumin  wrote:
>>>>>
>>>>> +1 (non-binding)
>>>>>
>>>>> I appreciate the community's efforts in maintaining 2.x for so long.
>>>>>
>>>>> Thanks,
>>>>> Okumin
>>>>>
>>>>> On Sat, May 11, 2024 at 1:57 AM Abhishek Gupta  
>>>>> wrote:
>>>>> >
>>>>> > Unsubscribe
>>>>> >
>>>>> > On Fri, 10 May 2024 at 10:26 PM, Aman Sinha  
>>>>> > wrote:
>>>>> >>
>>>>> >> +1 (non-binding)
>>>>> >>
>>>>> >> On Fri, May 10, 2024 at 7:57 AM Mahesh Raju Somalaraju 
>>>>> >>  wrote:
>>>>> >>>
>>>>> >>> +1(non-binding)
>>>>> >>>
>>>>> >>> Thanks
>>>>> >>> Mahesh Raju S
>>>>> >>>
>>>>> >>> On Fri, 10 May 2024, 06:15 Ayush Saxena,  wrote:
>>>>> >>>>
>>>>> >>>> Hi All,
>>>>> >>>> Following the discussion at [1]. Starting the official vote thread to
>>>>> >>>> mark Hive 2.x release line as EOL.
>>>>> >>>>
>>>>> >>>> Marking a release lines as EOL means there won't be any further
>>>>> >>>> release made for that release line
>>>>> >>>>
>>>>> >>>> I will start with my +1
>>>>> >>>>
>>>>> >>>> -Ayush
>>>>> >>>>
>>>>> >>>>
>>>>> >>>> [1] https://lists.apache.org/thread/91wk3oy1qo953md7941ojg2q97ofsl2d

[VOTE] Mark Hive 2.x EOL

[Ayush Saxena]

Hi All,
Following the discussion at [1]. Starting the official vote thread to
mark Hive 2.x release line as EOL.

Marking a release lines as EOL means there won't be any further
release made for that release line

I will start with my +1

-Ayush


[1] https://lists.apache.org/thread/91wk3oy1qo953md7941ojg2q97ofsl2d

Re: [DISCUSS] End of life for Hive 1.x, 2.x, 3.x

[Ayush Saxena]

We have had the last 2.x release. I don't think we did c-pick our
security issues into the last release either.
 Will start the EOL vote for the 2.x release line as we discussed earlier

-Ayush

On Sun, 21 Apr 2024 at 01:33, Chao Sun  wrote:
>
> Hi Cheng,
>
> Sorry for the long delay. I was distracted by other things.
>
> I just created RC0 for the Hive 2.3.10 release, and started a vote thread. 
> Please help to test if it looks good, especially from Spark side. Thanks!
>
> Best,
> Chao
>
>
>
> On Thu, Apr 18, 2024 at 5:15 AM Cheng Pan  wrote:
>>
>> Hi Chao,
>>
>> The Spark community is starting to discuss the 4.0 release[1], can we make 
>> the Hive 2.3.10 release happen soon?
>>
>> [1] https://lists.apache.org/thread/nxmvz2j7kp96otzlnl3kd277knlb6qgb
>>
>> Thanks,
>> Cheng Pan
>>
>> On 2024/01/17 17:50:37 Chao Sun wrote:
>> > Hi Ayush,
>> >
>> > I'm working on the last few commits to backport to the branch.
>> > Hopefully within 1-2 months I can start the release process. Our goal
>> > is to upgrade Hive 2.x before the Spark 4.0 release coming up mid this
>> > year.
>> >
>> > Chao
>> >
>> > On Tue, Jan 16, 2024 at 10:17 PM Ayush Saxena  wrote:
>> > >
>> > > Thanx everyone for the feedback, I have started a formal thread to mark 
>> > > 1.x
>> > > EOL. We can have one last release for 2.x as Chao mentioned, with some
>> > > required changes + our CVE's & get the release line marked as EOL then.
>> > >
>> > > @Chao Sun  Do let us know if you have a proposed
>> > > timeline for that.
>> > >
>> > > -Ayush
>> > >
>> > > On Wed, 17 Jan 2024 at 08:23, vihang karajgaonkar 
>> > > wrote:
>> > >
>> > > > I was confused about the subject line since it says 3.x as well along 
>> > > > with
>> > > > 1.x and 2.x. Does this discussion include all 1.x, 2.x and 3.x or just 
>> > > > 1.x
>> > > > and 2.x?
>> > > >
>> > > > I think it makes sense to EOL 1.x. Looks like 2.x is still being 
>> > > > maintained
>> > > > by Chao and I think we were backporting PRs to the 3.x line pretty 
>> > > > recently
>> > > > so I believe we should wait out for a release on Hive 3.x.
>> > > >
>> > > > Thanks,
>> > > > Vihang
>> > > >
>> > > > On Tue, Jan 16, 2024 at 3:40 PM Attila Turoczy
>> > > >  wrote:
>> > > >
>> > > > > Dear PMC's,
>> > > > >
>> > > > > Do we have a verdict / decision about this?
>> > > > >
>> > > > > -Attila
>> > > > >
>> > > > > On Wed, Jan 10, 2024 at 5:45 PM Chao Sun  wrote:
>> > > > >
>> > > > > > On Hive 2.x, I'm still preparing for another release 2.3.10 (Hive 
>> > > > > > 2.3
>> > > > > > branch is being actively maintained so far). Hopefully this will be
>> > > > > > the last release in the branch-2 line.
>> > > > > >
>> > > > > > +1 on making Hive 1 EOL for the time being.
>> > > > > >
>> > > > > > Chao
>> > > > > >
>> > > > > > On Wed, Jan 10, 2024 at 8:10 AM Sankar Hariappan
>> > > > > >  wrote:
>> > > > > > >
>> > > > > > > +1 for making both Hive 1&2 EOL
>> > > > > > >
>> > > > > > > -Sankar
>> > > > > > > -Original Message-
>> > > > > > > From: Attila Turoczy 
>> > > > > > > Sent: Wednesday, January 10, 2024 7:37 PM
>> > > > > > > To: dev@hive.apache.org
>> > > > > > > Subject: [EXTERNAL] Re: [DISCUSS] End of life for Hive 1.x, 2.x, 
>> > > > > > > 3.x
>> > > > > > >
>> > > > > > > [You don't often get email from aturo...@cloudera.com.invalid. 
>> > > > > > > Learn
>> > > > > > why this is important at 
>> > > > > > https://aka.ms/LearnAboutSenderIdentification
>> > > > ]
>> > > > > > >
>> > > > > > > +1 for making it EOL for Hive 1 and Hive 2. I do not think these 
>> > > > > > > 2
>> > > > > > product
>> > > > > > > branches are relevant in 2023.
>> > > > > > >
>> > > > > > > -Attila
>> > > > > > >
>> > > > > > > On Wed, Jan 10, 2024 at 12:59 PM Denys Kuzmenko <
>> > > > dkuzme...@apache.org>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > +1 for marking Hive 1.x EOL
>> > > > > > > >
>> > > > > > > > Assuming no volunteers willing to take ownership of branch-2
>> > > > > > maintenance,
>> > > > > > > > +1 to declare it EOL as well.
>> > > > > > > >
>> > > > > > > > Regards,
>> > > > > > > > Denys
>> > > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> >
>>
>>
>>

Re: Query Regarding Core Classified Hive-Exec Artifact in Future Releases

[Ayush Saxena]

Hi Mergu,
There are some reservations as of now, we are discussing that in another
thread. In the meantime, you can explore something as in [1]

-Ayush

[1] https://github.com/zabetak/hive-core-poc

On Mon, 22 Apr 2024 at 11:07, Mergu Ravi
 wrote:

> Hi Team,
> Is there any plan to include core classified HIve-exec artifacts in future
> releases?
>
> On Wed, Apr 17, 2024 at 1:09 PM Mergu Ravi <
> mergu.r...@gaianconsultants.com> wrote:
>
>> Hi Hive Team,
>> From this Hive ticket https://issues.apache.org/jira/browse/HIVE-25531,
>> I understood that the core classified hive-exec artifact was removed. Is
>> there any plan to include this core artifact in upcoming releases?
>> --
>>
>> Thanks & Regards,
>> Ravi Mergu
>>
>
>
> --
>
> Thanks & Regards,
>
>
> 
> Ravi Mergu
> SMTS-2
> o:  +91 40 64535355
> m: +91 9959618687
> w: www.gaiansolutions.com
>
> 
> 
> 
>
> *Please consider the environment before printing this email.* This
> message contains confidential information and is intended only for the
> individual named. If you are not the named addressee you should not
> disseminate, distribute or copy this e-mail
>
>

Re: [Discussion] HIVE-28211: Restore hive-exec:core jar

[Ayush Saxena]

Stamatis,
Isn't the removal itself an incompatible change? There are a lot of
projects using it & we suddenly removed a jar because there were some
people not sure how to properly use it and were complaining about it.

What about the projects which are now stuck? reading the thread at [1],
there were promises made that everything will be relocated and sorted
before the release, but we couldn't, AFAIK it isn't a naive task to just
relocate all the dependencies.

As I see here @Chao Sun , even raised concerns [2], that the removal just
stops the way for upgrading downstream projects and it got countered like
folks chasing the removal will help chase getting all the dependencies
relocated or solve the issues for downstream. I think none volunteered.

I would either recommend:
* Best case we relocate all the dependencies present in hive-exec, not just
one or two. Somebody volunteers to raise one PR relocating "all" and we can
commit that and we should be sorted.
* Restore back the core jar, because a lot of projects depend on it, the
removal itself was incompatible, the removal I don't think had a clear
community agreement, it was a conditional agreement, which I don't think
got sorted, so we should rollback.

On a lighter note, we might release with some 5000+ commits, with best
performance or so, but if nobody is able to consume those release bits, I
think those efforts are just getting waste, eventually people will just
stick to their older versions and not even try to upgrade & we will be
releasing for nobody or maybe for few folks who just have only Hive in
their stack (I don't know if there are folks like that), No matter how good
a product is, if people don't use it, it is gonna die :-(


I think we have a ticket which talks about relocating all dependencies, I
agree we should drop the core jar for sure, it leads to all the problems as
Stamatis mentioned but lets restore the core jar back & we can drop it when
that relocation ticket is resolved. Does that sound convincing, or even
worth a thought?

btw. having jars with a set of dependencies shaded and other ones unshaded
is done in hadoop as well, hadoop-minicluster vs hadoop-client-minicluster
& such problems by users keep on coming, eg [3]

Anyone else, any thoughts?

-Ayush

[1] https://lists.apache.org/thread/cwtxnffoqpwgmdtlc9hyor2cm22djpkg
[2] https://lists.apache.org/thread/23sshgolmbpcc01npqgt03woljdy6hdn
[3] https://lists.apache.org/thread/f47s6bxrtslkxbc8s2gybwrxps8vk63x



On Fri, 26 Apr 2024 at 16:37, Stamatis Zampetakis  wrote:

> Hey Simhadri, thanks for starting this discussion.
>
> Maven has many limitations when it comes to publishing multiple
> artifacts from the same module. In most cases, the end result is
> broken and hard to use. The pom file that is published for a given
> module is not able to describe correctly all artifacts of the module
> and that's why there is one main artifact for every module; dependency
> declarations are usually correct for the main artifact but are not
> representative for the rest.
>
> For example, end-users who consume the hive-exec-core module tend to
> think that maven will automatically resolve all transitive
> dependencies and things will work as usual which is not the case. In
> the past, this kind of assumption created a lot of confusion on
> consumers of the hive-core-exec.jar with tickets and open debates that
> spanned for multiple months. The discussions even reached a point
> where people requested certain features of Hive to be reverted in
> order to rectify some things around transitive dependencies and the
> core jar.
>
> I think we should stick to the usual maven convention and just publish
> one artifact for each module. Adding back and claiming to support the
> "core" jar is a step backwards that just postpones the real problems
> that we need to tackle.
>
> Furthermore, I don't think that the hive-exec module was ever meant to
> be used as a dependency. This is mainly an application module and not
> a library module and that's why shading takes place. Clearly some
> parts from hive-exec could be considered to become a library and that
> would be a promising direction going forward (splitting hive-exec into
> other modules) but a bit outside the scope of the current discussion.
>
> From the issues outlined above the only actionable item that I see
> concerns the joda library so we could try to simply relocate it if it
> is causing issues.
>
> Finally, if someone wants to create a jar with specific contents from
> the hive-exec module it is rather easy to do so. I created a small POC
> project [1] on how someone can create something similar to the
> hive-exec-core.jar and incorporate it in their build. Each project has
> separate needs so for such customization I feel that the burden
> shouldn't fall on the Hive community.
>
> Best,
> Stamatis
>
> [1] https://github.com/zabetak/hive-core-poc
>
> On Thu, Apr 25, 2024 at 11:12 AM Simhadri G  wrote:
> >
> > Hi Everyone,
> >
> > The hive-

[ANNOUNCE] New Committer: Simhadri Govindappa

[Ayush Saxena]

Hi All,
Apache Hive's Project Management Committee (PMC) has invited Simhadri
Govindappa to become a committer, and we are pleased to announce that he
has accepted.

Please join me in congratulating him, Congratulations Simhadri, Welcome
aboard!!!

-Ayush Saxena
(On behalf of Apache Hive PMC)

Re: [Blog] Apache Hive 4.0 Release blog for ASF M & P

[Ayush Saxena]

Update:
I submitted this last week, and I circled back to check, this is being
reviewed and they are dropping, tweaking a couple of things and writing
some Apache stuff & all to follow their template(Which I should have done
myself, they shared the template of Apache Wicket), those things we don’t
have any control over. M&P folks will take care of it. So, it will look a
bit different then what we submitted.

I have been communicated that:* It is supposed to be published before end
of month*

PS. They asked for some quotes for the release, which I have said I don’t
have. We can drop adding that, either they will drop or add from their own.
But I think they will write something themselves

I think it will be finalized soon, I will keep the thread updated as I get
anything or If I find it got published itself :-)

-Ayush


On 5 Apr 2024, at 5:39 PM, Ayush Saxena  wrote:


Hi Stamatis,
I have changed the permissions to allow everyone with the link to have
comment access. I have additionally given you edit access as well.

Let me know if things don't work

-Ayush

On Fri, 5 Apr 2024 at 17:34, Stamatis Zampetakis  wrote:

> Great initiative and nice content. Overall, it looks great!
>
> I have some minor comments. Is it possible to change permissions allow
> comments from anyone or it has to be done on a per user basis?
>
> Best,
> Stamatis
>
> On Fri, Apr 5, 2024 at 1:54 PM Ayush Saxena  wrote:
> >
> > Hi All,
> >
> > Have been talking to the ASF M & P team and they recongonise the 4.0
> release is a big milestone for our project.
> >
> > They are happy to have an entry for us in the their news column, ex:
> >
> https://news.apache.org/foundation/entry/apache-software-foundation-announces-apache-wicket-v10
> >
> > So, I along with Denys, Simhadri & tons of help from ChatGpt have
> prepared a draft to share with them.
> > The draft is here:
> >
> >
> https://docs.google.com/document/d/10Zu8pHvWNDRTqn7yvYqvU4-kw3Q1TXo7mGo5m5fUP2Y/edit
> >
> > If you have some feedback or concerns, please share with us.
> >
> > If you want some improvements or removals, let us know here & we will do
> that, or if you need write access to this page, just let me know.
> >
> > If nobody objects, I plan to send this to the team by next week Tuesday
> >
> >
> > -Ayush
>

Re: Issue with joda-time library bundled in hive-exec:4.0.0

[Ayush Saxena]

Thanx Cheng Pan for sharing the pointers, Do you have any list of issues or
pointers on what are the challenges for Spark to move to a higher Hive
version? I know upgrading libraries is quite challenging but it
is inevitable.

Hive is already in discussion of marking Hive-2.x EOL, so at very best we
would have one release and immediately after that we will announce it EOL,
maintaining a release line is quite an effort for us at Hive & doing it
because other projects doesn't want to upgrade isn't a convincing reason
for most of us. The best we can do is or are trying is to address issues
for Spark whatever we can do as part of Hive code & would definitely need
help/support from Spark side as well, since the move is from 2.x to 4.x, it
would be a big change and would offer resistance on both sides.

So, it would be great help if any pointers can be shared from Spark side
for the move, if there is no help/interest from Spark then we can't do
anything & there is no need for Hive-2.x either in that case :-)

-Ayush

On Wed, 17 Apr 2024 at 15:00, Cheng Pan  wrote:

> > … we are exploring ways to get Spark move from 2.3.9 to 4.0, Our initial
> hunch is that it would be quite challenging without a hive-exec slim jar …
>
> It should be challenging to upgrade Spark’s built-in Hive version.
> Actually, we already did lots of work on branch-2.3 which focuses on CVE
> reduction, for example, allowing Spark to upgrade Guava to modern versions
> to get rid of Guava 14, it was tested with the latest Spark master
> branch[1], maybe we need a release for 2.3.10 now.
>
> [1] https://github.com/apache/spark/pull/45372
>
> Thanks,
> Cheng Pan
>
>
>

Re: [DISCUS] Plan the next Hive release

[Ayush Saxena]

Hi Stamatis,
The plan is to have a release line cut from the branch-4.0, So, we plan to
pull in some critical bug fixes & improvements into the 4.0.1 release and
have a quicker release.
As of now we are just putting the label "hive-4.0.1-must" on the tickets
and we plan to make sure those get c-picked to the release line. AFAIK we
haven't started committing to any branch yet, was waiting if anyone feels
differently, so we can hold back if you have concerns or take a different
approach as well.

>From CI you mean to say the daily builds? else if you create a PR
targeting to branch-4.0, it will run the entire test suite I believe? In
the meantime I will update the instructions regarding the target branch &
the label if anyone wants that a particular ticket to be part of the 4.0.1
release.

-Ayush

On Wed, 17 Apr 2024 at 12:42, Stamatis Zampetakis  wrote:

> Thanks for starting the discussion Ayush.
>
> Having frequent releases is definitely needed so we should keep the
> momentum going.
>
> I had the impression from other threads that the next Hive release
> would be 4.1.0 and that it would be cut from master. I would like to
> understand how 4.0.1 is different and if it is, what is the
> contribution pattern that contributors and committers should follow?
> If the idea is to maintain and commit in two (or more) branches the
> steps should be documented and CI should be running on those branches.
>
> Best,
> Stamatis
>
> On Wed, Apr 10, 2024 at 1:18 PM Denys Kuzmenko 
> wrote:
> >
> > We might need it sooner as identified some critical issues in the recent
> code:
> > 1. HIVE-28166: Truncate on Iceberg table disregards the branch name and
> operates on a main;
> > 2. HIVE-28190: Materialized view rebuild lock heart-beating is broken;
>

Re: Issue with joda-time library bundled in hive-exec:4.0.0

[Ayush Saxena]

Some can't be relocated, it is pretty challenging to relocate them, one I
remember is calcite, I tried that here:
https://issues.apache.org/jira/browse/HIVE-23772 & it didn't work, I am not
sure if things have changed now.

Relocated dependencies shouldn't be problematic I believe, having jars
shaded but not relocated is problematic in some cases, you can't get rid of
them via exclusion and then they tend to create classpath conflicts, we are
exploring ways to get Spark move from 2.3.9 to 4.0, Our initial hunch is
 that it would be quite challenging without a hive-exec slim jar which
doesn't have these dependencies or has minimal shading.

-Ayush

On Tue, 16 Apr 2024 at 21:07, László Bodor 
wrote:

> isn't this as simple to solve as relocating joda (or what causes problems
> in general)? I cannot recall when and why we removed the core jar, and I'm
> not against it, but I feel that relocating something that is already shaded
> in the uber jar is much easier
>
> looks like we shade a lot:
> https://github.com/apache/hive/blob/f06cc2920424817da6405e0efe268ce6cd64a363/ql/pom.xml#L1032
> but relocate only a few:
> https://github.com/apache/hive/blob/f06cc2920424817da6405e0efe268ce6cd64a363/ql/pom.xml#L1110
>
> we already have a huge uber jar due to including everything, I feel
> relocation doesn't make it any worse
> relocation is only dangerous if the relocated classes are present in
> hive-exec public method signatures (TODO check joda from this aspect)
>
> Mergu Ravi  ezt írta (időpont:
> 2024. ápr. 16., K, 17:21):
>
>> When can we expect the core jar for 4.0.0 version.
>>
>> On Tue, Apr 16, 2024 at 4:57 PM Denys Kuzmenko 
>> wrote:
>>
>>> Hi Ravi,
>>>
>>> hive-exec is the uber jar, the same thing is done in Hive-3.x. We are
>>> considering restoring the hive-core jar that only has classes from ql
>>>
>>> On 2024/04/16 06:24:20 Mergu Ravi wrote:
>>> > Hi Hive Team,
>>> >
>>> > I'm currently using the Hive 4.0.0 libraries and facing an issue with
>>> the
>>> > hive-exec:4.0.0 library. It seems that the Joda-Time library is bundled
>>> > within it, preventing me from excluding it and using my existing
>>> Joda-Time
>>> > library. Could you please provide insight into why this bundling has
>>> been
>>> > implemented? Additionally, is there any plan to unbundle the Joda-Time
>>> > library in future releases?
>>> >
>>> > --
>>> >
>>> > Thanks & Regards,
>>> >
>>> >
>>> > 
>>> > Ravi Mergu
>>> > SMTS-2
>>> > o:  +91 40 64535355
>>> > m: +91 9959618687
>>> > w: www.gaiansolutions.com
>>> >
>>> > 
>>> > 
>>> > 
>>> >
>>> > *Please consider the environment before printing this email.* This
>>> message
>>> > contains confidential information and is intended only for the
>>> individual
>>> > named. If you are not the named addressee you should not disseminate,
>>> > distribute or copy this e-mail
>>> >
>>>
>>
>>
>> --
>>
>> Thanks & Regards,
>>
>>
>> 
>> Ravi Mergu
>> SMTS-2
>> o:  +91 40 64535355
>> m: +91 9959618687
>> w: www.gaiansolutions.com
>>
>> 
>> 
>> 
>>
>> *Please consider the environment before printing this email.* This
>> message contains confidential information and is intended only for the
>> individual named. If you are not the named addressee you should not
>> disseminate, distribute or copy this e-mail
>>
>>

[ANNOUNCE] Hive 1.x EOL

[Ayush Saxena]

Hi All,
The Apache Hive Community has voted to declare the 1.x release line as End of 
Life (EOL). This means no further updates or releases will be made for this 
series.

We urge all Hive 1.x users to upgrade to the latest versions promptly to 
benefit from new features and ongoing support.

-Ayush Saxena
(On Behalf of Apache Hive PMC)

[DISCUS] Plan the next Hive release

[Ayush Saxena]

Hi All,
As we all know Hive-4.0 is released. I think we should try to maintain a
regular cadence for the 4.x release line.
So, I propose having a 4.0.1 release in the next 3 months or so, with some
critical bug fixes and improvements on top of our last 4.0.0 release.

We would need someone to volunteer as the Release Manager as well, if folks
agree.

Thoughts?

-Ayush

Re: [Blog] Apache Hive 4.0 Release blog for ASF M & P

[Ayush Saxena]

A reminder, I have addressed comments from everyone so far. I plan to send
this tomorrow. Do give a check guys. If anyone wants me to hold or see I
missed any of their comments, just let me know, it might have got skipped
unintentionally.

-Ayush

On Fri, 5 Apr 2024 at 22:08, Simhadri G  wrote:

> Looks great,  thanks Ayush! :)
>
> On Fri, Apr 5, 2024, 8:54 PM Butao Zhang  wrote:
>
>> Good job Ayush!
>> Hope this can make more people know that Apache Hive 4.0 is really ready
>> to be used!
>>
>>
>> Thanks,
>> Butao Zhang
>>  Replied Message 
>> From Ayush Saxena 
>> Date 4/5/2024 19:55
>> To dev 
>> Subject [Blog] Apache Hive 4.0 Release blog for ASF M & P
>> Hi All,
>>
>> Have been talking to the ASF M & P team and they recongonise the 4.0
>> release is a big milestone for our project.
>>
>> They are happy to have an entry for us in the their news column, ex:
>>
>> https://news.apache.org/foundation/entry/apache-software-foundation-announces-apache-wicket-v10
>>
>> So, I along with Denys, Simhadri & tons of help from ChatGpt have
>> prepared a draft to share with them.
>> The draft is here:
>>
>>
>> https://docs.google.com/document/d/10Zu8pHvWNDRTqn7yvYqvU4-kw3Q1TXo7mGo5m5fUP2Y/edit
>>
>> If you have some feedback or concerns, please share with us.
>>
>> If you want some improvements or removals, let us know here & we will do
>> that, or if you need write access to this page, just let me know.
>>
>> If nobody objects, I plan to send this to the team by next week Tuesday
>>
>>
>> -Ayush
>>
>

Re: Next Jira Fix Version after 4.0.0

[Ayush Saxena]

Hi Laszlo,
The next version is 4.1.0 only. AFAIK the target fix version is by practice
the one defined in the pom.xml file.

https://github.com/apache/hive/blob/master/pom.xml#L24

-Ayush

On Fri, 5 Apr 2024 at 20:36, László Bodor  wrote:

> Now, as Apache Hive 4.0.0 has been released, it's time to find a new Fix
> Version to make devs able to pick a valid one when resolving new tickets.
> I'm proposing *4.1.0*, which can be bulk-changed later if we decide
> otherwise. Any opinions about this?
>
> Thanks,
> Laszlo Bodor
>
>

Re: [Blog] Apache Hive 4.0 Release blog for ASF M & P

[Ayush Saxena]

Hi Stamatis,
I have changed the permissions to allow everyone with the link to have
comment access. I have additionally given you edit access as well.

Let me know if things don't work

-Ayush

On Fri, 5 Apr 2024 at 17:34, Stamatis Zampetakis  wrote:

> Great initiative and nice content. Overall, it looks great!
>
> I have some minor comments. Is it possible to change permissions allow
> comments from anyone or it has to be done on a per user basis?
>
> Best,
> Stamatis
>
> On Fri, Apr 5, 2024 at 1:54 PM Ayush Saxena  wrote:
> >
> > Hi All,
> >
> > Have been talking to the ASF M & P team and they recongonise the 4.0
> release is a big milestone for our project.
> >
> > They are happy to have an entry for us in the their news column, ex:
> >
> https://news.apache.org/foundation/entry/apache-software-foundation-announces-apache-wicket-v10
> >
> > So, I along with Denys, Simhadri & tons of help from ChatGpt have
> prepared a draft to share with them.
> > The draft is here:
> >
> >
> https://docs.google.com/document/d/10Zu8pHvWNDRTqn7yvYqvU4-kw3Q1TXo7mGo5m5fUP2Y/edit
> >
> > If you have some feedback or concerns, please share with us.
> >
> > If you want some improvements or removals, let us know here & we will do
> that, or if you need write access to this page, just let me know.
> >
> > If nobody objects, I plan to send this to the team by next week Tuesday
> >
> >
> > -Ayush
>

[Blog] Apache Hive 4.0 Release blog for ASF M & P

[Ayush Saxena]

Hi All,

Have been talking to the ASF M & P team and they recongonise the 4.0
release is a big milestone for our project.

They are happy to have an entry for us in the their news column, ex:
https://news.apache.org/foundation/entry/apache-software-foundation-announces-apache-wicket-v10

So, I along with Denys, Simhadri & tons of help from ChatGpt have prepared
a draft to share with them.
The draft is here:

https://docs.google.com/document/d/10Zu8pHvWNDRTqn7yvYqvU4-kw3Q1TXo7mGo5m5fUP2Y/edit

If you have some feedback or concerns, please share with us.

If you want some improvements or removals, let us know here & we will do
that, or if you need write access to this page, just let me know.

If nobody objects, I plan to send this to the team by next week Tuesday


-Ayush

Re: [VOTE] Release Apache Hive 4.0.0 (Release Candidate 0)

[Ayush Saxena]

+1 (Binding)

* Built from source
* Verified checksums
* Verified signature
* Verified all code files have ASF Header
* Validated the Notice & License files
* No code diff b/w git tag & src tar
* Ran some basic operations on Iceberg, ACID & External Tables (Hive on Tez)
* Browsed through HS2 UI
* Built Docker image from source & tried some basic commands on the docker
environment.
* Skimmed over the contents of maven repo.

Thanx Denys for driving the release. Good Luck!!!

-Ayush

On Wed, 27 Mar 2024 at 21:05, Marta Kuczora 
wrote:

> +1 (binding)
>
> Thanks a lot Denys for driving the release!
>
> * Verified the checksum and signature [OK]
>
> * Built Hive 4.0.0 from source [OK]
>
> * Initialized metastore with MySQL [OK]
>
> * Built package and ran metastore and hiveserver [OK]
>
> * Deployed and start the binary tar with Hadoop 3.3.6 and Tez 0.10.3 [OK]
>
> * Ran some simple Hive queries with external/acid/iceberg tables [OK]
>
>
> Regards,
>
> Marta
>
> On Tue, Mar 26, 2024 at 8:26 AM Denys Kuzmenko 
> wrote:
>
> > Hi Everyone,
> >
> > We would like to thank everyone who has contributed to the project and
> > request
> > the Hive PMC members to review and vote on this new release candidate.
> >
> > Apache Hive 4.0.0 RC-0 artifacts are available here:*
> > https://people.apache.org/~dkuzmenko/apache-hive-4.0.0-rc0/
> >
> >
> > The checksums are as follows:
> > - 83eb88549ae88d3df6a86bb3e2526c7f4a0f21acafe21452c18071cee058c666
> > apache-hive-4.0.0-bin.tar.gz
> > - 4dbc9321d245e7fd26198e5d3dff95e5f7d0673d54d0727787d72956a1bca4f5
> > apache-hive-4.0.0-src.tar.gz
> >
> >
> > You can find the KEYS file here:
> >
> > * https://downloads.apache.org/hive/KEYS
> >
> >
> > A staged Maven repository URL is:*
> > https://repository.apache.org/content/repositories/orgapachehive-1127/
> >
> > The git commit hash is:*
> >
> >
> https://github.com/apache/hive/commit/183f8cb41d3dbed961ffd27999876468ff06690c
> >
> >
> > This corresponds to the tag: release-4.0.0-rc0
> > * https://github.com/apache/hive/tree/release-4.0.0-rc0
> >
> > The vote is open for the next 72 hours and passes if a majority of at
> least
> > three +1 PMC votes are cast.
> >
> > (Only PMC members have binding votes, however, other community members
> > are encouraged to cast non-binding votes.)
> >
> >
> > [ ] +1 Release this package as Apache Hive 4.0.0
> > [ ] +0
> > [ ] -1 Do not release this because...
> >
> >
> > Please download, verify, and test.
> >
> >
> > Regards,
> >
> > Denys
> >
>

Re: Re: [EXTERNAL] Re: [VOTE] Mark Hive 1.x EOL

[Ayush Saxena]

Thanx everyone for voting. If I count right, we have 13 +1 votes (8
binding) & no vetoes.

The vote passes. I will shoot an announcement in a couple of days and
update the website to reflect the decision

-Ayush

On Fri, 19 Jan 2024 at 18:19, Krisztian Kasa 
wrote:

> +1
>
> On Fri, Jan 19, 2024 at 1:33 PM dengzhhu653  wrote:
>
> > +1 (non-binding)Best,Zhihua
> > At 2024-01-17 17:07:01, "László Bodor" 
> wrote:
> > >+1 (binding)
> > >
> > >Sankar Hariappan  ezt írta
> > >(időpont: 2024. jan. 17., Sze, 10:05):
> > >
> > >> +1
> > >>
> > >> -Sankar
> > >>
> > >> -Original Message-
> > >> From: Stamatis Zampetakis 
> > >> Sent: Wednesday, January 17, 2024 1:56 PM
> > >> To: dev@hive.apache.org
> > >> Subject: [EXTERNAL] Re: [VOTE] Mark Hive 1.x EOL
> > >>
> > >> +1 (binding)
> > >>
> > >> Best,
> > >> Stamatis
> > >>
> > >> On Wed, Jan 17, 2024 at 8:21 AM Attila Turoczy
> > >>  wrote:
> > >> >
> > >> > +1
> > >> >
> > >> > -Attila
> > >> >
> > >> > On Tue, 16 Jan 2024 at 22:18, Butao Zhang 
> > wrote:
> > >> >
> > >> > > +1
> > >> > >
> > >> > >
> > >> > >
> > >> > > Thanks,
> > >> > > Butao Zhang
> > >> > >  Replied Message 
> > >> > > | From | Ayush Saxena | Date | 1/17/2024
> 14:15
> > |
> > >> > > | To | dev | Subject | [VOTE] Mark Hive 1.x
> > EOL
> > >> > > | |
> > >> > > Hi All,
> > >> > > Following the discussion in [1], Starting an official thread to
> mark
> > >> > > Hive 1.x EOL.
> > >> > >
> > >> > > Marking a release line EOL, means there won't be any further
> > >> > > releases for that release line.
> > >> > >
> > >> > > I will start with my +1
> > >> > >
> > >> > > -Ayush
> > >> > >
> > >> > > [1]
> > >> > > https://lists.apache.org/thread/sxcrcf4v9j630tl9domp0bn4m33bdq0s
> > >> > >
> > >>
> >
>

Re: [DISCUSS] Migrate precommit git repos from kgyrtkirk to apache

[Ayush Saxena]

Ok Zoltan, you are always right, but listening to other people sometimes
doesn't hurt, maybe even if they aren't as smart as you (like me & everyone
you consider not so smart like you).

Let me ask something:
* If something breaks like code or something like that? What do you do?
Humiliate or throw sarcastic comments?
-> In general no, We go and help fix it, or ask the guy to revert it &
share what you feel is right? if there isn't an agreement, we discuss not
like "Whatever you want to do", for you that might be fancy, but not for
others

-> You don't like an approach? The way I did it? or "I did it"
It is all Ok, that apache repo actually gives rights to all the hive
committers, whether I know them, you know them or nobody knows them. anyone
can do what they like & consider good for Apache Hive, that is opensource.
The amount of time we are spending on this thread, that much time would
have sorted things out

-> Your response mechanism?
How tough is it to spread a sense of inclusive community? How tough is it
to write in a humble way? I am not convinced with this approach? I know a
better approach? That is X->Y->Z, We should do that? or I will go ahead and
do that? Did anyone stop anyone from doing anything here? What is the point
being proven, that someone is superior & can go freely & yell or scold
people, or it is demonstrating "I deserve respect not the others"

-> Is there a discussion about the topic?
How long do you want the discussion? If you were particular about an
approach, you could have shared it, I went wrong, you can still share it,
you don't want me to do it, you can still do it yourself, and believe me
even if you screw up things. None of us will come up & throw any tantrums
or sarcastic comments. That is not most of us.

I maybe wrong, anyone can be wrong, the only person who didn't do anything
wrong or broke anything is someone who didn't do anything ambitious...

I will still try to answer your questions Zoltan

>  I was replacing the CI I've used it to get a good base ground for
running the tests - as it could prepare a lot of things already.
I had to do a lot of things - and the move of the repo was never at the top
of the list...

It is all "I" my friend, you thought, till you were only using it, it was
good. But Hive started using it, It wasn't. How many Apache projects rely
on even making a JDK change for their test to run on external folks, using
your personal fork itself at the very first point was violation of apache
way. Migration should have been done first, or other alternatives should
have been explored rather than relying on some non apache fork

> That's not true either as you can see in a week-or-so old comment from me

And again, You want all "Apache" contributors to follow your repo & follow
these comments?

-> Reading the comment?

We push it our own docker hub space & push a commit to my own docker hub
space or rely on "you" to build or add "me" as a collaborator? The project
& its component aren't own by any of the member, good news not me, bad news
not by you either. The project & it is infra, CI resources are owned by the
Project PMC (not any member) & the PMC is no way superior to any
contributor. That is basic Apache stuff, I know you know them better than
me.

I still don't want to drag it further, Zoltan you are the most important
person in this thread, the code is in your repo, whatever way you want, we
will get that done the same way. Does that make you happy? But please get
away with this attitude, none of us, not me, nor any contributor is at
mercy of anyone else, everyone is equal, equal right, have all rights to
make mistakes, discuss, get them corrected & still come back to this place
without any humiliation with his head held high. Community over Code

I think you will retrospect, you are a respected member of the community &
will work in a way it is good & peaceful for all, rest I can't help it.

I will let Stamatis & you or anyone you like go forward with this. Stamatis
has all rights around the repo, if you want any deletion or so, rest INFRA
ticket can get anything sorted, if it requires me to write something, I can
accept it was my bad, fortunately I don't have ego issues :-)

With a heavy heart!!!
-Ayush


On Tue, 23 Jan 2024 at 15:05, Zoltan Haindrich  wrote:

>
> On 1/23/24 10:10, Ayush Saxena wrote:
> > Ok I will get the repo deleted. I am not taking any sarcastic comments
> from Zoltan at this stage. Believe me I am not getting anything for having
> my name there.
> >
> > Why I did this?
> >
> > Someone was so obsessed with getting his name checked into the "Apache
> Code" that he developed something on his fork & checked in that code to the
> Apache Hive

Re: [DISCUSS] Migrate precommit git repos from kgyrtkirk to apache

[Ayush Saxena]

Ok I will get the repo deleted. I am not taking any sarcastic comments from
Zoltan at this stage. Believe me I am not getting anything for having my
name there.

Why I did this?

Someone was so obsessed with getting his name checked into the "Apache
Code" that he developed something on his fork & checked in that code to the
Apache Hive code, so, professional.

Many Hive Commiters have rights is a wrong phrase to quote: Many Hive
Committer who are your friends have rights. To push an image we need to
catch Zoltan, but ok do whatever you want.

I just want to say, Zoltan, you might be a very good developer, but please
change your "whatever you want to do" tone,

Not following this further

-Ayush

On Tue, 23 Jan 2024 at 14:30, Zoltan Haindrich  wrote:

>
>  > I just copied the repo: cp -R and Put Zoltan's name & reference to his
>  > repo. I didn't knew any better way than that, you can definitely force
> push
>  > with another fancy approach
>
> lol...what a sophisticated approach - I wonder if you don't know the
> `fancy approach` then why you've done it?
>
> I wonder what you've copied - because you missed the addition of the
> github action which builds the image for every PR
>
> Now you are the sole contributor of all existing stuff (congrats)...but do
> whatever you want...
> It was always there and available to use - many hive commiters had push
> and approve rights on those repos.
>
> I think you might also want to do the same with
> https://github.com/kgyrtkirk/hive-toolbox
> because your contribution references it here:
> https://github.com/apache/hive-dev-box/blob/663625bc74e799f35c6bab1c1485530367287c61/tools/install_toolbox#L21C1-L21C115
> and probably also cp -R
> https://github.com/kgyrtkirk/hive-test-kube/
>
> cheers,
> Zoltan
>
>
> On 1/23/24 09:29, Ayush Saxena wrote:
> > I just copied the repo: cp -R and Put Zoltan's name & reference to his
> > repo. I didn't knew any better way than that, you can definitely force
> push
> > with another fancy approach, just c-pick the other commits for NOTICE &
> all
> > on top of it. The old code & commits had some cloudera references, which
> I
> > personally wanted to avoid, but yep we can take another approach as well.
> > Good with me.
> >
> > For the Jira, yep we should, we aren't going to release this, so for fix
> > version, maybe I will create a dev-box-1.0.0 which we can use to resolve
> > the tickets, shouldn't put main repo versions, else that will pop up in
> our
> > release notes, or let me know if you want a separate Jira project under
> > Hive for these repos as well, We can explore that route if folks feel
> that
> > way.
> >
> > -Ayush
> >
> >
> > On Tue, 23 Jan 2024 at 13:35, Stamatis Zampetakis 
> wrote:
> >
> >> Thanks for helping advance this Ayush!
> >>
> >> I saw that the commit history was not retained. Is there any reason
> >> for dropping it? Keeping the history and the people who contributed
> >> thus far would be nice to have.
> >>
> >> For the contribution model to this repository, I would recommend the
> >> usual process. Raise a JIRA ticket, file a PR, wait for review, and
> >> then merge.
> >>
> >> Best,
> >> Stamatis
> >>
> >> On Tue, Jan 23, 2024 at 8:45 AM Ayush Saxena 
> wrote:
> >>>
> >>> This is the new repo:
> >>> https://github.com/apache/hive-dev-box
> >>>
> >>> It has the initial code from Zoltan, LICENSE, NOTICE & Disclaimer-WIP
> >>> file + I added Apache Header to all the files wherever possible. We
> need
> >> a
> >>> docker space to push these built images, have requested INFRA for the
> >> same.
> >>>
> >>> The repo is in WIP stage, If you find something problematic, please
> push
> >> a
> >>> fix to the repo or let me know.
> >>>
> >>> Some observations:
> >>> * The build command works on x86 box only not on aarch64,
> >>> * The github action to push the images doesn't work, that needs to be
> >> fixed
> >>>
> >>> -Ayush
> >>>
> >>> On Mon, 22 Jan 2024 at 21:55, Ayush Saxena  wrote:
> >>>
> >>>> I think we are now not using Zoltan's repo. We are using a fork from a
> >>>> contributor in the hive code.[1], I will go ahead and create a repo
> >> under
> >>>> Apache Hive for hive-dev-box tomorrow & put the LICENSE, NOTICE &
> 

Re: [DISCUSS] Migrate precommit git repos from kgyrtkirk to apache

[Ayush Saxena]

I just copied the repo: cp -R and Put Zoltan's name & reference to his
repo. I didn't knew any better way than that, you can definitely force push
with another fancy approach, just c-pick the other commits for NOTICE & all
on top of it. The old code & commits had some cloudera references, which I
personally wanted to avoid, but yep we can take another approach as well.
Good with me.

For the Jira, yep we should, we aren't going to release this, so for fix
version, maybe I will create a dev-box-1.0.0 which we can use to resolve
the tickets, shouldn't put main repo versions, else that will pop up in our
release notes, or let me know if you want a separate Jira project under
Hive for these repos as well, We can explore that route if folks feel that
way.

-Ayush


On Tue, 23 Jan 2024 at 13:35, Stamatis Zampetakis  wrote:

> Thanks for helping advance this Ayush!
>
> I saw that the commit history was not retained. Is there any reason
> for dropping it? Keeping the history and the people who contributed
> thus far would be nice to have.
>
> For the contribution model to this repository, I would recommend the
> usual process. Raise a JIRA ticket, file a PR, wait for review, and
> then merge.
>
> Best,
> Stamatis
>
> On Tue, Jan 23, 2024 at 8:45 AM Ayush Saxena  wrote:
> >
> > This is the new repo:
> > https://github.com/apache/hive-dev-box
> >
> > It has the initial code from Zoltan, LICENSE, NOTICE & Disclaimer-WIP
> > file + I added Apache Header to all the files wherever possible. We need
> a
> > docker space to push these built images, have requested INFRA for the
> same.
> >
> > The repo is in WIP stage, If you find something problematic, please push
> a
> > fix to the repo or let me know.
> >
> > Some observations:
> > * The build command works on x86 box only not on aarch64,
> > * The github action to push the images doesn't work, that needs to be
> fixed
> >
> > -Ayush
> >
> > On Mon, 22 Jan 2024 at 21:55, Ayush Saxena  wrote:
> >
> > > I think we are now not using Zoltan's repo. We are using a fork from a
> > > contributor in the hive code.[1], I will go ahead and create a repo
> under
> > > Apache Hive for hive-dev-box tomorrow & put the LICENSE, NOTICE &
> > > DISCLAIMER-WIP files in it, Then will take things from there, atleast
> it
> > > would be a starting point and all of us can take care of things from
> there
> > > slowly-slowly :-)
> > >
> > > Shout out, if anyone has objections around it.
> > >
> > > -Ayush
> > >
> > >
> > > [1]
> > >
> https://github.com/apache/hive/blob/1aeaff2057a2f4c241f8bcc53a2a529e6e7f45d4/Jenkinsfile#L124C44-L124C65
> > >
> > > On Wed, 6 Sept 2023 at 20:11, Stamatis Zampetakis 
> > > wrote:
> > >
> > >> Based on the discussion under LEGAL-653, it seems that the only
> > >> requirement to migrate the repos under the Apache namespace is to
> > >> apply the AL2 license in the majority of the files in there.
> > >>
> > >> I am looking for volunteers so that we can review the existing code in
> > >> those repo and apply the AL2 license where possible. Depending on how
> > >> many people step up we can divide the work accordingly.
> > >>
> > >> It would be interesting to see if we can use RAT [1] to
> > >> automate/assist  in this process.
> > >>
> > >> Best,
> > >> Stamatis
> > >>
> > >> [1] https://creadur.apache.org/rat/apache-rat-plugin/usage.html
> > >>
> > >> On Thu, Aug 24, 2023 at 11:05 AM Stamatis Zampetakis <
> zabe...@gmail.com>
> > >> wrote:
> > >> >
> > >> > For the licensing question, I just created LEGAL-653 [1].
> > >> >
> > >> > [1] https://issues.apache.org/jira/browse/LEGAL-653
> > >> >
> > >> > On Thu, Aug 24, 2023 at 11:55 AM Stamatis Zampetakis <
> zabe...@gmail.com>
> > >> wrote:
> > >> > >
> > >> > > Creating the new repos should be kind of trivial. I think I will
> be
> > >> > > able to do it using https://selfserve.apache.org/.
> > >> > >
> > >> > > Since this will bring quite a bit of code under the ASF I will
> wait a
> > >> > > couple of days till I create the new repos.
> > >> > >
> > >> > > Once this is done, I think we can simply push the content from
> the old
> > >

Re: [DISCUSS] Migrate precommit git repos from kgyrtkirk to apache

[Ayush Saxena]

This is the new repo:
https://github.com/apache/hive-dev-box

It has the initial code from Zoltan, LICENSE, NOTICE & Disclaimer-WIP
file + I added Apache Header to all the files wherever possible. We need a
docker space to push these built images, have requested INFRA for the same.

The repo is in WIP stage, If you find something problematic, please push a
fix to the repo or let me know.

Some observations:
* The build command works on x86 box only not on aarch64,
* The github action to push the images doesn't work, that needs to be fixed

-Ayush

On Mon, 22 Jan 2024 at 21:55, Ayush Saxena  wrote:

> I think we are now not using Zoltan's repo. We are using a fork from a
> contributor in the hive code.[1], I will go ahead and create a repo under
> Apache Hive for hive-dev-box tomorrow & put the LICENSE, NOTICE &
> DISCLAIMER-WIP files in it, Then will take things from there, atleast it
> would be a starting point and all of us can take care of things from there
> slowly-slowly :-)
>
> Shout out, if anyone has objections around it.
>
> -Ayush
>
>
> [1]
> https://github.com/apache/hive/blob/1aeaff2057a2f4c241f8bcc53a2a529e6e7f45d4/Jenkinsfile#L124C44-L124C65
>
> On Wed, 6 Sept 2023 at 20:11, Stamatis Zampetakis 
> wrote:
>
>> Based on the discussion under LEGAL-653, it seems that the only
>> requirement to migrate the repos under the Apache namespace is to
>> apply the AL2 license in the majority of the files in there.
>>
>> I am looking for volunteers so that we can review the existing code in
>> those repo and apply the AL2 license where possible. Depending on how
>> many people step up we can divide the work accordingly.
>>
>> It would be interesting to see if we can use RAT [1] to
>> automate/assist  in this process.
>>
>> Best,
>> Stamatis
>>
>> [1] https://creadur.apache.org/rat/apache-rat-plugin/usage.html
>>
>> On Thu, Aug 24, 2023 at 11:05 AM Stamatis Zampetakis 
>> wrote:
>> >
>> > For the licensing question, I just created LEGAL-653 [1].
>> >
>> > [1] https://issues.apache.org/jira/browse/LEGAL-653
>> >
>> > On Thu, Aug 24, 2023 at 11:55 AM Stamatis Zampetakis 
>> wrote:
>> > >
>> > > Creating the new repos should be kind of trivial. I think I will be
>> > > able to do it using https://selfserve.apache.org/.
>> > >
>> > > Since this will bring quite a bit of code under the ASF I will wait a
>> > > couple of days till I create the new repos.
>> > >
>> > > Once this is done, I think we can simply push the content from the old
>> > > repos to the new ones. To avoid any kind of IP problems it would be
>> > > best if Zoltan does this step.
>> > >
>> > > One thing that we may need to be careful about is the licensing of
>> > > these repos. We are not going to make source releases from there but
>> > > since the code will be under the ASF namespace people will assume that
>> > > it is ASF licensed so they may start copy-pasting stuff from there.
>> > >
>> > > Is there anything preventing us from putting the code under the AL2
>> license?
>> > >
>> > > Best,
>> > > Stamatis
>> > >
>> > > On Wed, Aug 23, 2023 at 6:14 PM Attila Turoczy
>> > >  wrote:
>> > > >
>> > > > Thank you, Stamatis! Also, Zoltan for the "donation" :)
>> > > >
>> > > > -Attila
>> > > >
>> > > > On Wed, Aug 23, 2023 at 4:53 PM Ayush Saxena 
>> wrote:
>> > > >
>> > > > > +1,
>> > > > > Thanx Stamatis foe initiating this. This was something which was
>> in my
>> > > > > mind as well since long but couldn’t find time.
>> > > > >
>> > > > > -Ayush
>> > > > >
>> > > > > > On 23-Aug-2023, at 6:19 PM, Zoltan Haindrich 
>> wrote:
>> > > > > >
>> > > > > > Hey Stamatis!
>> > > > > >
>> > > > > > I'm happy to donate these repos / help with the migration!
>> > > > > > I should have done it earlier - but it was never top
>> priority...thank
>> > > > > you for initiating it!
>> > > > > >
>> > > > > > cheers,
>> > > > > > Zoltan
>> > > > > >
>> > > > > >> On 8/23/23 14:00, Stamatis Zampetakis wrote:
>> > > > > >> Hi all,
>> > > > > >> Our precommit infrastructure uses code that resides in the
>> following
>> > > > > repos.
>> > > > > >> * https://github.com/kgyrtkirk/hive-test-kube
>> > > > > >> * https://github.com/kgyrtkirk/hive-toolbox
>> > > > > >> * https://github.com/kgyrtkirk/hive-dev-box
>> > > > > >> These are mainly maintained by Zoltán Haindrich who is always
>> helpful
>> > > > > >> and kind to investigate and resolve issues.
>> > > > > >> For facilitating contributions from the apache community and
>> also
>> > > > > >> removing some burden from Zoltan's shoulders it may be a good
>> time to
>> > > > > >> migrate those and put them under the apache namespace.
>> > > > > >> For the initial migration, we could have a straightforward 1
>> to 1
>> > > > > >> mapping as shown below:
>> > > > > >> * https://github.com/apache/hive-test-kube
>> > > > > >> * https://github.com/apache/hive-toolbox
>> > > > > >> * https://github.com/apache/hive-dev-box
>> > > > > >> How do you feel about this?
>> > > > > >> Best,
>> > > > > >> Stamatis
>> > > > >
>>
>

Re: [DISCUSS] Migrate precommit git repos from kgyrtkirk to apache

[Ayush Saxena]

I think we are now not using Zoltan's repo. We are using a fork from a
contributor in the hive code.[1], I will go ahead and create a repo under
Apache Hive for hive-dev-box tomorrow & put the LICENSE, NOTICE &
DISCLAIMER-WIP files in it, Then will take things from there, atleast it
would be a starting point and all of us can take care of things from there
slowly-slowly :-)

Shout out, if anyone has objections around it.

-Ayush


[1]
https://github.com/apache/hive/blob/1aeaff2057a2f4c241f8bcc53a2a529e6e7f45d4/Jenkinsfile#L124C44-L124C65

On Wed, 6 Sept 2023 at 20:11, Stamatis Zampetakis  wrote:

> Based on the discussion under LEGAL-653, it seems that the only
> requirement to migrate the repos under the Apache namespace is to
> apply the AL2 license in the majority of the files in there.
>
> I am looking for volunteers so that we can review the existing code in
> those repo and apply the AL2 license where possible. Depending on how
> many people step up we can divide the work accordingly.
>
> It would be interesting to see if we can use RAT [1] to
> automate/assist  in this process.
>
> Best,
> Stamatis
>
> [1] https://creadur.apache.org/rat/apache-rat-plugin/usage.html
>
> On Thu, Aug 24, 2023 at 11:05 AM Stamatis Zampetakis 
> wrote:
> >
> > For the licensing question, I just created LEGAL-653 [1].
> >
> > [1] https://issues.apache.org/jira/browse/LEGAL-653
> >
> > On Thu, Aug 24, 2023 at 11:55 AM Stamatis Zampetakis 
> wrote:
> > >
> > > Creating the new repos should be kind of trivial. I think I will be
> > > able to do it using https://selfserve.apache.org/.
> > >
> > > Since this will bring quite a bit of code under the ASF I will wait a
> > > couple of days till I create the new repos.
> > >
> > > Once this is done, I think we can simply push the content from the old
> > > repos to the new ones. To avoid any kind of IP problems it would be
> > > best if Zoltan does this step.
> > >
> > > One thing that we may need to be careful about is the licensing of
> > > these repos. We are not going to make source releases from there but
> > > since the code will be under the ASF namespace people will assume that
> > > it is ASF licensed so they may start copy-pasting stuff from there.
> > >
> > > Is there anything preventing us from putting the code under the AL2
> license?
> > >
> > > Best,
> > > Stamatis
> > >
> > > On Wed, Aug 23, 2023 at 6:14 PM Attila Turoczy
> > >  wrote:
> > > >
> > > > Thank you, Stamatis! Also, Zoltan for the "donation" :)
> > > >
> > > > -Attila
> > > >
> > > > On Wed, Aug 23, 2023 at 4:53 PM Ayush Saxena 
> wrote:
> > > >
> > > > > +1,
> > > > > Thanx Stamatis foe initiating this. This was something which was
> in my
> > > > > mind as well since long but couldn’t find time.
> > > > >
> > > > > -Ayush
> > > > >
> > > > > > On 23-Aug-2023, at 6:19 PM, Zoltan Haindrich 
> wrote:
> > > > > >
> > > > > > Hey Stamatis!
> > > > > >
> > > > > > I'm happy to donate these repos / help with the migration!
> > > > > > I should have done it earlier - but it was never top
> priority...thank
> > > > > you for initiating it!
> > > > > >
> > > > > > cheers,
> > > > > > Zoltan
> > > > > >
> > > > > >> On 8/23/23 14:00, Stamatis Zampetakis wrote:
> > > > > >> Hi all,
> > > > > >> Our precommit infrastructure uses code that resides in the
> following
> > > > > repos.
> > > > > >> * https://github.com/kgyrtkirk/hive-test-kube
> > > > > >> * https://github.com/kgyrtkirk/hive-toolbox
> > > > > >> * https://github.com/kgyrtkirk/hive-dev-box
> > > > > >> These are mainly maintained by Zoltán Haindrich who is always
> helpful
> > > > > >> and kind to investigate and resolve issues.
> > > > > >> For facilitating contributions from the apache community and
> also
> > > > > >> removing some burden from Zoltan's shoulders it may be a good
> time to
> > > > > >> migrate those and put them under the apache namespace.
> > > > > >> For the initial migration, we could have a straightforward 1 to
> 1
> > > > > >> mapping as shown below:
> > > > > >> * https://github.com/apache/hive-test-kube
> > > > > >> * https://github.com/apache/hive-toolbox
> > > > > >> * https://github.com/apache/hive-dev-box
> > > > > >> How do you feel about this?
> > > > > >> Best,
> > > > > >> Stamatis
> > > > >
>

[Discuss] Enable Attachments for Hive mailing lists

[Ayush Saxena]

Hi All,
As of now we don't allow having attachments on the hive mailing lists
(apart from security ML), This prevents us from attaching patches/design
doc or even screenshots of issues being reported on our mailing lists.

A lot of projects allow that, I feel we should enable this for our Hive
mailing lists as well for better dev experience.

Let me know your thoughts!!!

Obviously a +1 from me

-Ayush

Subscribe to security ML (Hive Committers)

[Ayush Saxena]

Hi Folks,
In case any of the committers or PMC members are not subscribed to the
security mailing list, please subscribe & help address or share pointers if
you can.

if you are a committer, please send a mail to:
security-subscr...@hive.apache.org

This mail list is moderated, so the request needs to be approved, so please
send a request from an email id which the moderator (like me) can identify.

If you have any issues with the process, do reach out to me or any other
moderator.

-Ayush

Re: Cleanup remote feature/wip branches

[Ayush Saxena]

+1

-Ayush 

> On 19-Jan-2024, at 3:41 PM, Stamatis Zampetakis  wrote:
> 
> Hey everyone,
> 
> I noticed that in our official git repo [1] we have some kind of
> feature/WIP branches (see list below). Most of them (if not all) are
> stale, add noise, and some of them eat CI resources (storage and CPU)
> since Jenkins picks them up for builds/precommits.
> 
> I would like to drop those at the end of this email. Please +1 if you agree.
> 
> Best,
> Stamatis
> 
> [1] https://github.com/apache/hive/branches/all
> 
> git branch -r | grep origin | grep -v "branch-" | grep -v "master"
>  origin/HIVE-23274_280_rb
>  origin/HIVE-23337_280_rb
>  origin/HIVE-23403_280_rb
>  origin/HIVE-23440_280_rb
>  origin/HIVE-23470_rb
>  origin/HIVE-4115
>  origin/branc-2.3
>  origin/cbo
>  origin/dependabot/maven/com.google.protobuf-protobuf-java-3.21.7
>  
> origin/dependabot/maven/itests/qtest-druid/org.eclipse.jetty-jetty-server-9.4.51.v20230217
>  origin/dependabot/maven/org.apache.commons-commons-text-1.10.0
>  origin/dependabot/maven/org.eclipse.jetty-jetty-server-9.4.51.v20230217
>  origin/dependabot/maven/org.postgresql-postgresql-42.4.3
>  
> origin/dependabot/maven/standalone-metastore/com.google.protobuf-protobuf-java-3.21.7
>  
> origin/dependabot/maven/standalone-metastore/org.eclipse.jetty-jetty-server-9.4.51.v20230217
>  origin/dependabot/maven/standalone-metastore/org.postgresql-postgresql-42.4.3
>  origin/ptf-windowing
>  origin/release-1.1
>  origin/revert-1365-upgrade-guava
>  origin/revert-1855-HIVE-24624
>  origin/revert-2694-HIVE-25355
>  origin/revert-3624-HIVE-26567
>  origin/revert-4247-hive-23256
>  origin/revert-4306-HIVE-27330
>  origin/revert-4452-HIVE-57988-BetweenBugFix
>  origin/revert-4501-OptimizeGetPartitionAPI
>  origin/vectorization

Re: [EXTERNAL] Re: [DISCUSS] End of life for Hive 1.x, 2.x, 3.x

[Ayush Saxena]

Thanx everyone for the feedback, I have started a formal thread to mark 1.x
EOL. We can have one last release for 2.x as Chao mentioned, with some
required changes + our CVE's & get the release line marked as EOL then.

@Chao Sun  Do let us know if you have a proposed
timeline for that.

-Ayush

On Wed, 17 Jan 2024 at 08:23, vihang karajgaonkar 
wrote:

> I was confused about the subject line since it says 3.x as well along with
> 1.x and 2.x. Does this discussion include all 1.x, 2.x and 3.x or just 1.x
> and 2.x?
>
> I think it makes sense to EOL 1.x. Looks like 2.x is still being maintained
> by Chao and I think we were backporting PRs to the 3.x line pretty recently
> so I believe we should wait out for a release on Hive 3.x.
>
> Thanks,
> Vihang
>
> On Tue, Jan 16, 2024 at 3:40 PM Attila Turoczy
>  wrote:
>
> > Dear PMC's,
> >
> > Do we have a verdict / decision about this?
> >
> > -Attila
> >
> > On Wed, Jan 10, 2024 at 5:45 PM Chao Sun  wrote:
> >
> > > On Hive 2.x, I'm still preparing for another release 2.3.10 (Hive 2.3
> > > branch is being actively maintained so far). Hopefully this will be
> > > the last release in the branch-2 line.
> > >
> > > +1 on making Hive 1 EOL for the time being.
> > >
> > > Chao
> > >
> > > On Wed, Jan 10, 2024 at 8:10 AM Sankar Hariappan
> > >  wrote:
> > > >
> > > > +1 for making both Hive 1&2 EOL
> > > >
> > > > -Sankar
> > > > -Original Message-
> > > > From: Attila Turoczy 
> > > > Sent: Wednesday, January 10, 2024 7:37 PM
> > > > To: dev@hive.apache.org
> > > > Subject: [EXTERNAL] Re: [DISCUSS] End of life for Hive 1.x, 2.x, 3.x
> > > >
> > > > [You don't often get email from aturo...@cloudera.com.invalid. Learn
> > > why this is important at https://aka.ms/LearnAboutSenderIdentification
> ]
> > > >
> > > > +1 for making it EOL for Hive 1 and Hive 2. I do not think these 2
> > > product
> > > > branches are relevant in 2023.
> > > >
> > > > -Attila
> > > >
> > > > On Wed, Jan 10, 2024 at 12:59 PM Denys Kuzmenko <
> dkuzme...@apache.org>
> > > > wrote:
> > > >
> > > > > +1 for marking Hive 1.x EOL
> > > > >
> > > > > Assuming no volunteers willing to take ownership of branch-2
> > > maintenance,
> > > > > +1 to declare it EOL as well.
> > > > >
> > > > > Regards,
> > > > > Denys
> > > > >
> > >
> >
>

[VOTE] Mark Hive 1.x EOL

[Ayush Saxena]

Hi All,
Following the discussion in [1], Starting an official thread to mark Hive
1.x EOL.

Marking a release line EOL, means there won't be any further releases for
that release line.

I will start with my +1

-Ayush

[1] https://lists.apache.org/thread/sxcrcf4v9j630tl9domp0bn4m33bdq0s

Re: [EXTERNAL] Re: [DISCUSS] Deprecate/Drop upgrade-acid module from 4.x

[Ayush Saxena]

Thanx Everyone for the feedback. I have created
https://issues.apache.org/jira/browse/HIVE-28005 for the removal of this
module from master branch.

-Ayush

On Wed, 10 Jan 2024 at 21:42, Sankar Hariappan
 wrote:

> +1, It is needed only for upgrading from Hive 1.x/2.x to 3.x and not
> relevant for 4.x. We can retain it in 3.x release line only.
>
> -Sankar
>
> -Original Message-
> From: Attila Turoczy 
> Sent: Wednesday, January 10, 2024 7:04 PM
> To: dev@hive.apache.org
> Subject: [EXTERNAL] Re: [DISCUSS] Deprecate/Drop upgrade-acid module from
> 4.x
>
> [You don't often get email from aturo...@cloudera.com.invalid. Learn why
> this is important at https://aka.ms/LearnAboutSenderIdentification ]
>
> Big +1 from me. As we shift our focus from ACID to Iceberg I do not think
> it is relevant anymore. Also as Butao highlighted it has a CVE as well.
> Let's remove it, and if eventually something is needed (highly doubt) then
> we can revisit the decision at that time.
> Due to the extensive history of Hive and the numerous legacy components
> that haven't been touched since 1972, it is crucial for us all to be more
> decisive in determining what to keep and maintain. The size of the codebase
> makes it extremely challenging, time-consuming, and potentially frustrating
> for OSS contributors to thoroughly review all 67 (just a number :) )
> aspects of the Hive.
>
> -Attila
>
> On Wed, Jan 10, 2024 at 2:55 AM Butao Zhang  wrote:
>
> > +1. I am not sure the use case of upgrade-acid module, but it seems
> > +that
> > this module is rarely&never used in my world. I think maybe the first
> > safe step is deprecating this module to let users&dev know that this
> > module should not be used any more.
> >
> > BTW, my idea tells me that this module used the old Hive2.3.3 which
> > has some vulnerability. Should we consider upgrading this dependency to
> hive4?
> > :
> > "Dependency maven:org.apache.hive:hive-metastore:2.3.3 is vulnerable,
> > safe version 4.0.0-alpha-2"
> > CVE-2021-34538 7.5 Missing Authentication for Critical Function
> > vulnerability
> >
> >
> >
> > Thanks,
> > Butao Zhang
> >  Replied Message 
> > From Ayush Saxena  Date
> > 1/10/2024 07:45 To dev 
> > Subject [DISCUSS] Deprecate/Drop upgrade-acid module from 4.x Hi
> > Folks, Wanted to know thoughts on removing the upgrade-acid module[1]
> > from 4.x. The javadoc on one of the main files[2] read "This utility
> > is designed to help with upgrading Hive 2.x to Hive 3.0". I think this
> > is a 2.x to 3.x thing and doesn't look relevant for Hive-4.x. Checking
> > the git log, I don't find any relevant development happening on this
> > either.
> >
> > The main challenge that this brings is that it depends on legacy
> > Hive(2.3.3) & Hadoop(2.7.2) [3], which aren't JDK-11 compliant & it
> > blocks the way for Hive JDK-11 compile time support.
> >
> > Let me know your thoughts!!!
> >
> > -Ayush
> >
> > [1]
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> > ub.com%2Fapache%2Fhive%2Ftree%2Fmaster%2Fupgrade-acid&data=05%7C02%7CS
> > ankar.Hariappan%40microsoft.com%7C2b9cb6396ced4d0e629e08dc11e1125c%7C7
> > 2f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638404905616518454%7CUnknown
> > %7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJ
> > XVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=5VD1wpffcaTp6fI0a0q5Svtjn1SHWWKBQZSd
> > fF0BbcY%3D&reserved=0
> > [2]
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> > ub.com%2Fapache%2Fhive%2Fblob%2Fmaster%2Fupgrade-acid%2Fpre-upgrade%2F
> > src%2Fmain%2Fjava%2Forg%2Fapache%2Fhadoop%2Fhive%2Fupgrade%2Facid%2FPr
> > eUpgradeTool.java%23L86C4-L86C72&data=05%7C02%7CSankar.Hariappan%40mic
> > rosoft.com%7C2b9cb6396ced4d0e629e08dc11e1125c%7C72f988bf86f141af91ab2d
> > 7cd011db47%7C1%7C0%7C638404905616525910%7CUnknown%7CTWFpbGZsb3d8eyJWIj
> > oiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C
> > %7C%7C&sdata=lBMFnBrWsSMR3y%2FPlCzjz3ZuJR7jwOq5uJ91EGTEpTs%3D&reserved
> > =0
> > [3]
> > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith
> > ub.com%2Fapache%2Fhive%2Fblob%2Fmaster%2Fupgrade-acid%2Fpre-upgrade%2F
> > pom.xml%23L38-L39&data=05%7C02%7CSankar.Hariappan%40microsoft.com%7C2b
> > 9cb6396ced4d0e629e08dc11e1125c%7C72f988bf86f141af91ab2d7cd011db47%7C1%
> > 7C0%7C638404905616532153%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiL
> > CJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=Su
> > 9HZ%2FwUq%2B6Rel5RR8p2%2BFhJz4Jf3HemTkulKsIcH4k%3D&reserved=0
> >
>

Re: [DISCUSS] End of life for Hive 1.x, 2.x, 3.x

[Ayush Saxena]

I will start a vote to mark Hive 1.x EOL next week. Let me know if
anyone has concerns around it.

The main reason to mark a release line EOL is: if we have a CVE & if
we don't release all the active lines with the fix we can't announce
that & the PMC would be flagged every quarter for delaying the
process, So, sooner or later we need to find a way to reduce the
number of active release lines.

-Ayush

On Fri, 29 Jul 2022 at 01:35, Chao Sun  wrote:
>
> Hive 2.x is still being used by other projects like Spark and Iceberg,
> and periodically there are bug fixes & CVE fixes coming into the
> branch. So I would suggest keeping it alive for a bit longer (maybe
> after 2.3.10/11 release) until the other projects are ready to move
> away from it (which could take some significant efforts).
>
> Chao
>
> On Thu, Jul 28, 2022 at 5:51 AM Ayush Saxena  wrote:
> >
> > +1, to start EOL vote for 1.x, and we can keep a doc or a reference in the
> > Hive Wiki/Website to mark the lines EOL
> >
> > Sharing thoughts about the other release lines.
> > Though there were assertions that we have a lot of users on 2.x & 3.x
> > lines, I don't think marking these lines as  EOL will impact them that
> > badly.
> > Marking a release line seems to be a Dev agreement that we as the
> > developers aren't putting enough efforts now maintaining these branches and
> > they aren't very up to date.
> >
> > Quoting the example from Hadoop. Hadoop 3.1.x line is marked as EOL and
> > still almost every second person on Hadoop 3.x line is on a heavily patched
> > version of 3.1.x, and from the other half still a bunch of them are on 2.x
> > family, out of which only 2.10.x isn't EOL. Side note: As of today Hive in
> > master branch also depends on an unstable EOL version of hadoop, that is
> > 3.1.0(Upgrade in progress)
> >
> > From the stability point of view, I agree with Stamatis that 4.x in alpha
> > stage is still better than a bunch of previous releases in many aspects,
> > and supporting older releases will just slow down the chances of
> > adaptability of the new 4.x.
> > If we see the git history even of these old branches, the frequency of
> > commits are even too low, so I don't think most of the
> > developers/committers aren't putting efforts maintaining these
> > branches.(Subjective Opinion)
> >
> > IMO, We should consider marking 1.x & 2.x as EOL, Resolve upgrade issues
> > mentioned for 3.x->4.x and once resolved, if that doesn't require any
> > changes on 3.x line and everyone is happy then mark that even as EOL or
> > else have a last bridge release for this branch to move to 4.x
> >
> > Just my 2 cents.
> >
> > -Ayush
> >
> >
> >
> > On Mon, 25 Jul 2022 at 19:38, Stamatis Zampetakis  wrote:
> >
> > > Hi all,
> > >
> > > In the last exchanges there was a general consensus to EOL Hive 1.X but no
> > > additional action.
> > > I believe the next step would be to start a VOTE and move forward with an
> > > official announcement.
> > >
> > > I think it would be helpful for the end-users to know which releases are
> > > supported and which are strongly discouraged.
> > > The Hadoop community keeps this information in their wiki [1].
> > >
> > > Although, I am still not convinced that we should encourage users to use
> > > the older release lines (2.X, 3.X) we can postpone the decision for the
> > > time being and proceed just for 1.X.
> > >
> > > Best,
> > > Stamatis
> > >
> > > [1]
> > >
> > > https://cwiki.apache.org/confluence/display/HADOOP/EOL+%28End-of-life%29+Release+Branches
> > >
> > > On Tue, May 10, 2022 at 2:51 PM Stamatis Zampetakis 
> > > wrote:
> > >
> > > > Thanks everyone for sharing your thoughts. I am happy to see so many
> > > > people involved in the discussion.
> > > >
> > > > I would say that the current 4.0.0-alpha-1 is better in many aspects 
> > > > than
> > > > previous stable releases, although this might be a bit subjective.
> > > >
> > > > I am afraid that if we keep supporting older releases it will take too
> > > > much time till people start using the 4.x.
> > > > Having real deployments of Hive 4 is the only way to go from alpha to
> > > > stable releases with confidence.
> > > >
> > > > I checked the download statistics for Hive releases [1], [2] for the 
> > > > past
>

[DISCUSS] Deprecate/Drop upgrade-acid module from 4.x

[Ayush Saxena]

Hi Folks,
Wanted to know thoughts on removing the upgrade-acid module[1] from
4.x. The javadoc on one of the main files[2] read "This utility is
designed to help with upgrading Hive 2.x to Hive 3.0". I think this is
a 2.x to 3.x thing and doesn't look relevant for Hive-4.x. Checking
the git log, I don't find any relevant development happening on this
either.

The main challenge that this brings is that it depends on legacy
Hive(2.3.3) & Hadoop(2.7.2) [3], which aren't JDK-11 compliant & it
blocks the way for Hive JDK-11 compile time support.

Let me know your thoughts!!!

-Ayush

[1] https://github.com/apache/hive/tree/master/upgrade-acid
[2] 
https://github.com/apache/hive/blob/master/upgrade-acid/pre-upgrade/src/main/java/org/apache/hadoop/hive/upgrade/acid/PreUpgradeTool.java#L86C4-L86C72
[3] 
https://github.com/apache/hive/blob/master/upgrade-acid/pre-upgrade/pom.xml#L38-L39

Re: 4.0 documentation - Confluence limitations?

[Ayush Saxena]

When we discussed last time, all of us were in favour and wanted to
have documentation as part of our website per version, but we realised
that isn't very much chasable considering the number of volunteers we
have, so we decided to go with improving the existing wiki pages & add
new additions to those existing pages. I went a step ahead and rather
than just tweaking those pages, created a separate space for 4.0
documentation, not sure if we would be able to complete that either
but that is a work in progress and a stop gap solution till we get the
documentation properly on our website. needless to say I am +1 on
coining a proper framework for the documentation & to get them on our
website.

-Ayush

On Mon, 8 Jan 2024 at 15:52, Simhadri G  wrote:
>
> Hi Zsolt,
>
> The current hive website is built with hugo,  so +1 from me :)
>
> We do have a few doc pages written in hugo, example :
> https://hive.apache.org/developement/quickstart/
>
> To add a new page we will need to add a new markdown file in the correct
> location in the hive-site repo and hugo will render the same in the hive
> website.
> For reference , there is a readme section here on how to add new pages as
> well: https://github.com/apache/hive-site#to-add-new-content
> We can definitely change the formatting/style of docs as needed.
>
>
> Thanks!
> Simhadri G
>
> On Mon, Jan 8, 2024 at 3:04 PM Stamatis Zampetakis 
> wrote:
>
> > Hey Zsolt,
> >
> > There have been a few discussions in the past about moving the
> > documentation from the wiki to the website and from what I recall
> > people were more or less in favor of moving towards this direction.
> > The main thing missing is volunteers that are willing to take on this
> > migration step.
> >
> > Personally, I am very much in favor of going into this direction not
> > only for solving namespacing issues but also for traceability purposes
> > and facilitating doc contributions and reviews.
> >
> > Big +1 from me.
> >
> > Best,
> > Stamatis
> >
> > On Mon, Jan 8, 2024 at 10:15 AM Zsolt Miskolczi
> >  wrote:
> > >
> > > In confluence, page names should be unique in a given space. As I see,
> > > Apache Hive has its own space.
> > > And now comes the tricky part: with 4.0 documentation, we didn't create a
> > > new space, just a 4.0 parent page. We create a copy of existing pages
> > under
> > > the umbrella of this page:
> > > https://cwiki.apache.org/confluence/display/Hive/Apache+Hive+4.0.0
> > >
> > > The problem is the unique naming of pages: it would make sense to keep
> > the
> > > page names the same as in the older documents but unfortunately, we
> > cannot.
> > > So we try to create names that are almost the same, or just delay the
> > > decisions.
> > > Two examples:
> > > - AdminManual Installation
> > > <
> > https://cwiki.apache.org/confluence/display/Hive/AdminManual+Installation>
> > > became Manual Installation
> > > 
> > > - Hive Schema Tool
> > > became
> > Copy
> > > of Hive Schema Tool - [TODO: move it under a 4.0 admin manual page, find
> > a
> > > proper name]
> > > <
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=284790216
> > >
> > >
> > > I feel multiple issues with that: Consistency is gone. And also, I'm not
> > > sure how it can support search engines. Also, it can be confusing for
> > > people who want to use the wiki pages.
> > >
> > > I was thinking about different solutions. Creating a Hive 4.0 space in
> > > Confluence can solve the problem of page uniqueness. But doesn't address
> > > the issue of searchability and ease of use.
> > >
> > > We can also keep the current one but in that case, it would be
> > recommended
> > > to figure out a great naming convention about the pages.
> > >
> > > At this point, my best idea is to move to an engine that has better
> > offers
> > > to document a software product. For example, Iceberg uses Hugo. It is a
> > > markup-based engine, it can be kept in source control and pretty fast.
> > > Example page: https://iceberg.apache.org/docs/1.4.1/.
> > >
> > >
> > > What do you think of that?
> > >
> > > Thank you,
> > > Zsolt
> >

Re: Force coding style in hive precommit

[Ayush Saxena]

+1, to have a checkstyle build. I am strongly against doing that big
refactor to make just checkstyle happy, such a refactor will make
backports to Hive lower branches tough and the life of folks
maintaining downstream forks quite painful.

We should enforce same kind of stuff like in Tez/Hadoop, where
checkstyle violations are highlighted and the committer before
committing can check that & decide whether that in unavoidable or not

-Ayush

On Mon, 8 Jan 2024 at 14:05, László Bodor  wrote:
>
> thanks for the responses so far!
> I'm a bit against the one-time huge refactor commit as we don't need that
> (but I can be convinced of course), because checkstyle can be set up to
> warn only on style issues in the new/touched bits in the PR (or at least
> that's how it works in tez), that's what we need, so we don't have to make
> that huge commit to simply introduce this enforcement
>
> Butao Zhang  ezt írta (időpont: 2024. jan. 8., H,
> 9:28):
>
> > +1
> >
> >
> >
> > BTW, We have a independent checkstyle file under iceberg module
> > https://github.com/apache/hive/tree/master/iceberg/checkstyle . I think
> > we need to consider unifing the checkstyle in all the sub-module.
> >
> >
> > Thanks,
> > Butao Zhang
> >  Replied Message 
> > | From | Zsolt Miskolczi |
> > | Date | 1/8/2024 16:19 |
> > | To |  |
> > | Subject | Re: Force coding style in hive precommit |
> > +1
> >
> > In case there is an agreement about the coding style, we can prepare a tool
> > that enforces that style at compile time. Run a tool one time to re-format
> > all the existing code once. And turn on a compile time check. Iceberg did
> > the same approach, they had one huge commit with almost 4k files changed
> > and from that point, it worked well. And there are no issues about
> > formatting.
> > I don't think putting a warning message helps at all. Also, it should be
> > enforced on compile time.
> >
> > Zsolt
> >
> > Kirti Ruge  ezt írta (időpont: 2024. jan. 8., H,
> > 7:20):
> >
> > +1
> > As it would improve maintainability and code reviews. Sometimes small
> > indentation/styling issues would kill review cycle time and we can easily
> > avoid it before requesting review.
> > Enforcing more rules around it definitely boost guaranteeing quality. We
> > can integrate it with git hooks. If we are going for this, I can work on
> > getting it in place .
> >
> > Thanks,
> > Kirti
> >
> > On 08-Jan-2024, at 11:36 AM, Akshat m  wrote:
> >
> > +1, We do have a documentation round it as well:
> >
> >
> > https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-CodingConventions
> > so it makes sense to enforce it as well.
> >
> > Right now we have a small section around this in documentation, We can
> > also
> > expand this to a new page and add more Java practices to it as well which
> > are followed in the project while we are at this, Will be a great
> > addition
> > to Hive 4 documentation, I can pick it up.
> >
> > I suggest we add this style check as a pre-commit git hook as well, so it
> > is enforced when the author is committing locally as well, this can save
> > the wait time for pre-commit failure in the PR for the author to realise
> > the styling issues, ideally this should be taken care of with the ide
> > style
> > configuration but in case we miss it this would error out while
> > committing the changes.
> >
> > Regards,
> > Akshat
> >
> > On Sat, Jan 6, 2024 at 10:17 AM László Bodor 
> > wrote:
> >
> > Hi All!
> >
> > What do you think about forcing coding style in Hive precommit?
> >
> > I remember, back in the old days, precommit printed some warnings in
> > case
> > some coding style (formatting, indentation, naming convention, etc.)
> > problems were found in the patch, now it's simply not used, I guess
> > since
> > we're using GitHub PRs.
> >
> > For example: I remember I simply approved a PR a few months ago which
> > LGTM, and later just realized it's full of 4-spaces indentation, which
> > is
> > wrong if we assume that code should be formatted according to the style
> > definition here:
> >
> > https://github.com/apache/hive/blob/master/dev-support/eclipse-styles.xml
> >
> > I have just attached an example of Tez PR to open minds and start a
> > conversation.
> >
> > Regards,
> > Laszlo Bodor
> >
> >
> >
> >
> >

Re: Help with Docker Apache/Hive metastore using mysql remote database

[Ayush Saxena]

I think the similar problem is being chased as part of
https://github.com/apache/hive/pull/4948

On Mon, 18 Dec 2023 at 09:48, Sanjay Gupta  wrote:
>
>
>
>
> Issue with Docker container using mysql RDBMS ( Failed to load driver)
>
> https://hub.docker.com/r/apache/hive
>
> According to readme
>
> Launch Standalone Metastore With External RDBMS (Postgres/Oracle/MySql/MsSql)
>
> I want to use MySQL
>
> I tried com.mysql.jdbc.Driver or com.mysql.cj.jdbc.Driver
>
> docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore 
> --add-host=host.docker.internal:host-gateway \
>  --env DB_DRIVER=mysql \
>  --env 
> SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.jdbc.Driver 
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
>  -Djavax.jdo.option.ConnectionUserName=hive 
> -Djavax.jdo.option.ConnectionPassword=password" \
>  --mount source=warehouse,target=/opt/hive/data/warehouse \
>  --name metastore-standalone apache/hive:${HIVE_VERSION}
>
>
> docker run -it -d -p 9083:9083 --env SERVICE_NAME=metastore 
> --add-host=host.docker.internal:host-gateway \
>  --env DB_DRIVER=mysql \
>  --env 
> SERVICE_OPTS="-Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver
>   
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
>  -Djavax.jdo.option.ConnectionUserName=hive 
> -Djavax.jdo.option.ConnectionPassword=password" \
>  --mount source=warehouse,target=/opt/hive/data/warehouse \
>  --name metastore-standalone apache/hive:${HIVE_VERSION}
>
> Docker logs shows this for both drivers ( same error )
>
> docker logs f3
> + : mysql
> + SKIP_SCHEMA_INIT=false
> + export HIVE_CONF_DIR=/opt/hive/conf
> + HIVE_CONF_DIR=/opt/hive/conf
> + '[' -d '' ']'
> + export 'HADOOP_CLIENT_OPTS= -Xmx1G 
> -Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver  
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
>  -Djavax.jdo.option.ConnectionUserName=hive 
> -Djavax.jdo.option.ConnectionPassword=hive'
> + HADOOP_CLIENT_OPTS=' -Xmx1G 
> -Djavax.jdo.option.ConnectionDriverName=com.mysql.cj.jdbc.Driver  
> -Djavax.jdo.option.ConnectionURL=jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
>  -Djavax.jdo.option.ConnectionUserName=hive 
> -Djavax.jdo.option.ConnectionPassword=hive'
> + [[ false == \f\a\l\s\e ]]
> + initialize_hive
> + /opt/hive/bin/schematool -dbType mysql -initSchema
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Metastore connection URL: 
> jdbc:mysql://host.docker.internal:3306/hive?createDatabaseIfNotExist=true
> Metastore Connection Driver : com.mysql.cj.jdbc.Driver
> Metastore connection User: hive
> org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
> Underlying cause: java.lang.ClassNotFoundException : com.mysql.cj.jdbc.Driver
> Use --verbose for detailed stacktrace.
> *** schemaTool failed ***
> + '[' 1 -eq 0 ']'
> + echo 'Schema initialization failed!'
> Schema initialization failed!
> + exit 1
>
> Any idea, why I am getting failed to load driver for MySQL DB.
>
> Isn't docker container comes with MySQL Driver ?
>
> Docker container exits so I can't check whether driver is already installed.
>
> Let me know, what I can do to make it work.
>
> --
>
>
> Thanks
> Sanjay Gupta
>
>
>
> --
>
> Thanks
> Sanjay Gupta
>
>
>
> --
>
> Thanks
> Sanjay Gupta
>

Re: Flaky test testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites

[Ayush Saxena]

Thanx Butao Zhang for flagging this, As I said on the PR, If the test
is flaky we can disable that for sure, just create a ticket with all
the relevant logs attached around the test, so that anyone can chase
that in future.

-Ayush

On Fri, 8 Dec 2023 at 20:48, Butao Zhang  wrote:
>
> Hi Hive dev,
>
>
> I find the test testBootstrapAcidTablesDuringIncrementalWithConcurrentWrites 
> often fails in CI. We had to rerun the total CI because of the single flaky 
> test, and this wastes development time and CI resoures.
>
> I raised a ticket https://issues.apache.org/jira/browse/HIVE-27936 to track 
> this flay test and found it failed in many PR tests. Also, the flaky report 
> shows that this test is flaky, see 
> http://ci.hive.apache.org/job/hive-flaky-check/756/testReport/.
> If we don't have good way to fix the flaky test, i think we should disable it.
>
>
> Thanks,
> Butao Zhang

Re: Fix Version is now mandatory in Jira

[Ayush Saxena]

Hi Pravin,
The fix version for master is 4.1.0 now. It is being tracked under [1] & [2]
I have created the version 4.1.0 in Jira, so you should be able to
resolve the tickets with it now.

-Ayush

[1] https://issues.apache.org/jira/browse/HIVE-27928
[2] https://github.com/apache/hive/pull/4914

On Mon, 4 Dec 2023 at 22:57, Pravin sinha  wrote:
>
> Hi Ayush
>
> Thanks for the update. Given that the branch for 4.0.0 is cut, for
> resolving jira in master branch what is the "fix version" decided to be
> given? Do we already have a place-holder created for the version next to
> 4.0.0?
>  Asking this anticipating that not everything from master will be cherry
> picked to branch-4.0 <https://github.com/apache/hive/tree/branch-4.0>.
>
> Thanks,
> Pravin
>
> On Thu, Nov 23, 2023 at 8:00 PM Ayush Saxena  wrote:
>
> > Hi All,
> > Following INFRA-24974, now for any Hive ticket, it is mandatory to
> > provide the Fix Version, if not it won't let you resolve the ticket.
> >
> > Fix Version is a mandatory column which is used to populate the
> > release notes, So, please put the correct fix version while resolving
> > the tickets.
> >
> > If you aren't sure, just check the POM for the hive version in the
> > branch where the code was merged.
> >
> > Let me know if there are any issues. In case the code isn't merged to
> > any of the release branches, or if it is an Invalid or Dupe ticket,
> > please add Not Applicable as the fix version & resolve.
> >
> > PS. If you want to give it a try, can try on
> > https://issues.apache.org/jira/browse/HIVE-27909, I created that to
> > try myself :-)
> >
> > -Ayush
> >

Fix Version is now mandatory in Jira

[Ayush Saxena]

Hi All,
Following INFRA-24974, now for any Hive ticket, it is mandatory to
provide the Fix Version, if not it won't let you resolve the ticket.

Fix Version is a mandatory column which is used to populate the
release notes, So, please put the correct fix version while resolving
the tickets.

If you aren't sure, just check the POM for the hive version in the
branch where the code was merged.

Let me know if there are any issues. In case the code isn't merged to
any of the release branches, or if it is an Invalid or Dupe ticket,
please add Not Applicable as the fix version & resolve.

PS. If you want to give it a try, can try on
https://issues.apache.org/jira/browse/HIVE-27909, I created that to
try myself :-)

-Ayush

[ANNOUNCE] New committer: Butao Zhang (zhangbutao)

[Ayush Saxena]

Hi All,
Apache Hive's Project Management Committee (PMC) has invited Butao
Zhang  to become a committer, and we are pleased to announce that he
has accepted.

Butao Zhang welcome, thank you for your contributions, and we look
forward to your further interactions with the community!

Ayush Saxena
(On behalf of Apache Hive PMC)

Re: Re: Hive's performance for querying the Iceberg table is very poor.

[Ayush Saxena]

HIVE-27734 is in progress, as I see we have a POC attached to the ticket,
we should have it in 2-3 week I believe.

> Also, after the release of 4.0.0, will we be able to do all TPCDS queries
on ICEBERG except for normal HIVE tables?

Yep, I believe most of the TPCDS queries would be supported even today on
Hive master, but 4.0.0 would have them running for sure.

-Ayush

On Tue, 24 Oct 2023 at 14:51, lisoda  wrote:

> Thanks.
> I would like to know if hive currently supports push to ICEBERG table
> partition under JOIN condition.
> Because I see HIVE-27734 is not yet complete, what is its progress so
> far?
> Also, after the release of 4.0.0, will we be able to do all TPCDS queries
> on ICEBERG except for normal HIVE tables?
>
>
>
>
>
> 在 2023-10-24 11:03:07，"Ayush Saxena"  写道：
>
> Hi Lisoda,
>
> The iceberg jar for hive 3.1.3 doesn't have a lot of changes, We did a
> bunch of improvements on the 4.x line for Hive-Iceberg. You can give
> iceberg a try on the 4.0.0-beta-1 release mentioned here [1], we have a
> bunch of improvements like vecotrization and stuff like that. If you wanna
> give it a quick try on docker, we have docker image published for that here
> [2] & Iceberg works out of the box there.
>
> Rest feel free to create tickets, if you find some specific queries or
> scenarios which are problematic, we will be happy to chase them & get them
> sorted.
>
> PS. Not sure about StarRocks, FWIW. That is something we don't develop as
> part of Apache Hive nor as part of Apache Software Foundation to best of my
> knowledge, so would refrain from or commenting about that on "Apache Hive"
> ML
>
> -Ayush
>
>
> [1] https://hive.apache.org/general/downloads/
> [2] https://hub.docker.com/r/apache/hive/tags
>
> On Tue, 24 Oct 2023 at 05:28, Albert Wong 
> wrote:
>
>> Too bad.   Tencent Games used StarRocks with Apache Iceberg to power
>> their analytics.
>> https://medium.com/starrocks-engineering/tencent-games-inside-scoop-the-road-to-cloud-native-with-starrocks-d7dcb2438e25.
>>
>>
>> On Mon, Oct 23, 2023 at 10:55 AM lisoda  wrote:
>>
>>> We are not going to use starrocks.
>>> mpp architecture databases have natural limitations, and starrocks does
>>> not necessarily perform better than hive llap.
>>>
>>>
>>>  Replied Message 
>>> From Albert Wong 
>>> Date 10/24/2023 01:39
>>> To u...@hive.apache.org
>>> Cc
>>> Subject Re: Hive's performance for querying the Iceberg table is very
>>> poor.
>>> I would try http://starrocks.io.   StarRocks is an MPP OLAP database
>>> that can query Apache Iceberg and we can cache the data for faster
>>> performance.  We also have additional features like building materialized
>>> views that span across Apache Iceberg, Apache Hudi and Apache Hive.   Here
>>> is a video of connecting the 2 products through a webinar StarRocks did
>>> with Tabular (authors of Apache Iceberg).
>>> https://www.youtube.com/watch?v=bAmcTrX7hCI&t=10s
>>>
>>> On Mon, Oct 23, 2023 at 7:18 AM lisoda  wrote:
>>>
>>>> Hi Team.
>>>>   I recently was testing Hive query Iceberg table , I found that
>>>> Hive query Iceberg table performance is very very poor . Almost impossible
>>>> to use in the production environment . And Join conditions can not be
>>>> pushed down to the Iceberg partition.
>>>>   I'm using the 1.3.1 Hive Runtime Jar from the Iceberg community.
>>>>   Currently I'm using Hive 3.1.3, Iceberg 1.3.1.
>>>>   Now I'm very frustrated because the performance is so bad that I
>>>> can't deliver to my customers. How can I solve this problem?
>>>>  Details:
>>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1695050248606629
>>>> I would be grateful if someone could guide me.
>>>>
>>>

Re: Hive's performance for querying the Iceberg table is very poor.

[Ayush Saxena]

Hi Lisoda,

The iceberg jar for hive 3.1.3 doesn't have a lot of changes, We did a
bunch of improvements on the 4.x line for Hive-Iceberg. You can give
iceberg a try on the 4.0.0-beta-1 release mentioned here [1], we have a
bunch of improvements like vecotrization and stuff like that. If you wanna
give it a quick try on docker, we have docker image published for that here
[2] & Iceberg works out of the box there.

Rest feel free to create tickets, if you find some specific queries or
scenarios which are problematic, we will be happy to chase them & get them
sorted.

PS. Not sure about StarRocks, FWIW. That is something we don't develop as
part of Apache Hive nor as part of Apache Software Foundation to best of my
knowledge, so would refrain from or commenting about that on "Apache Hive"
ML

-Ayush


[1] https://hive.apache.org/general/downloads/
[2] https://hub.docker.com/r/apache/hive/tags

On Tue, 24 Oct 2023 at 05:28, Albert Wong  wrote:

> Too bad.   Tencent Games used StarRocks with Apache Iceberg to power their
> analytics.
> https://medium.com/starrocks-engineering/tencent-games-inside-scoop-the-road-to-cloud-native-with-starrocks-d7dcb2438e25.
>
>
> On Mon, Oct 23, 2023 at 10:55 AM lisoda  wrote:
>
>> We are not going to use starrocks.
>> mpp architecture databases have natural limitations, and starrocks does
>> not necessarily perform better than hive llap.
>>
>>
>>  Replied Message 
>> From Albert Wong 
>> Date 10/24/2023 01:39
>> To u...@hive.apache.org
>> Cc
>> Subject Re: Hive's performance for querying the Iceberg table is very
>> poor.
>> I would try http://starrocks.io.   StarRocks is an MPP OLAP database
>> that can query Apache Iceberg and we can cache the data for faster
>> performance.  We also have additional features like building materialized
>> views that span across Apache Iceberg, Apache Hudi and Apache Hive.   Here
>> is a video of connecting the 2 products through a webinar StarRocks did
>> with Tabular (authors of Apache Iceberg).
>> https://www.youtube.com/watch?v=bAmcTrX7hCI&t=10s
>>
>> On Mon, Oct 23, 2023 at 7:18 AM lisoda  wrote:
>>
>>> Hi Team.
>>>   I recently was testing Hive query Iceberg table , I found that
>>> Hive query Iceberg table performance is very very poor . Almost impossible
>>> to use in the production environment . And Join conditions can not be
>>> pushed down to the Iceberg partition.
>>>   I'm using the 1.3.1 Hive Runtime Jar from the Iceberg community.
>>>   Currently I'm using Hive 3.1.3, Iceberg 1.3.1.
>>>   Now I'm very frustrated because the performance is so bad that I
>>> can't deliver to my customers. How can I solve this problem?
>>>  Details:
>>> https://apache-iceberg.slack.com/archives/C025PH0G1D4/p1695050248606629
>>> I would be grateful if someone could guide me.
>>>
>>

Re: Update pmc page

[Ayush Saxena]

Hi Edward,
I don't know who did that, it has been there since long but I am
pretty sure the intentions won't be trolling or hurting you in any
way.
I have updated the website. Can check once, If there are still issues,
Let me know
In case you want to add your current organisation or anybody following
this ML wants to do that, can play with the file as in the commit [1]

-Ayush

[1] 
https://github.com/apache/hive-site/commit/10d09ce2c2a807891226e43573429f72fc03fe26

On Thu, 19 Oct 2023 at 07:27, Edward Capriolo  wrote:
>
> https://hive.apache.org/community/people/
>
> On Wednesday, October 18, 2023, Edward Capriolo 
> wrote:
>
> > Don't know if you guys are trolling me or not but I dont currently nor
> > ever worked at hortonwoks lol
> >
> > --
> > Sorry this was sent from mobile. Will do less grammar and spell check than
> > usual.
> >
>
>
> --
> Sorry this was sent from mobile. Will do less grammar and spell check than
> usual.

Re: [EXTERNAL] Re: [ANNOUNCE] New committer: Sourabh Badhya

[Ayush Saxena]

Congratulations Sourabh!!!

-Ayush

> On 04-Oct-2023, at 9:28 AM, Sankar Hariappan 
>  wrote:
> 
> Congratulations Sourabh! Welcome to the Hive committers club! 😊
> 
> 
> 
> Thanks,
> 
> Sankar
> 
> 
> 
> -Original Message-
> From: Sourabh Badhya 
> Sent: Wednesday, October 4, 2023 9:19 AM
> To: dev@hive.apache.org; u...@hive.apache.org
> Subject: [EXTERNAL] Re: [ANNOUNCE] New committer: Sourabh Badhya
> 
> 
> 
> [You don't often get email from 
> sbad...@cloudera.com.invalid. Learn why 
> this is important at https://aka.ms/LearnAboutSenderIdentification ]
> 
> 
> 
> Thank you to the PMC members, committers and everyone who have helped me with 
> their advice and reviews. It's been a pleasure working on Hive over the past 
> couple of years. I hope to contribute and collaborate more for the project in 
> the future.
> 
> 
> 
> About me: I am working at Cloudera for the past 2 years, mainly engaged in 
> Apache Hive and related products. My current focus is on Iceberg support, 
> however I have had the opportunity to work on other areas of Hive such as 
> ACID compaction, optimising writes and related improvements.
> 
> 
> 
> Regards,
> 
> Sourabh Badhya
> 
> 
> 
>> On Tue, Oct 3, 2023 at 2:22 PM Stamatis Zampetakis 
>> mailto:zabe...@gmail.com>>
>> 
>> wrote:
>> 
>> 
>> 
>> Apache Hive's Project Management Committee (PMC) has invited Sourabh
> 
>> Badhya to become a committer, and we are pleased to announce that he
> 
>> has accepted.
> 
>> 
> 
>> Sourabh has been doing some great work for the project. He has landed
> 
>> important fixes in critical parts of Hive and made significant
> 
>> contributions to the stabilization of ACID compactions, Direct Write
> 
>> functionality, and Iceberg support. Apart from code contributions,
> 
>> Sourabh has been regularly reviewing others' work and providing
> 
>> valuable feedback as well as testing and validating releases.
> 
>> 
> 
>> Sourabh, welcome, thank you for your contributions, and we look
> 
>> forward to your further interactions with the community! If you wish,
> 
>> please feel free to tell us more about yourself and what you are
> 
>> working on.
> 
>> 
> 
>> Stamatis (on behalf of the Apache Hive PMC)
> 
>> 

Re: Request write access to the Hive wiki.

[Ayush Saxena]

I gave you edit permissions for the doc you mentioned, give a check if it
works for you

-Ayush

On Fri, 22 Sept 2023 at 01:00, Albert Wong 
wrote:

> In https://cwiki.apache.org/confluence/display/Hive/ on "user
> documentation", I'd like to add "StarRocks Integration".   StarRocks is an
> OLAP database that can query data in Apache Hive (
> https://docs.starrocks.io/en-us/latest/data_source/catalog/hive_catalog).
>
> On Thu, Sep 21, 2023 at 12:23 PM Ayush Saxena  wrote:
>
>> Hi Albert,
>>
>> Can you share some more details like which page you want to modify and
>> details around the content
>>
>> -Ayush
>>
>> On 22-Sep-2023, at 12:43 AM, Albert Wong 
>> wrote:
>>
>> 
>> username is albertatcelerdata.com
>>
>> --
>> [image: linkedin] <https://linkedin.com/in/atwong>
>> Albert Wong
>>
>> Community, Developer Relations, Technology Partnerships for
>> StarRocks | CelerData
>> [image: mobilePhone] 949 689 6412
>> [image: emailAddress] albert.w...@celerdata.com
>>
>>

Re: Inquiry about Stable Release Timeline for Hive-Serde 4.X

[Ayush Saxena]

Hi,

The GA release is in planning stage. We have some blockers, once we
get them sorted, we will be pushing for a new release.

At best it would take a minimum of 3 months, though that ain't a
strict timeline...

Thanx

-Ayush

On Thu, 21 Sept 2023 at 17:39, Mergu Ravi
 wrote:
>
> I'm currently using the hive-serde and its dependent libraries in my
> project. I'm interested in utilizing the feature introduced in version
> 4.0.0-beta1, specifically, the feature where INT64 Parquet timestamp maps
> to numeric types.
>
> In our project, we typically rely on stable versions of libraries. Do you
> have any information on when this feature might be released in a stable 4.x
> version?
>
>
> --
>
> Thanks & Regards,
>
>
> 
> Ravi Mergu
> SMTS-2
> o:  +91 40 64535355
> m: +91 9959618687
> w: www.gaiansolutions.com
>
> 
> 
> 
>
> *Please consider the environment before printing this email.* This message
> contains confidential information and is intended only for the individual
> named. If you are not the named addressee you should not disseminate,
> distribute or copy this e-mail

Re: CVE reports and process to completion

[Ayush Saxena]

Hi Stamatis,
Thanx for starting the thread, I can volunteer as well.

-Ayush

On Tue, 12 Sept 2023 at 13:43, Stamatis Zampetakis  wrote:
>
> Hey everyone,
>
> When someone discovers a potential security vulnerability for Hive (or
> any other Apache project) they can opt to inform the PMC of the
> project by following the ASF guidelines [1]. For Hive, the report
> should be sent to secur...@hive.apache.org.
>
> Next, the PMC follows the steps outlined in [2] to process the report
> and if it is deemed necessary release a fix for the vulnerability.
>
> In order to make the CVE process as smooth as possible and ensure that
> CVE reports are addressed in a timely manner I would like to introduce
> the notion of a "CVE mentor".
>
> The "CVE mentor" is the one responsible for bringing the reported CVE
> to completion ensuring that the steps in [2] are followed. They are
> the principal contact person between the reporter of the vulnerability
> and the PMC and the one who leads the discussions. The triage and fix
> can be done by the mentor or entrusted to a committer (ensuring of
> course that everything remains private till a fix is officially
> released). Given that we need to release a fix very soon after a
> vulnerability is fixed the mentor may also need to act as the release
> manager. Since the reports arrive in the private list the CVE mentor
> should be someone that has access to the security list (all PMC and
> few other individuals).
>
> However, for the idea to work we need a few people (preferably PMC) to
> volunteer for the role of the "CVE mentor". Then the volunteers can
> pick incoming CVE reports in a round robin fashion. Needless to say
> that since I am the one proposing it, I would like to be part of the
> list.
>
> Any additional thoughts or suggestions on how to improve this process
> are very welcomed. Also if you like the idea and want to volunteer
> please reply to this email to add yourself to the list.
>
> Best,
> Stamatis Zampetakis
>
> [1] https://www.apache.org/security/
> [2] https://www.apache.org/security/committers.html#possible

Hive 4.0 GA Planning & Blockers

[Ayush Saxena]

Hi All,
Have been discussing with a couple of folks around Hive 4.0 GA release
and we classified only:
https://issues.apache.org/jira/browse/HIVE-26654 as the blocker for
now & that typically is a parent ticket for failures around TPCDS
queries, rest all stays in 'best effort' state. I will be moving out
all other tickets or maybe reduce their priority from blocker in a day
or two.

We would definitely require some volunteers for that, so in case
anyone has some experience around that part of code and can spare some
cycles, do volunteer!!!

Rest, I along with a couple of folks will try to review/contribute to
the code which we have some idea to get that going as fast as
possible.

Let me know if anyone is aware of any other ticket which you consider
as a blocker apart from the above mentioned for the 4.x GA release.

-Ayush

Re: Release managers

[Ayush Saxena]

I can volunteer for one as well, I plan to create a wiki page around
the release management stuff most specifically where we can update the
Release Managers & tentative release dates & planned features,
blockers for those releases.

Will add a page around basic release validation as well, so that we
can have more volunteers to validate the RC during the release time
:-)

-Ayush

On Thu, 23 Mar 2023 at 21:09, Sai Hemanth Gantasala
 wrote:
>
> Hi all,
>
> I would like to volunteer for the 4.2.0 release.
>
> Thanks,
> Sai.
>
> On Thu, Mar 23, 2023 at 2:47 PM Denys Kuzmenko  wrote:
>
> > Hi, I can take the following one: 4.1.0
> >

Re: Problem encountered when following hive docker quickstart.

[Ayush Saxena]

Hi Away Hua,
Thanx for the report. I think that is indeed a bug in the docker image.

Here it is calling initOrUpgrade schema here [1]
Where as initOrUpgrade is introduced in HIVE-20357 which is there post
4.0.0-alpha-1 [2]

You can raise a hive ticket for this & the fix most probably should be
to use initSchema directly rather than initOrUpgradeSchema.

you can try locally as well by changing it in line [1] and then

cd packaging/src/docker/

./build.sh -hive 3.1.3 -hadoop 3.1.0 -tez 0.9.1

This should build an image locally with the change & you can retry, it
should work. If it does, feel free to create a PR with the fix, if it
doesn't work, let me know. I tried locally and it works for me.

I have pushed the updated image as well

-Ayush


[1] 
https://github.com/apache/hive/blob/master/packaging/src/docker/entrypoint.sh#L27
[2] https://issues.apache.org/jira/browse/HIVE-20357

On Tue, 5 Sept 2023 at 09:10, Away Hua  wrote:
>
> I followed the QuickStart section of hive quickstart to start hiveserver2 
> with version 3.1.3 in docker container. However, I can't start hiveserver2 
> container successfully.  This failed container outputs the following content,
>
> + : derby
> + SKIP_SCHEMA_INIT=false
> + export HIVE_CONF_DIR=/opt/hive/conf
> + HIVE_CONF_DIR=/opt/hive/conf
> + '[' -d '' ']'
> + export 'HADOOP_CLIENT_OPTS= -Xmx1G '
> + HADOOP_CLIENT_OPTS=' -Xmx1G '
> + [[ false == \f\a\l\s\e ]]
> + initialize_hive
> + /opt/hive/bin/schematool -dbType derby -initOrUpgradeSchema
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/opt/hive/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/opt/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> HiveSchemaTool:Parsing failed.  Reason: Missing required option: [-help print 
> this message, -alterCatalog Alter a catalog, requires --catalogLocation 
> and/or --catalogDescription parameter as well, -initSchemaTo Schema 
> initialization to a version, -upgradeSchemaFrom Schema upgrade from a 
> version, -moveDatabase Move a database between catalogs.  Argument is the 
> database name. Requires --fromCatalog and --toCatalog parameters as well, 
> -moveTable Move a table to a different database.  Argument is the table name. 
> Requires --fromCatalog, --toCatalog, --fromDatabase, and --toDatabase  
> parameters as well., -initSchema Schema initialization, -createCatalog Create 
> a catalog, requires --catalogLocation parameter as well, -upgradeSchema 
> Schema upgrade, -info Show config and schema details, -validate Validate the 
> database]
> usage: schemaTool
>  -alterCatalog Alter a catalog, requires
> --catalogLocation and/or
> --catalogDescription parameter as well
>  -catalogDescription   Description of new catalog
>  -catalogLocation  Location of new catalog, required when
> adding a catalog
>  -createCatalogCreate a catalog, requires
> --catalogLocation parameter as well
>  -dbOpts  Backend DB specific options
>  -dbType  Metastore database type
>  -driverdriver name for connection
>  -dryRunlist SQL scripts (no execute)
>  -fromCatalog  Catalog a moving database or table is
> coming from.  This is required if you
> are moving a database or table.
>  -fromDatabase Database a moving table is coming
> from.  This is required if you are
> moving a table.
>  -help  print this message
>  -ifNotExists   If passed then it is not an error to
> create an existing catalog
>  -info  Show config and schema details
>  -initSchemaSchema initialization
>  -initSchemaTo  Schema initialization to a version
>  -metaDbType  Used only if upgrading the system
> catalog for hive
>  -moveDatabase Move a database between catalogs.
> Argument is the database name.
> Requires --fromCatalog and --toCatalog
> parameters as well
>  -moveTableMove a table to a different database.
> Argument is the table name. Requires
> --fromCatalog, --toCatalog,

Re: Include ARM binaries with next release

[Ayush Saxena]

Thanx Stamatis,
I think this: is binary voted or not was discussed long back as well:
[1] , so maybe everyone derived a version from it and so do I :-)

There were references quoted from the past as well like here [2]

Quoting Doug Cutting [3] from a release vote long back
```
Folks should not primarily evaluate binaries when voting. The ASF
primarily produces and publishes source-code so voting artifacts
should be optimized for evaluation of that.
```

But I see your point, thanx for the feedback. Will keep this thread
open & we can revisit if ARM becomes the next big thing in future, who
knows :-)

-Ayush


[1] 
https://issues.apache.org/jira/browse/LEGAL-323?focusedCommentId=16110652&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16110652
[1.1] 
https://issues.apache.org/jira/browse/LEGAL-323?focusedCommentId=16109157&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16109157
[2] https://lists.apache.org/thread/q5o64mh0hz1trwc99fs94l041nhlwgh1
[3] https://en.wikipedia.org/wiki/Doug_Cutting

On Fri, 25 Aug 2023 at 21:21, Stamatis Zampetakis  wrote:
>
> Hey Ayush,
>
> I just wanted to highlight that the vote applies to all released
> artifacts, not only the source packages. The source package is of
> course the primary and most important deliverable but the PMC is
> responsible for everything under downloads.apache.org and similar
> places. Any additional binaries will need to be verified by the PMC to
> ensure that there are no violations of the ASF policy.
>
> While I was working on preparing the 4.0.0-beta-1 it took me quite a
> bit of time to ensure that our convenience binaries comply with the
> ASF guidelines and I am not yet 100% sure that I covered everything. I
> would be more eager to drop the existing convenience binaries rather
> than introducing more.
>
> The additional binaries would also put additional strain on the ASF servers.
>
> I see the benefits for ARM binaries but I would prefer to keep
> releases simple and let those who are interested in those build them
> themselves. We can do whatever we can to facilitate the build process
> of such binaries but not necessarily deliver and host them ourselves.
>
> I am somewhere -0 for this. I am not gonna vote against the idea but
> not supporting it either.
>
> Best,
> Stamatis
>
>
> On Fri, Aug 25, 2023 at 4:48 PM Attila Turoczy
>  wrote:
> >
> > Love it! In 2023 where ARM became an industrial standard. Also ARM perform
> > very well plus the cloud arm vm's are so much cheaper.
> >
> > -Attila
> >
> > On 2023. Aug 25., Fri at 12:48, Ayush Saxena  wrote:
> >
> > > Hi All,
> > > Considering now we do support building Hive on both x86 & ARM, can we
> > > explore having additional binaries built for ARM architecture?
> > >
> > > A lot of projects do release both x86 & ARM binaries example hadoop
> > > [1], can check the Binary Download column in the 3.3.6 row
> > >
> > > As for the process, the release vote is on the source code, which
> > > stays the same for both x86 & ARM. It is just an additional
> > > convenience binary built, signed & released. We can consider making
> > > this step optional as well.
> > >
> > > Let me know what people think!!!
> > >
> > > -Ayush
> > >
> > > [1] https://hadoop.apache.org/releases.html
> > >

Include ARM binaries with next release

[Ayush Saxena]

Hi All,
Considering now we do support building Hive on both x86 & ARM, can we
explore having additional binaries built for ARM architecture?

A lot of projects do release both x86 & ARM binaries example hadoop
[1], can check the Binary Download column in the 3.3.6 row

As for the process, the release vote is on the source code, which
stays the same for both x86 & ARM. It is just an additional
convenience binary built, signed & released. We can consider making
this step optional as well.

Let me know what people think!!!

-Ayush

[1] https://hadoop.apache.org/releases.html

Re: [DISCUSS] Migrate precommit git repos from kgyrtkirk to apache

[Ayush Saxena]

+1,
Thanx Stamatis foe initiating this. This was something which was in my mind as 
well since long but couldn’t find time.

-Ayush 

> On 23-Aug-2023, at 6:19 PM, Zoltan Haindrich  wrote:
> 
> Hey Stamatis!
> 
> I'm happy to donate these repos / help with the migration!
> I should have done it earlier - but it was never top priority...thank you for 
> initiating it!
> 
> cheers,
> Zoltan
> 
>> On 8/23/23 14:00, Stamatis Zampetakis wrote:
>> Hi all,
>> Our precommit infrastructure uses code that resides in the following repos.
>> * https://github.com/kgyrtkirk/hive-test-kube
>> * https://github.com/kgyrtkirk/hive-toolbox
>> * https://github.com/kgyrtkirk/hive-dev-box
>> These are mainly maintained by Zoltán Haindrich who is always helpful
>> and kind to investigate and resolve issues.
>> For facilitating contributions from the apache community and also
>> removing some burden from Zoltan's shoulders it may be a good time to
>> migrate those and put them under the apache namespace.
>> For the initial migration, we could have a straightforward 1 to 1
>> mapping as shown below:
>> * https://github.com/apache/hive-test-kube
>> * https://github.com/apache/hive-toolbox
>> * https://github.com/apache/hive-dev-box
>> How do you feel about this?
>> Best,
>> Stamatis

Re: [VOTE] Release Apache Hive 4.0.0-beta-1 (Release Candidate 0)

[Ayush Saxena]

+1(Binding)

* Built from source
* Verified checksums
* Verified signatures
* Verified no code diff between the git tag & source tar
* Checked the NOTICE & LICENSE files.
* Skimmed over HS2 UI
* Deployed with Derby and tried some operations on ACID & Iceberg tables.

Thanx Stamatis for driving the release, Good Luck!!!

-Ayush

On Thu, 10 Aug 2023 at 18:27, Denys Kuzmenko  wrote:
>
> +1
>
> * Verified signatures and checksum;
> * Checked binary content and successfully built from the source;
> * Skimmed through the release notes;
> * Initialized backend DB schema and launched HMS & HS2 locally;
> * Conducted basic checks via beeline:
> - Created a few ACID & Iceberg tables and loaded data into them;
> - Executed Select/Insert/Update/Delete/Merge/IOW queries.
>
> Thanks, Stamatis for driving the release.
>
> Regards,
> Denys

Re: Request for write access to Hive wiki

[Ayush Saxena]

Hi Simhadri,
It is done.

-Ayush

On Wed, 2 Aug 2023 at 15:40, Simhadri G  wrote:
>
> Hi Everyone,
>
> I need to update the Hive column stats page in the hive wiki :
> https://cwiki.apache.org/confluence/display/Hive/Column+Statistics+in+Hive
> because of a PR.
>
> I kindly request write access to the hive wiki(Confluence username:
> simhadri064).
>
> Thanks!
> Simhadri G

Re: [Twitter] Quickstart dock

[Ayush Saxena]

Thanx Attila, it is up:
https://twitter.com/ApacheHive/status/1683841189987102722?s=20

-Ayush

On Tue, 25 Jul 2023 at 19:31, Attila Turoczy
 wrote:
>
> 🐝🧊 Interested in Apache Hive and @ApacheIceberg? Check out the QuickStart
> documentation at https://iceberg.apache.org/hive-quickstart/ for all the
> details! #ApacheHive #ApacheIceberg
>
> First 2 chars are icons :)

Fwd: Signing releases using automated release infra

[Ayush Saxena]

JFYI.
Maybe worth exploring

-Ayush

Begin forwarded message:

> From: Volkan Yazıcı 
> Date: 19 July 2023 at 1:24:49 AM IST
> To: d...@community.apache.org
> Subject: Signing releases using automated release infra
> Reply-To: d...@community.apache.org
> 
> Abstract: Signing release artifacts using an automated release
> infrastructure has been officially approved by LEGAL. This enables
> projects to sign artifacts using, say, GitHub Actions.
> 
> I have been trying to overhaul the Log4j release process and make it
> as frictionless as possible since last year. As a part of that effort,
> I wanted to sign artifacts in CI during deployment and in a
> `members@a.o` thread[0] I explained how one can do that securely with
> the help of Infra. That was in December 2022. It has been a long,
> rough journey, but we succeeded. In this PR[1], Legal has updated the
> release policy to reflect that this process is officially allowed.
> Further, Infra put together guides[2][3] to assist projects. Logging
> Services PMC has already successfully performed 4 Log4j Tools releases
> using this approach, see its release process[4] for a demonstration.
> 
> [0] (members only!)
> https://lists.apache.org/thread/1o12mkjrhyl45f9pof94pskg55vhs61n
> [1] https://github.com/apache/www-site/pull/235
> [2] https://infra.apache.org/release-publishing.html#signing
> [3] https://infra.apache.org/release-signing.html#automated-release-signing
> [4] https://github.com/apache/logging-log4j-tools/blob/master/RELEASING.adoc
> 
> # F.A.Q.
> 
> ## Why shall a project be interested in this?
> 
> It greatly simplifies the release process. See Log4j Tools release
> process[4], probably the simplest among all Java-based ASF projects.
> 
> ## How can a project get started?
> 
> 1. Make sure your project builds are reproducible (otherwise there is
> no way PMC can verify the integrity of CI-produced and -signed
> artifacts)
> 2. Clone and adapt INFRA-23996 (GPG keys in GitHub secrets)
> 3. Clone and adapt INFRA-23974 (Nexus creds. in GitHub secrets for
> snapshot deployments)
> 4. Clone and adapt INFRA-24051 (Nexus creds. in GitHub secrets for
> staging deployments)
> 
> You might also want to check this[5] GitHub Action workflow for inspiration.
> 
> [5] 
> https://github.com/apache/logging-log4j-tools/blob/master/.github/workflows/build.yml
> 
> ## Does the "automated release infrastructure" (CI) perform the full release?
> 
> No. CI *only* uploads signed artifacts to Nexus. The release manager
> (RM) still needs to copy the CI-generated files to SVN, PMC needs to
> vote, and, upon consensus, RM needs to "close" the release in Nexus
> and so on.
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> For additional commands, e-mail: dev-h...@community.apache.org
> 

Re: [Twitter] Blog on Hive's Data Federation Capabilities

[Ayush Saxena]

Thanx Akshat, it is up on twitter: https://twitter.com/ApacheHive

-Ayush

On Wed, 12 Jul 2023 at 10:08, Akshat m  wrote:
>
> Delve into the world of data federation with Apache Hive in this blog. Explore
> the essence of data federation, uncover Hive's capabilities, and learn
> about its supported integrations. Don't miss out on the insights:
> https://akshatmat.medium.com/data-federation-with-apache-hive-74b3bc5fb72
> #DataFederation #ApacheHive #DataAnalytics
>
> Regards,
> Akshat Mathur

Re: Is there a way to test schema evolution scripts?

[Ayush Saxena]

Hi Zsolt,
You can start by exploring this:
https://github.com/apache/hive/blob/master/standalone-metastore/DEV-README

It has steps to run those metatore scripts over different database
types locally.

-Ayush

On Mon, 10 Jul 2023 at 20:30, Zsolt Miskolczi  wrote:
>
> Hi,
>
> I just stared at a pull request about using liquibase for schema evolution
> and I was thinking how such a change can be validated.
>
> I think the main issue is that we support multiple types of relational
> databases but precommit tests don't test them at all.
>
> I think if we want to check if the idea works, it is enough to:
> - introduce a proper docker based instance for each type of database
> (postgres and derby first).
> - run the upgrade scripts on them
> - check if any error occurs
>
> And the end goal should be something like having validation scripts and
> check if we are able to run selects after the upgrades and fresh installs.
>
>
> What do you think about that?
>
> Thank you,
> Zsolt Miskolczi

Re: Idea: Remove PowerMock

[Ayush Saxena]

+1, PowerMock as far as I remember has issues with JDK-11+ as well,
one such ref : https://stackoverflow.com/questions/52966897/powermock-java-11

-Ayush

On Mon, 10 Jul 2023 at 20:18, Zsolt Miskolczi  wrote:
>
> Hi,
>
> Hive heavily uses PowerMock . The main
> purpose of it is having static mocking.
>
> The sad thing is it seems PowerMock is dead:
> - The main branch got it's lot commit in 2022 and and most of the
> contributions last year were simple dependency upgrades:
> https://github.com/powermock/powermock/commits/release/2.x
> - The last release was in 2020
> - And their mailing list looks dead as well. That is the last email on that
> list: https://groups.google.com/g/powermock/c/JdYY3naZlbU. It asked if it
> was discontinued and didn't get an answer at all.
>
> So officially, it is not dead but it seems it is.
>
> Back then when PowerMock development started, there were no static mocking
> in mockito. But since then, it is possible using mockito-inline.
>
> I won't lie, it is hard to switch from PowerMock: it enables some coding
> patterns that are considered bad patterns and it leads to code that is
> harder to test. Last year I played with it and removed it from the
> hive-exec module: https://github.com/apache/hive/pull/3798.
>
> The hard part in removing it is that PowerMock and mockito-inline don't
> work together. So when we want to remove it, we have to do it in one pull
> request for a given module. It cannot be separated into smaller steps.
> The good news is as it relates to testing, pre commit tests can validate
> the refactor.
>
> What do you think? Should we move away from PowerMock or keep it as it is?
>
> Thank you,
> Zsolt Miskolczi

Re: Blogs/Designs For Apache Hive

[Ayush Saxena]

Have played a bit with this today along with Simhadri.

The page lies here: https://hive.blog.apache.org/

It is pretty simple, not very fancy, considering my Web Designing
skills(which are technically non-existent). But gives us a template to put
blogs related to hive in a single place.

As it goes always, This place belongs to all of us, please feel free to
share links to the blogs or directly raise PR to the hive-site repo to
asf-blog branch and copy the same template as on [1]

I will announce this on our Apache Hive Twitter account and let folks know
they can have their Apache Hive blogs referenced from our website.

Thanx Everyone for the help, do share blogs related to hive and lets give a
good reading experience to folks coming around :-)

-Ayush

[1] https://github.com/apache/hive-site/blob/asf-blog/index.html#L128-L139



On Wed, 7 Jun 2023 at 20:58, Denys Kuzmenko  wrote:

> Nice, tnx Ayush!
>

Re: [DISCUSS] Automatic rerunning of failed tests in Hive Pre-commit

[Ayush Saxena]

+1 from me as well to rerun the failing tests.
The oracle docker is also a pain, it is one of the main reasons for
retriggers, these retriggers wastes a lot of resources and increases the
time to get build results for genuine runs.

-Ayush

On Thu, 8 Jun 2023 at 12:31, Butao Zhang  wrote:

> +1. I often have to rerun whole pre-commit job due to individual unstable
> test, ant it is too time-consuming. It would be much better if we can set
> maven config to retry automatically.
>
>
>
> Thanks,
>
> Butao Zhang
>
>  Replied Message 
> | From | r12 t45 |
> | Date | 6/8/2023 14:52 |
> | To |  |
> | Subject | [DISCUSS] Automatic rerunning of failed tests in Hive
> Pre-commit |
> Hi All,
>
> It often happens that Hive unit tests fail during pre-commit which requires
> rerunning the whole pre-commit job and creates hours of delays.
> What if we set Maven config to retry failed tests automatically X times?
> There is "rerunFailingTestsCount" property in maven-surefire-plugin which
> can be used for that.
> I would like to hear the feedback and if it is positive I could open a JIRA
> ticket and work on it.
>
> Thanks,
> Dmitriy
>

Re: Request to join Hive slack channel

[Ayush Saxena]

Have added everyone. Let me know if it didn't work.

@Aman that mail id was giving me errors, I added you manually, Let me know
if it works, if not, we would need another mail id :-)

-Ayush

On Wed, 7 Jun 2023 at 21:25, Alagappan M 
wrote:

>
> Hi Ayush,
> Can you add aswin.alagappa...@gmail.com to the Hive slack channel.
>
> On Wed, Jun 7, 2023 at 12:48 AM Ayush Saxena  wrote:
>
>> Hi Dayakar,
>> I have added you, you must get a mail once that request gets approved
>> from Infra side
>>
>> -Ayush
>>
>> > On 07-Jun-2023, at 11:20 AM, Dayakar Morem 
>> wrote:
>> >
>> > Hi Ayush,
>> >
>> > Can you please add me to the channel. My email id is
>> daya.apa...@gmail.com
>> >
>> > Thanks & Regards,
>> > Dayakar.
>> >
>> >> On Sat, May 20, 2023 at 10:02 AM Ayush Saxena 
>> wrote:
>> >>
>> >> Have added all the people who requested here + Attila Turoczy and
>> Simhadri
>> >> G (received offline requests).
>> >>
>> >> I hope everyone uses the medium of contact in the correct way and just
>> for
>> >> "Apache" Hive related stuff. I won't be personally following that
>> channel
>> >> much but in case someone needs help in that context, do let us know
>> here,
>> >> anyone from the community should get you sorted.
>> >>
>> >> -Ayush
>> >>
>> >>> On Fri, 19 May 2023 at 12:23, Butao Zhang 
>> wrote:
>> >>>
>> >>> Thanks Ayush. The invitation is valid. I have joined the hive slack
>> >>> channel.
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Butao Zhang
>> >>>  Replied Message 
>> >>> From Soumyakanti Das
>> >>> 
>> >>> Date 5/19/2023 14:43
>> >>> To  
>> >>> Subject Re: Request to join Hive slack channel
>> >>> Hey Ayush,
>> >>>
>> >>> Please add me to the channel as well. My email is
>> >>> soumyakanti.das...@gmail.com
>> >>>
>> >>> Thanks!
>> >>>
>> >>>> On Thu, May 18, 2023 at 23:34 Ayush Saxena 
>> wrote:
>> >>>
>> >>> Sure Sungwoo, I will add you as well.
>> >>>
>> >>> Butao Zhang was added to the channel, I can see him in the list of
>> >>> members and I received a notification as well that his account has
>> >>> been approved by the infra team, I think I am not the only one to
>> >>> approve, it has to be approved by Infra as well.
>> >>>
>> >>> Let me wait for sometime if there are other non-committer interested
>> >>> as well in joining the #hive slack channel, so I can submit the
>> >>> request all together, approving one-one request might piss off Infra
>> >>> folks :-)
>> >>>
>> >>> -Ayush
>> >>>
>> >>>> On Fri, 19 May 2023 at 11:44, Sungwoo Park 
>> wrote:
>> >>>
>> >>>
>> >>> I am sorry for spamming -- My email address is: glap...@gmail.com
>> >>>
>> >>> Thanks,
>> >>> --- Sungwoo Park
>> >>>
>> >>>> On Fri, May 19, 2023 at 3:11 PM Sungwoo Park 
>> wrote:
>> >>>
>> >>> If non-committers can join the slack channel, I would like to join,
>> >>>
>> >>> too.
>> >>>
>> >>> An invitation will be appreciated very much (glapa...@gmail.com).
>> >>>
>> >>> Thanks,
>> >>>
>> >>> --- Sungwoo Park
>> >>>
>> >>>
>> >>> On Fri, May 19, 2023 at 2:49 PM Butao Zhang 
>> >>>
>> >>> wrote:
>> >>>
>> >>>
>> >>> Hi, Hive dev
>> >>>
>> >>>
>> >>> I just saw this updated page:
>> >>> https://cwiki.apache.org/confluence/display/Hive/HowToCommit. It
>> >>>
>> >>> seems
>> >>>
>> >>> we individual  can request to join the slack channel.
>> >>> If that is possible, I want to join the slack, please give me a
>> >>> invitation, Thanks.
>> >>>
>> >>>
>> >>> My Gmail address:  butaozha...@gmail.com
>> >>>
>> >>>
>> >>>
>> >>> Thanks,
>> >>>
>> >>> Butao Zhang
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>
>>
>

Re: Hive 4.0 branching

[Ayush Saxena]

Hi Vihang,
Regarding " branch-4.0.0-alpha1 and branch-4.0.0-alpha-2" : Those branches
were created for releasing those alpha1 & alpha2 versions.
Currently, the master branch points to 4.0.0-Snapshot [1], so that would be
used for the next release. A branch would be cut from the current master
branch.

So, in an ideal scenario, we would have cut a branch off the master branch
and named it as branch-4.0.0 and used it for releasing the hive-4 version.
This thread [2] has details around the next releases, We have decided to
have one more beta release before the 4.0.0 release, so a branch like
branch-4.0.0-beta1 would be created from master and then post that beta
release when we go for 4.0.0 release,

As per the timelines: 4.0.0-beta1 branch should be there by next week & may
be a release vote by End of Month, and then we would be planning for hive-4
GA release in a couple of months time, when we sort out all the release
blockers for that release.

Hope I didn't contribute more to the confusion :-)

-Ayush


[1]
https://github.com/apache/hive/blob/a73194342cf99a907531565e0c98af7effb79637/pom.xml#L24
[2] https://lists.apache.org/thread/vpr0hfypf9f3g7lffwk4pz5g5g59qwzh

On Thu, 8 Jun 2023 at 10:44, vihang karajgaonkar 
wrote:

> Hello Dev community,
>
> I noticed that we have a branch-4.0.0-alpha1 and branch-4.0.0-alpha-2 but I
> don't see a branch-4. Does anyone know which branch would be used to cut
> out the final 4.0.0 release and when would a branch-4 be available?
>
> Thanks,
> Vihang
>

Blogs/Designs For Apache Hive

[Ayush Saxena]

Hi All,
As Infra has decommissioned blogs.apache.org, each project gets an
individual space for blogs.
I have configured one(With tons of help from Infra people) for us.

It stays here:
https://hive.blog.apache.org/

It is just an initial commit, to make sure our space works. We need to do
some polishing and configure Hugo most probably to do the publishing stuff
automatically.

Should be ready in a week's time, so we can get going publishing our blogs
or anything similar to that page.

If someone already has something to publish, do share, If someone wants to
help with this design stuff for this space, that would be really Awesome!!!

-Ayush

Re: Request to join Hive slack channel

[Ayush Saxena]

Hi Dayakar,
I have added you, you must get a mail once that request gets approved from 
Infra side

-Ayush

> On 07-Jun-2023, at 11:20 AM, Dayakar Morem  wrote:
> 
> Hi Ayush,
> 
> Can you please add me to the channel. My email id is daya.apa...@gmail.com
> 
> Thanks & Regards,
> Dayakar.
> 
>> On Sat, May 20, 2023 at 10:02 AM Ayush Saxena  wrote:
>> 
>> Have added all the people who requested here + Attila Turoczy and Simhadri
>> G (received offline requests).
>> 
>> I hope everyone uses the medium of contact in the correct way and just for
>> "Apache" Hive related stuff. I won't be personally following that channel
>> much but in case someone needs help in that context, do let us know here,
>> anyone from the community should get you sorted.
>> 
>> -Ayush
>> 
>>> On Fri, 19 May 2023 at 12:23, Butao Zhang  wrote:
>>> 
>>> Thanks Ayush. The invitation is valid. I have joined the hive slack
>>> channel.
>>> 
>>> Thanks,
>>> 
>>> Butao Zhang
>>>  Replied Message 
>>> From Soumyakanti Das
>>> 
>>> Date 5/19/2023 14:43
>>> To  
>>> Subject Re: Request to join Hive slack channel
>>> Hey Ayush,
>>> 
>>> Please add me to the channel as well. My email is
>>> soumyakanti.das...@gmail.com
>>> 
>>> Thanks!
>>> 
>>>> On Thu, May 18, 2023 at 23:34 Ayush Saxena  wrote:
>>> 
>>> Sure Sungwoo, I will add you as well.
>>> 
>>> Butao Zhang was added to the channel, I can see him in the list of
>>> members and I received a notification as well that his account has
>>> been approved by the infra team, I think I am not the only one to
>>> approve, it has to be approved by Infra as well.
>>> 
>>> Let me wait for sometime if there are other non-committer interested
>>> as well in joining the #hive slack channel, so I can submit the
>>> request all together, approving one-one request might piss off Infra
>>> folks :-)
>>> 
>>> -Ayush
>>> 
>>>> On Fri, 19 May 2023 at 11:44, Sungwoo Park  wrote:
>>> 
>>> 
>>> I am sorry for spamming -- My email address is: glap...@gmail.com
>>> 
>>> Thanks,
>>> --- Sungwoo Park
>>> 
>>>> On Fri, May 19, 2023 at 3:11 PM Sungwoo Park  wrote:
>>> 
>>> If non-committers can join the slack channel, I would like to join,
>>> 
>>> too.
>>> 
>>> An invitation will be appreciated very much (glapa...@gmail.com).
>>> 
>>> Thanks,
>>> 
>>> --- Sungwoo Park
>>> 
>>> 
>>> On Fri, May 19, 2023 at 2:49 PM Butao Zhang 
>>> 
>>> wrote:
>>> 
>>> 
>>> Hi, Hive dev
>>> 
>>> 
>>> I just saw this updated page:
>>> https://cwiki.apache.org/confluence/display/Hive/HowToCommit. It
>>> 
>>> seems
>>> 
>>> we individual  can request to join the slack channel.
>>> If that is possible, I want to join the slack, please give me a
>>> invitation, Thanks.
>>> 
>>> 
>>> My Gmail address:  butaozha...@gmail.com
>>> 
>>> 
>>> 
>>> Thanks,
>>> 
>>> Butao Zhang
>>> 
>>> 
>>> 
>>> 
>>> 
>> 

Re: [DISCUSS] HIVE 4.0.0 GA Release Proposal

[Ayush Saxena]

+1, for a pre GA release

Beta sounds better to me, we don’t have any half baked major features, just
probabilities of some bugs

-Ayush

On Thu, 1 Jun 2023 at 4:24 PM, Attila Turoczy 
wrote:

> Ayush just told me, the mailing list does not support images. It is a very
> sad world :-(
>
> Previous meme:
> https://imgflip.com/i/7nuzql
>
> On Thu, Jun 1, 2023 at 11:46 AM Attila Turoczy 
> wrote:
>
>> OK. Then let's go with beta. Please start. (if not started already) a
>> vote for a release.
>>
>> I think we should have a more frequent release cadence. The community
>> needs it, we need this, also without frequent release nobody will believe
>> this project is healthy. I know the first will be the hardest, but we need
>> to fix those issues and release them. Even a monthly release could be
>> possible for hive. We are capable to do this!
>>
>> [image: image.png]
>>
>> -Attila
>>
>> On Thu, Jun 1, 2023 at 11:24 AM Stamatis Zampetakis 
>> wrote:
>>
>>> +1 from me as well. Any alpha or beta name should be fine I have no
>>> strong preferences.
>>>
>>> On Wed, May 31, 2023, 2:30 PM László Bodor 
>>> wrote:
>>>
>>> > Hi!
>>> >
>>> > +1 for creating a new release before GA in the presence of possible
>>> > correctness problems. I'm not 100% sure about alpha or beta, I'm fine
>>> with
>>> > alpha-3.
>>> >
>>> > Regards,
>>> > Laszlo Bodor
>>> >
>>> > Denys Kuzmenko  ezt írta (időpont: 2023. máj.
>>> 31.,
>>> > Sze, 14:22):
>>> >
>>> > > Hi folks,
>>> > >
>>> > > The master branch has many new features, bug fixes, and performance
>>> > > improvements since alpha-2. However, we still have several
>>> correctness
>>> > bugs
>>> > > [HIVE-26654] and performance issues that should be eliminated before
>>> the
>>> > > GA.
>>> > >
>>> > > Could we consider doing a beta release to keep at least a 6-month
>>> release
>>> > > cadence and also show the community that 4.0.0 GA is the next stop?
>>> > >
>>> > > Thanks,
>>> > > Denys
>>> > >
>>> >
>>>
>>

Re: Reg: Discussion on removal of deprecated APIs in the HMS thrift interface

[Ayush Saxena]

+1 to what Stamatis said, if it is there in 3.X we can explore their removal, 
else let them go in 4.x GA release and we can remove then in the subsequent 
release

-Ayush

> On 01-Jun-2023, at 3:08 PM, Stamatis Zampetakis  wrote:
> 
> Hello,
> 
> Ideally we should deprecate APIs in one release and remove them in a
> subsequent major release. If the HMS deprecations were added in Hive
> 3.X then I am ok removing them now. Otherwise it is not really that we
> will remove deprecated APIs but we will remove regular APIs without
> any notice.
> 
> Best,
> Stamatis
> 
>> On Thu, Jun 1, 2023 at 2:57 AM Sai Hemanth Gantasala
>>  wrote:
>> 
>> Hi everyone,
>> 
>> This thread is to initiate a discussion on the removal of deprecated APIs
>> in the HMS thrift class. Any client including HiveMetastoreClient talks to
>> HiveMetaStore Server via the thrift layer. Over the past few years, the
>> thrift class is bloated with duplicated APIs with varying parameters
>> (function overloading) in the API definition. The reason why the APIs are
>> being deprecated is that the API might need an additional argument, so a
>> new API is added with an additional argument, and mark the old API as
>> deprecated.
>> 
>> I'm working on HIVE-26537 
>> to clean up the code around the interaction between HiveMetaStoreClient and
>> HMS to not use the deprecated APIs (the HMS client will now be using
>> request-based APIs instead of APIs using individual arguments). Going
>> forward, using these request-based APIs is ideal as we can just add an
>> additional field to request object definition in the thrift class and API
>> remains unchanged. This would hopefully require minimal changes between
>> client and server interaction in the future.
>> 
>> I would like to hear the community member's opinions regarding the
>> deprecated APIs,
>> 1) Keep the deprecated APIs for the 4.x release, HMSClient will use the
>> request-based APIs, So that would keep the older clients compatible with
>> the new HMS server.
>> 2) Remove the deprecated APIs for the 4.x release. This would break
>> backward compatibility with the older clients but we have the opportunity
>> to clean up a lot of deprecated code. Since we are making a major release
>> after 5 years, I hope this incompatibility is acceptable.
>> 
>> Please let me know your thoughts.
>> 
>> Thanks,
>> Sai.

Move to JDK-11

[Ayush Saxena]

Hi Everyone,
Want to pull in the attention of folks towards moving to JDK-11 compile
time support in Hive. There was a ticket in the past [1] which talks about
it and If I could decode it right, it was blocked because the Hadoop
version used by Hive didn't had JDK-11 runtime support, But with [2] in we
have upgraded the Hadoop version, so that problem is sorted out. I couldn't
even see any unresolved tickets in the blocked state either.

I quickly tried* a  mvn clean install -DskipTests -Piceberg -Pitests
-Dmaven.javadoc.skip=true

And no surprises it failed with some weird exceptions towards the end. But
I think that should be solvable.

So, Questions?

   - What do folks think about this? Should we put in some effort towards
   JDK-11
   - Should we support both JDK-11 & JDK-8?
   - Ditch JDK-11 and directly shoot for JDK-17?

Let me know your thoughts, In case anyone has some experience in this area
and have tried something in the context, feel free to share or may be if
someone has any potential action plan or so

-Ayush

[1] https://issues.apache.org/jira/browse/HIVE-22415
[2] https://issues.apache.org/jira/browse/HIVE-24484

* changed the maven.compiler.source & maven.compiler.target to 11

Apache Hive on Twitter

[Ayush Saxena]

Hi All,
I am happy to announce: We have got the Apache Hive Twitter account active
again or maybe in other words we have got creds to use it now.

The twitter account stays here:

https://twitter.com/ApacheHive

The account belongs to all of us at Hive. As we decided, if anyone wants to
get anything posted on the Twitter account, related to Apache Hive. He/She
can drop a mail to the Hive Dev mailing with the request, with a label in
the subject [Twitter].

For the record as of today, following people have access to post:

Alan Gates, Ayush Saxena, Carl Steinbach, Joydeep Sen Sharma, Owen
O'Malley, Sushanth Sowmyan, Szehon Ho, Thejas Nair & Vikram Dixit

A note of thanks to Joydeep Sen Sharma, Carl Steinbach, Stamatis Zampetakis
& Naveen Gangam for helping with the process. Attila Turoczy for the
initial thoughts/idea.

-Ayush

Re: [DISCUSS] Nightly snaphot builds

[Ayush Saxena]

Hi Vihang,
+1, We were even exploring publishing the docker images of the snapshot
version as well per commit or maybe weekly, so just shoot 2 docker commands
and you get a Hive cluster running with master code.

Sai, I think to spin up an env via Docker with all these things should be
doable for sure, but would require someone with real good expertise with
docker as well as setting up these services with Hive. Obviously, I am not
that guy :-)

@Simhadri has a PR which publishes docker images once a release tag is
pushed, you can explore to have similar stuff for the Snapshot version,
maybe if that sounds cool

-Ayush

On Tue, 23 May 2023 at 04:26, Sai Hemanth Gantasala
 wrote:

> Hi Vihang,
>
> +1 on the idea.
>
> This is a great idea to quickly test if a certain feature is working as
> expected on a certain branch.
> This way we test data loss, correctness, or any other unexpected scenarios
> that are Hive specific only. However, I'm wondering if it is possible to
> deploy/test in a kerberized environment or issues involving authorization
> services like sentry/ranger.
>
> Thanks,
> Sai.
>
> On Mon, May 22, 2023 at 11:15 AM vihang karajgaonkar 
> wrote:
>
> > Hello Team,
> >
> > I have observed that it is a common use-case where users would like to
> test
> > out unreleased features/bug fixes either to unblock them or test out if
> the
> > bug fixes really work as intended in their environments. Today in the
> case
> > of Apache Hive, this is not very user friendly because it requires the
> end
> > user to build the binaries directly from the hive source code.
> >
> > I found that Apache Spark has a very useful infrastructure [1] which
> > deploys nightly snapshots [2] [3] from the branch using github actions.
> > This is super useful for any user who wants to try out the latest and
> > greatest using the nightly builds.
> >
> > I was wondering if we should also adopt this. We can use github actions
> to
> > upload the snapshot jars to the public repository (e.g github packages)
> and
> > schedule it as a nightly job.
> >
> > [1] https://issues.apache.org/jira/browse/INFRA-21167
> > [2] https://github.com/apache/spark/pkgs/container/apache-spark-ci-image
> > [3] https://github.com/apache/spark/pull/30623
> >
> > I can take a stab at this if the community thinks that this is a nice
> thing
> > to have.
> >
> > Thanks,
> > Vihang
> >
>

Re: Request to join Hive slack channel

[Ayush Saxena]

Have added all the people who requested here + Attila Turoczy and Simhadri
G (received offline requests).

I hope everyone uses the medium of contact in the correct way and just for
"Apache" Hive related stuff. I won't be personally following that channel
much but in case someone needs help in that context, do let us know here,
anyone from the community should get you sorted.

-Ayush

On Fri, 19 May 2023 at 12:23, Butao Zhang  wrote:

> Thanks Ayush. The invitation is valid. I have joined the hive slack
> channel.
>
> Thanks,
>
> Butao Zhang
>  Replied Message 
> From Soumyakanti Das
> 
> Date 5/19/2023 14:43
> To  
> Subject Re: Request to join Hive slack channel
> Hey Ayush,
>
> Please add me to the channel as well. My email is
> soumyakanti.das...@gmail.com
>
> Thanks!
>
> On Thu, May 18, 2023 at 23:34 Ayush Saxena  wrote:
>
> Sure Sungwoo, I will add you as well.
>
> Butao Zhang was added to the channel, I can see him in the list of
> members and I received a notification as well that his account has
> been approved by the infra team, I think I am not the only one to
> approve, it has to be approved by Infra as well.
>
> Let me wait for sometime if there are other non-committer interested
> as well in joining the #hive slack channel, so I can submit the
> request all together, approving one-one request might piss off Infra
> folks :-)
>
> -Ayush
>
> On Fri, 19 May 2023 at 11:44, Sungwoo Park  wrote:
>
>
> I am sorry for spamming -- My email address is: glap...@gmail.com
>
> Thanks,
> --- Sungwoo Park
>
> On Fri, May 19, 2023 at 3:11 PM Sungwoo Park  wrote:
>
> If non-committers can join the slack channel, I would like to join,
>
> too.
>
> An invitation will be appreciated very much (glapa...@gmail.com).
>
> Thanks,
>
> --- Sungwoo Park
>
>
> On Fri, May 19, 2023 at 2:49 PM Butao Zhang 
>
> wrote:
>
>
> Hi, Hive dev
>
>
> I just saw this updated page:
> https://cwiki.apache.org/confluence/display/Hive/HowToCommit. It
>
> seems
>
> we individual  can request to join the slack channel.
> If that is possible, I want to join the slack, please give me a
> invitation, Thanks.
>
>
> My Gmail address:  butaozha...@gmail.com
>
>
>
> Thanks,
>
> Butao Zhang
>
>
>
>
>

Re: Updating the Hive Committer Guide Wiki

[Ayush Saxena]

I won't even suggest people to get into any dev related discussions on
slack, some casual stuff/conversations is ok, It was there in the doc
around irc channel, I just updated it and didn't want to say no to
anyone interested to join. The channel already had some ~140 people.

I just added the interested people. But yes, as established any
technical or project level discussion should happen on the relevant
Mailing Lists. I suppose the contributors joining those will be mature
enough to know what resource to be used in which way.

As mentioned in the Apache Docs: "If it didn't happen on the Mailing
List, means it didn't happen", The apache way [1]

btw. there is way to integrate slack to mailing lists and stuff like
that, but I don't think it is the time to chase that

Yep, let's update the other wiki as well, it would be great if we get
some more volunteers as well :-)

-Ayush

[1] https://theapacheway.com/on-list/

On Fri, 19 May 2023 at 14:55, Stamatis Zampetakis  wrote:
>
> Thanks for updating the wiki Ayush! Definitely very helpful and
> hopefully we can do it for other pages as well.
>
> Slack is a very useful tool but personally I don't have much time to
> monitor yet another channel of communication. I don't know if we
> should encourage people to start discussions there especially since
> access is moderated and search archives are not openly available. I
> would prefer to direct people to dev@ or user@ and not slack but this
> is just my personal opinion.
>
> Best,
> Stamatis
>
> On Fri, May 19, 2023 at 6:27 AM Ayush Saxena  wrote:
> >
> > Hi All,
> > I recently observed that our Hive Committer guide is pretty outdated
> > and has mentioned legacy ways of committing, but still has a lot of
> > relevant information.
> >
> > After discussing with some friends offline, I have updated the doc.
> > Feel free to share feedback or improvements.
> >
> > Committers to the projects already have access to the wiki, so they
> > can directly update it, If anyone else has any feedback, feel free to
> > share and someone amongst the committer group would be happy to get
> > things updated.
> >
> > The Wiki page lies here:
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27362108
> >
> > -Ayush

Re: Request to join Hive slack channel

[Ayush Saxena]

Sure Sungwoo, I will add you as well.

Butao Zhang was added to the channel, I can see him in the list of
members and I received a notification as well that his account has
been approved by the infra team, I think I am not the only one to
approve, it has to be approved by Infra as well.

Let me wait for sometime if there are other non-committer interested
as well in joining the #hive slack channel, so I can submit the
request all together, approving one-one request might piss off Infra
folks :-)

-Ayush

On Fri, 19 May 2023 at 11:44, Sungwoo Park  wrote:
>
> I am sorry for spamming -- My email address is: glap...@gmail.com
>
> Thanks,
> --- Sungwoo Park
>
> On Fri, May 19, 2023 at 3:11 PM Sungwoo Park  wrote:
>
> > If non-committers can join the slack channel, I would like to join, too.
> > An invitation will be appreciated very much (glapa...@gmail.com).
> >
> > Thanks,
> >
> > --- Sungwoo Park
> >
> >
> > On Fri, May 19, 2023 at 2:49 PM Butao Zhang  wrote:
> >
> >> Hi, Hive dev
> >>
> >>
> >> I just saw this updated page:
> >> https://cwiki.apache.org/confluence/display/Hive/HowToCommit. It seems
> >> we individual  can request to join the slack channel.
> >> If that is possible, I want to join the slack, please give me a
> >> invitation, Thanks.
> >>
> >>
> >> My Gmail address:  butaozha...@gmail.com
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Butao Zhang
> >
> >

Re: Request to join Hive slack channel

[Ayush Saxena]

Hi Butao,
I have sent you an invite, please check, you must have received a
mail. I have done it for the first time, so I might have messed it up.
Lemme know if it works.

For others the doc on how to add lies here:
https://infra.apache.org/slack.html

-Ayush

On Fri, 19 May 2023 at 11:19, Butao Zhang  wrote:
>
> Hi, Hive dev
>
> I just saw this updated page: 
> https://cwiki.apache.org/confluence/display/Hive/HowToCommit. It seems we 
> individual  can request to join the slack channel.
> If that is possible, I want to join the slack, please give me a invitation, 
> Thanks.
>
> My Gmail address:  butaozha...@gmail.com
>
> Thanks,
>
> Butao Zhang

Updating the Hive Committer Guide Wiki

[Ayush Saxena]

Hi All,
I recently observed that our Hive Committer guide is pretty outdated
and has mentioned legacy ways of committing, but still has a lot of
relevant information.

After discussing with some friends offline, I have updated the doc.
Feel free to share feedback or improvements.

Committers to the projects already have access to the wiki, so they
can directly update it, If anyone else has any feedback, feel free to
share and someone amongst the committer group would be happy to get
things updated.

The Wiki page lies here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27362108

-Ayush

Re: [DISCUSS] Disable JIRA worklog for GitHub PRs

[Ayush Saxena]

+1

Thanx Stamatis, makes sense

-Ayush

> On 12-May-2023, at 10:14 PM, Attila Turoczy  
> wrote:
> 
> +1
> 
>> On Fri, May 12, 2023 at 4:01 PM Alessandro Solimando <
>> alessandro.solima...@gmail.com> wrote:
>> Hi Stamatis,
>> I am experiencing the same too, so +1 from me.
>> Best regards,
>> Alessandro
>> On Fri, 12 May 2023 at 15:58, Stamatis Zampetakis 
>> wrote:
>>> Hello,
>>> Everything that happens in a GitHub PR creates a worklog entry under
>>> the respective JIRA ticket.
>>> For every worklog entry we receive a notification from j...@apache.org
>>> when we are watching an issue. The worklog entry and email
>>> notification usually appear messy.
>>> Moreover, if we are watching the GitHub PR we are going to get a
>>> notification from notificati...@github.com which has the same content
>>> with the JIRA worklog entry and is much more readable.
>>> Finally, the PR notification is also going to
>>> iss...@hive.apache.org and git...@hive.apache.org so those who are
>>> subscribed to these lists
>>> will get the same notification multiple times.
>>> Personally, I never read the JIRA worklog notifications and I largely
>>> prefer those from notificati...@github.com.
>>> How do you feel about disabling the worklog entries in JIRA coming
>>> from GitHub PRs?
>>> For archiving purposes, the notifications already go to gitbox@ so we
>>> don't lose anything from disabling the worklog entries. On the
>>> contrary, I find that this would reduce the noise and redundancy in
>>> our inboxes.
>>> Concretely this is what I have in mind in terms of change:
>>> https://github.com/apache/hive/pull/4318
>>> Best,
>>> Stamatis

Re: I will subscript to this

[Ayush Saxena]

Send a mail to dev-subscr...@hive.apache.org

-Ayush

On Fri, 28 Apr 2023 at 9:54 AM, Carlos Guerrero 
wrote:

>

[jira] [Created] (HIVE-27223) Show Compactions failing with NPE

[Ayush Saxena (Jira)]

Ayush Saxena created HIVE-27223:
---

 Summary: Show Compactions failing with NPE
 Key: HIVE-27223
 URL: https://issues.apache.org/jira/browse/HIVE-27223
 Project: Hive
  Issue Type: Bug
Reporter: Ayush Saxena
Assignee: Ayush Saxena


{noformat}
java.lang.NullPointerException: null
at java.io.DataOutputStream.writeBytes(DataOutputStream.java:274) ~[?:?]
at 
org.apache.hadoop.hive.ql.ddl.process.show.compactions.ShowCompactionsOperation.writeRow(ShowCompactionsOperation.java:135)
 
at 
org.apache.hadoop.hive.ql.ddl.process.show.compactions.ShowCompactionsOperation.execute(ShowCompactionsOperation.java:57)
 
at org.apache.hadoop.hive.ql.ddl.DDLTask.execute(DDLTask.java:84)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:213) 
at 
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:105) 
at org.apache.hadoop.hive.ql.Executor.launchTask(Executor.java:360) 
{noformat}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27208) Iceberg: Add support for rename table

[Ayush Saxena (Jira)]

Ayush Saxena created HIVE-27208:
---

 Summary: Iceberg: Add support for rename table
 Key: HIVE-27208
 URL: https://issues.apache.org/jira/browse/HIVE-27208
 Project: Hive
  Issue Type: Improvement
Reporter: Ayush Saxena
Assignee: Ayush Saxena


Add support for renaming iceberg tables.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Issue related to apache hive 3.1.3

[Ayush Saxena]

Hi Omkar,

If Impala has some issues, you should reach to Impala mailing lists

-Ayush

> On 29-Mar-2023, at 1:13 PM, Omkar Rohadkar  
> wrote:
> 
> 
> Hi team,
> 
> We are using apache hive 3.1.3 and we are using Mapreduce execution engine 
> and it worked fine , but while launching the impala service we get an error 
> that MR execution engine is not supported .
> 
> 
> 
> Is there any way that we can still proceed with Mapreduce execution engine 
> and launch Impala service.
> 
> Thanks and regards,
> Omkar Rohadkar
> 
> Privileged/Confidential information may be contained in this message and may 
> be subject to legal privilege. Access to this e-mail by anyone other than the 
> intended is unauthorized. If you are not the intended recipient (or 
> responsible for delivery of the message to such person), you may not use, 
> copy, distribute or deliver to anyone this message (or any part of its 
> contents ) or take any action in reliance on it. In such case, you should 
> destroy this message, and notify us immediately. If you have received this 
> email in error, please notify us immediately by e-mail or telephone and 
> delete the e-mail from any computer. If you or your employer does not consent 
> to internet e-mail messages of this kind, please notify us immediately by 
> e-mail. All reasonable precautions have been taken to ensure no viruses are 
> present in this e-mail. As our company cannot accept responsibility for any 
> loss or damage arising from the use of this e-mail or attachments we 
> recommend that you subject these to your virus checking procedures prior to 
> use. The views, opinions, conclusions and other information expressed in this 
> electronic mail are not given or endorsed by the company unless otherwise 
> indicated by an authorized representative independent of this message.

[jira] [Created] (HIVE-27185) Iceberg: Cache iceberg table while loading for stats

[Ayush Saxena (Jira)]

Ayush Saxena created HIVE-27185:
---

 Summary: Iceberg: Cache iceberg table while loading for stats
 Key: HIVE-27185
 URL: https://issues.apache.org/jira/browse/HIVE-27185
 Project: Hive
  Issue Type: Improvement
Reporter: Ayush Saxena
Assignee: Ayush Saxena


Presently iceberg for stats loads the iceberg table multiple times for stats 
via different routes.
Cache it to avoid reading/loading the iceberg table multiple times.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27182) tez_union_with_udf.q with TestMiniTezCliDriver is flaky

[Ayush Saxena (Jira)]

Ayush Saxena created HIVE-27182:
---

 Summary: tez_union_with_udf.q with TestMiniTezCliDriver is flaky
 Key: HIVE-27182
 URL: https://issues.apache.org/jira/browse/HIVE-27182
 Project: Hive
  Issue Type: Improvement
Reporter: Ayush Saxena


Looks like memory issue:

{noformat}
< Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: 
java.lang.OutOfMemoryError: GC overhead limit exceeded
< Serialization trace:
< genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
< colExprMap (org.apache.hadoop.hive.ql.plan.SelectDesc)
< conf (org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator)
< childOperators (org.apache.hadoop.hive.ql.exec.vector.VectorLimitOperator)
< childOperators (org.apache.hadoop.hive.ql.exec.TableScanOperator)
{noformat}




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (HIVE-27177) Add alter table...Convert to Iceberg command

[Ayush Saxena (Jira)]

Ayush Saxena created HIVE-27177:
---

 Summary: Add alter table...Convert to Iceberg command
 Key: HIVE-27177
 URL: https://issues.apache.org/jira/browse/HIVE-27177
 Project: Hive
  Issue Type: Improvement
Reporter: Ayush Saxena
Assignee: Ayush Saxena


Add an alter table  convert to Iceberg [TBLPROPERTIES('','')] to 
convert exiting external tables to iceberg tables



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [DISCUSS] Move Jira notification emails out of dev@hive

[Ayush Saxena]

+1,

Thanx Stamatis for starting the thread

-Ayush

> On 25-Mar-2023, at 3:57 PM, Stamatis Zampetakis  wrote:
> 
> Hi everyone,
> 
> In the last Hive board report someone mentioned that the volume of Jira
> notification emails to the dev list is huge especially when compared to
> emails send by actual humans making it hard for someone to follow what's
> happening in the project.
> 
> I personally share their viewpoint. For a long time I have been relying on
> client side (Gmail) filters to separate Jira notifications from other
> emails to the dev list.
> 
> I think it would be better to direct the traffic from jira to a separate
> list namely jira@hive to keep the dev@hive list clean and dedicated to
> human interaction.
> 
> What do you think?
> 
> Best,
> Stamatis

Fwd: TAC supporting Berlin Buzzwords

[Ayush Saxena]

Forwarded as Received.

-Ayush

-- Forwarded message -
From: Gavin McDonald 
Date: Fri, 24 Mar 2023 at 15:27
Subject: TAC supporting Berlin Buzzwords
To: 


PMCs,

Please forward to your dev and user lists.

Hi All,

The ASF Travel Assistance Committee is supporting taking up to six (6)
people
to attend Berlin Buzzwords In June this year.

This includes Conference passes, and travel & accommodation as needed.

Please see our website at https://tac.apache.org for more information and
how to apply.

Applications close on 15th April.

Good luck to those that apply.

Gavin McDonald (VP TAC)

Re: [DISCUSS] Incremental and cadence predictable release activity for HIVE

[Ayush Saxena]

Hi Kirti,
Thanx for the initiative. This sounds very interesting, but I doubt if it
is that easy to incorporate. Sharing my thoughts:

   - Regarding "Unpredictable" : I don't think we are like doing very
   unpredictable releases. It should be a formal mail, like Release x.y.z and
   then the RM usually shares a potential Branch freeze date, then a
   margin number of days for blockers or critical tickets. And this entire
   process would be around a minimum of 1 month and usually will go around 3
   months.
   - Regarding "Regressions": Quicker releases doesn't certainly mean more
   stable releases.
   - Regarding half-baked features: We are mostly developing on master
   branch, we don't have a concept of feature branch(a lot of projects have
   that), So, if a bunch of features are running in parallel by different set
   of people, with a "fixed" date it is practically impossible to achieve,
   this thing needs to be negotiated b/w all of them.
   - Even if we pin a date, that ain't sufficient, we need volunteers who
   can take up the RM role, If we proceed with this we should decide the RM as
   well beforehand.
   - This timeline thing can get screwed up in case you hit a security
   issue: AFAIK you can't announce a CVE unless you have a release on all
   active release lines with the fix. So, in that case this schedule will get
   messed up and the RM, the dates would require to be renegotiated.
   - Sometimes you need to release early because a downstream project needs
   a fix, which blocks their way to upgrade Hive. Standard practice, almost
   All apache projects are concerned about each other and help others in
   upgrading, so in that case I am not sure holding them for a fixed date is
   cool or not
   - Mostly what I have observed, A release takes place when we have enough
   tickets to release, We don't want to just keep on releasing with just 20-25
   fixes, nor we want to push straight 800-900 fixes in one go. The number of
   fixes, the nature of fixes all should be taken in account while planning
   the release date.


In general: Good Idea, We should definitely encourage more frequent
releases, having a "strict" date or not is debatable.

-Ayush

On Sun, 12 Mar 2023 at 19:44, Kirti Ruge  wrote:

> Hello HIVE Dev,
>
> I would like to discuss/propose incremental and cadence predictable
> process for HIVE releases.
>
> https://hive.apache.org/general/downloads/
>
> Currently, our releases have a very random span in between, and those have
> sometimes caused problems like-
>
> 1. All downstream and end users have unpredictable schedules because of
> upstream.
> 2. More chances of regression issues when there is an unplanned release
> date. As developers and release managers have to rush, this prevents us
> from focusing on having a proper regression-free release.
>
> I would like to propose a branch cut twice a year to have two strict
> releases yearly. It would make release cadence predictable for end users
> and bring some disciplinary schedules for all users, including downstream
> projects.
>
> Advantages of this approach-
>
> 1. If we pin a branch cut date, features can be prioritized better so that
> no half-baked stuff goes into release.
> 2. Such Incremental release will help in better regression and reduce the
> burden from release management activity( result is reduced issues and
> problems with quality). It will eventually help to streamline release
> management activity.
>
>
> Let me know your thoughts.
>
> Thanks,
> Kirti

Re: [DISCUSS] Jira Public Signup Disabled

[Ayush Saxena]

Folks,
New stuff now, INFRA has introduced a new Utility which can be used for
Jira id creation[1], It is mentioned over here as well in the
announcement[2] from Infra team.

Guess we should update our contributor docs[3] to reflect that and ask
folks to route their request via this util.

-Ayush

[1] https://selfserve.apache.org/jira-account.html
[2] https://infra.apache.org/blog/brand-new-selfserve-page.html
[3]
https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-JIRA

On Thu, 17 Nov 2022 at 16:43, Stamatis Zampetakis  wrote:

> The jira-reque...@hive.apache.org has been created and I added relevant
> instructions on how to request a JIRA account in the wiki [1]. Feel free to
> improve as you see fit!
>
> Best,
> Stamatis
>
> [1]
> https://cwiki.apache.org/confluence/display/Hive/HowToContribute#HowToContribute-JIRA
>
> On Tue, Nov 15, 2022 at 9:59 PM Stamatis Zampetakis 
> wrote:
>
>> Logged https://issues.apache.org/jira/browse/INFRA-23905 for the
>> creation of the new mailing list.
>>
>> On Tue, Nov 15, 2022 at 9:57 PM Abhay Chennagiri <
>> achennag...@cloudera.com> wrote:
>>
>>> +1, Thank you, Stamatis.
>>>
>>> On Tue, Nov 15, 2022 at 12:42 PM Pravin Sinha 
>>> wrote:
>>>
 +1, Thanks, Stamatis.

 -Pravin

 On Tue, Nov 15, 2022 at 5:57 PM Stamatis Zampetakis 
 wrote:

> Hi everyone,
>
> Due to the large amount of spam account creation the ASF INFRA team
> has disabled the JIRA account creation [1].
>
> From the 11th of November, contributors who wish to have a JIRA
> account (to create, assign, watch, etc issues) will need to request an
> account through an ASF PMC.
>
> Other projects, such as Calcite, have already taken the necessary
> actions to streamline the process for new contributors [2].
>
> I would suggest drawing inspiration from Calcite and take similar
> actions in Hive.
>
> If you all agree we can start by creating a dedicated (private)
> mailing lists for such requests:
> jira-reque...@hive.apache.org
>
> and then proceed with a brief documentation of the process in the wiki
> or website.
>
> What do you think?
>
> Best,
> Stamatis
>
> [1] https://blogs.apache.org/infra/entry/jira-public-signup-disabled
> [2] https://lists.apache.org/thread/5odg6wyvwfkryk96ls2w3vxnrkftw50s
>

Re: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from branch-3

[Ayush Saxena]

t; > +1 on Vihang's suggestion
> > > I remember that spark removal was a debated thing even on master, so
> > > completely removing it backwards from a "maintenance" branch-3 line is
> > not
> > > really acceptable (actually, I'm surprised it's not -1ed yet by hive on
> > > spark folks), but it depends on what *deprecation* really means: I mean
> > > disabling some spark tests to stabilize precommit is completely fine in
> > the
> > > absence of community aspiration to fix them properly
> > >
> > > regarding the motivation: "This would ensure that branch-3 is aligned
> > with
> > > the master as done in ..."  <-- I don't think we're targeting this, we
> > are
> > > about to make 3.x releases as simply as possible
> > >
> > > I'm hoping/assuming that most of the +1s so far are in line with
> Vihang's
> > > suggestion
> > >
> > > vihang karajgaonkar  ezt írta (időpont: 2023.
> febr.
> > > 23., Cs, 16:37):
> > >
> > > > +1 to deprecate Hive on Spark.
> > > >
> > > > I feel directly removing it in a minor release is probably a bad
> idea.
> > > Most
> > > > users will upgrade to 3.2 first and go to 4.0 later. If we deprecate
> it
> > > in
> > > > 3.2 it transitions well into its removal as users upgrade to 4.0
> > > > eventually.
> > > >
> > > > If the goal to stabilize the branch-3, we can disable the failing
> tests
> > > on
> > > > Hive on Spark.
> > > >
> > > > Thanks,
> > > > Vihang
> > > >
> > > > On Thu, Feb 23, 2023 at 12:32 AM Alessandro Solimando <
> > > > alessandro.solima...@gmail.com> wrote:
> > > >
> > > > > +1 from me too
> > > > >
> > > > > On Thu, 23 Feb 2023 at 06:09, Ayush Saxena 
> > wrote:
> > > > >
> > > > > > +1 on removing Hive on Spark from branch-3
> > > > > >
> > > > > > -Ayush
> > > > > >
> > > > > > > On 23-Feb-2023, at 6:40 AM, Wang, Yuming
> >  > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > +1.
> > > > > > >
> > > > > > > From: Naresh P R 
> > > > > > > Date: Thursday, February 23, 2023 at 02:49
> > > > > > > To: dev@hive.apache.org 
> > > > > > > Subject: Re: [EXTERNAL] Re: Proposal to deprecate Hive on Spark
> > > from
> > > > > > branch-3
> > > > > > > External Email
> > > > > > >
> > > > > > > +1 to remove Hive on Spark in branch-3
> > > > > > > ---
> > > > > > > Regards,
> > > > > > > Naresh P R
> > > > > > >
> > > > > > >> On Wed, Feb 22, 2023 at 5:37 AM Sankar Hariappan
> > > > > > >>  wrote:
> > > > > > >>
> > > > > > >> +1, to remove Hive on Spark in branch-3.
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >> Sankar
> > > > > > >>
> > > > > > >> -Original Message-
> > > > > > >> From: Rajesh Balamohan 
> > > > > > >> Sent: Wednesday, February 22, 2023 6:58 PM
> > > > > > >> To: dev@hive.apache.org
> > > > > > >> Subject: [EXTERNAL] Re: Proposal to deprecate Hive on Spark
> from
> > > > > > branch-3
> > > > > > >>
> > > > > > >> +1 on removing Hive on Spark in branch-3.
> > > > > > >>
> > > > > > >> It was not done earlier since it was removing a feature in the
> > > > branch.
> > > > > > But
> > > > > > >> if there is enough consensus, we should consider removing it.
> > > > > > >>
> > > > > > >> ~Rajesh.B
> > > > > > >>
> > > > > > >> On Wed, Feb 22, 2023 at 12:48 PM Aman Raj
> > > > >  > > > > > >
> > > > > > >> wrote:
> > > > > > >>
> > > > > >

Re: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from branch-3

[Ayush Saxena]

+1 on removing Hive on Spark from branch-3

-Ayush

> On 23-Feb-2023, at 6:40 AM, Wang, Yuming  wrote:
> 
> +1.
> 
> From: Naresh P R 
> Date: Thursday, February 23, 2023 at 02:49
> To: dev@hive.apache.org 
> Subject: Re: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from branch-3
> External Email
> 
> +1 to remove Hive on Spark in branch-3
> ---
> Regards,
> Naresh P R
> 
>> On Wed, Feb 22, 2023 at 5:37 AM Sankar Hariappan
>>  wrote:
>> 
>> +1, to remove Hive on Spark in branch-3.
>> 
>> Thanks,
>> Sankar
>> 
>> -Original Message-
>> From: Rajesh Balamohan 
>> Sent: Wednesday, February 22, 2023 6:58 PM
>> To: dev@hive.apache.org
>> Subject: [EXTERNAL] Re: Proposal to deprecate Hive on Spark from branch-3
>> 
>> +1 on removing Hive on Spark in branch-3.
>> 
>> It was not done earlier since it was removing a feature in the branch. But
>> if there is enough consensus, we should consider removing it.
>> 
>> ~Rajesh.B
>> 
>> On Wed, Feb 22, 2023 at 12:48 PM Aman Raj 
>> wrote:
>> 
>>> Hi team,
>>> 
>>> We have been trying to fix Hive on Spark test failures for a long
>>> time. As of now, branch-3 has less than 12 test failures (whose fix
>>> have not been identified). 8 of them are related to Hive on Spark. I
>>> had mailed about the failures in my previous mail threads. Thanks to
>>> Vihang for working on them as well. But we have not been able to
>> identify the root cause till now.
>>> These fixes can be tracked in the following tickets : [HIVE-27087] Fix
>>> TestMiniSparkOnYarnCliDriver test failures on branch-3 - ASF JIRA (
>>> apache.org)>> 3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-27087&data=05%7C01%7C
>>> Sankar.Hariappan%40microsoft.com%7C687a6a4dbd41454568e008db14d8cc23%7C
>>> 72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnknow
>>> n%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLC
>>> JXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RbAqrwK6fQFDStufXYfpusNc81EzjtpiaHm
>>> qv5CFiAs%3D&reserved=0> and [HIVE-26940] Backport of HIVE-19882 : Fix
>>> QTestUtil session lifecycle - ASF JIRA
>>> (apache.org)>> %3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FHIVE-26940&data=05%7C01%7
>>> CSankar.Hariappan%40microsoft.com%7C687a6a4dbd41454568e008db14d8cc23%7
>>> C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnkno
>>> wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>>> CJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=PaDtEZD569Sv0ER9sC4l6q1ZxyoBeER3zn
>>> Bsc51PWI8%3D&reserved=0>
>>> 
>>> Until we have a green branch-3, we cannot go ahead to push new
>>> features for the Hive-3.2.0 release. This is kind of a blocker for this
>> release.
>>> Already bringing the test fixes to the current state took more than 2
>>> months.
>>> 
>>> I wanted to bring up a proposal to deprecate Hive on Spark from
>>> branch-3 altogether. This would ensure that branch-3 is aligned with
>>> the master as done in
>>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissu%2F&data=05%7C01%7Cyumwang%40ebay.com%7C2bd54cc0c84a4e44a59e08db150574e5%7C46326bff992841a0baca17c16c94ea99%7C0%7C0%7C638126885411646147%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=n51cF5fFuwSFFhX%2B0S828W3jYN3G3YwRwJWne1AMGtg%3D&reserved=0
>>> es.apache.org
>> %2Fjira%2Fbrowse%2FHIVE-26134&data=05%7C01%7CSankar.Hariappan%
>> 40microsoft.com%7C687a6a4dbd41454568e008db14d8cc23%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638126693641861742%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=j%2F%2BYTakSvfk1Mm%2Fq8kI84gKm6s%2Fg2iA1abIPzY7t%2Bsg%3D&reserved=0.
>> Just wanted to have a vote on this in parallel working on the test fixes.
>> If we have the approval from the community, we can deprecate it altogether.
>>> 
>>> Please feel free to suggest any concerns or suggestions you have.
>>> Also, I welcome any possible fix suggestion for the test failures.
>>> 
>>> Thanks,
>>> Aman.
>>> 
>>