[jira] [Created] (HIVE-26884) Iceberg: V2 Vectorization returns wrong results with deletes

2022-12-21 Thread Ayush Saxena (Jira)
Ayush Saxena created HIVE-26884:
---

 Summary: Iceberg: V2 Vectorization returns wrong results with 
deletes
 Key: HIVE-26884
 URL: https://issues.apache.org/jira/browse/HIVE-26884
 Project: Hive
  Issue Type: Bug
Reporter: Ayush Saxena
Assignee: Ayush Saxena


In case of Iceberg V2 reads, if we have delete files, and a couple of parquet 
blocks are skipped in that case the row number calculation is screwed and that 
leads to mismatch with delete filter row positions and hence leading to wrong 
results.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


Re: Lock branch-3 in order for PR build to run successfully.

2022-12-21 Thread Stamatis Zampetakis
Hello,

I don't believe a lock is necessary. I think that people with write access
to the repository already know the processes and how to behave.
If someone decides to push a commit to the repo without running pre-commit
tests there should be a good reason to do so.
I am hoping that circumventing the usual workflow should be a rather rare
event.

Best,
Stamatis

On Tue, Dec 20, 2022 at 8:50 AM Aman Raj 
wrote:

> Hi community,
>
> I see a couple of commits that went in directly to branch-3 before setting
> up the Jenkins pipeline for branch-3. To prevent this, can we lock the
> branch-3 of Hive in order to provide PR's the only way to merge commits in
> branch-3.
>
> Can someone help me in locking branch-3 so that we have a clean release
> process. I do not have the access to do it.
>
> Thanks,
> Aman.
> 
> From: Aman Raj 
> Sent: Friday, December 9, 2022 9:33 AM
> To: dev@hive.apache.org 
> Subject: Re: [EXTERNAL] Re: Sync of Branch-3 & Branch-3.1 for 3.2.0
> pipeline
>
> Thanks Pravin for your support. Can someone please help me merge this PR
> to branch-3 HIVE-26816 : Add Jenkins file for branch-3 by amanraj2520 ·
> Pull Request #3841 · apache/hive (github.com)<
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhive%2Fpull%2F3841=05%7C01%7Crajaman%40microsoft.com%7C94c1ac2c4ddd40437b5f08dad99a6017%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638061554335365489%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=D7CwaRcRaQ5ubjz3Ki95HkyclN2a%2BBZ7lvTddDQpTLY%3D=0>.
> I do not have access to do that. Then we will start development on it.
>
> Thanks,
> Aman.
>
>
> 
> From: Pravin Sinha 
> Sent: Friday, December 9, 2022 1:55 AM
> To: dev@hive.apache.org 
> Subject: Re: [EXTERNAL] Re: Sync of Branch-3 & Branch-3.1 for 3.2.0
> pipeline
>
> [You don't often get email from mailpravi...@gmail.com. Learn why this is
> important at https://aka.ms/LearnAboutSenderIdentification ]
>
> Hi Aman,
>  I also think that we can merge the PR to enable the test pipeline if the
> change looks fine and subsequently we can fix the tests to bring it to
> green state (hopefully by cherry picking a few commits from branch-3.1
> which is already in green state) . Looks like currently the tests are
> broken in branch-3.
>
> Thanks,
> Pravin
>
> On Thu, Dec 8, 2022 at 3:59 PM Aman Raj 
> wrote:
>
> > Hi team,
> >
> > For the addition of Jenkins file for branch-3, branch-3 has some existing
> > tests failing which was because Jenkins was not running on branch-3. We
> are
> > planning to merge this Jenkins file irrespective of this PR having test
> > failures, since this does not change the code. We will create separate
> > tasks for ensuring that branch-3 has a green build.
> >
> > Link to the PR :
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fhive%2Fpull%2F3841=05%7C01%7Crajaman%40microsoft.com%7C94c1ac2c4ddd40437b5f08dad99a6017%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638061554335365489%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=D7CwaRcRaQ5ubjz3Ki95HkyclN2a%2BBZ7lvTddDQpTLY%3D=0
> >
> > Fyi, branch-3.1 has a green build.
> >
> > Thanks,
> > Aman.
> > 
> > From: Aman Raj 
> > Sent: Wednesday, December 7, 2022 3:19 PM
> > To: dev@hive.apache.org 
> > Subject: Re: [EXTERNAL] Re: Sync of Branch-3 & Branch-3.1 for 3.2.0
> > pipeline
> >
> > Hi Ayush,
> >
> > Thanks for clarifying. Will wait for it to turn green.
> >
> > Thanks,
> > Aman.
> > 
> > From: Ayush Saxena 
> > Sent: Wednesday, December 7, 2022 3:11 PM
> > To: dev@hive.apache.org 
> > Subject: Re: [EXTERNAL] Re: Sync of Branch-3 & Branch-3.1 for 3.2.0
> > pipeline
> >
> > Hi Aman,
> > The build is already running for your PR:
> >
> >
> https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.hive.apache.org%2Fblue%2Forganizations%2Fjenkins%2Fhive-precommit%2Fdetail%2FPR-3841%2F1%2Fpipeline=05%7C01%7Crajaman%40microsoft.com%7C94c1ac2c4ddd40437b5f08dad99a6017%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638061554335365489%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C=osIglpTld3PtOPyFhGBLTSU9Ku1FWngPMofNQXILpyM%3D=0
> >
> > The JenkinsFile is picked from the PR while running rather than the
> target
> > branch.
> >
> > -Ayush
> >
> > > On 07-Dec-2022, at 3:03 PM, Aman Raj 
> > wrote:
> > >
> > > Hi Stamatis,
> > >
> > > How can we ensure that unless the PR is merged. Please suggest.
> > > I was thinking of merging this and raising a sample PR on branch-3 to
> > check whether it works or not. Is there some other way?
> > >
> > > Thanks,
> > > Aman.
> > > 
> > > From: Stamatis Zampetakis 
> > > Sent: Wednesday, December 7, 2022 2:51 PM
> > > To: 

Re: [ANNOUNCE] New PMC Member: Ayush Saxena

2022-12-21 Thread Pravin Sinha
Congratulations, Ayush ! Well deserved.

-Pravin

On Wed, Dec 21, 2022 at 10:18 AM Kirti Ruge  wrote:

> Congratulations Ayush.
>
> On Wed, 21 Dec 2022 at 12:15 AM, Chris Nauroth 
> wrote:
>
>> Congratulations, Ayush!
>>
>> Chris Nauroth
>>
>>
>> On Tue, Dec 20, 2022 at 10:02 AM Sai Hemanth Gantasala <
>> saihema...@cloudera.com> wrote:
>>
>> > Congratulations Ayush, Very well deserved!!.
>> >
>> > On Mon, Dec 19, 2022 at 5:12 PM Naveen Gangam 
>> > wrote:
>> >
>> >> Hello Hive Community,
>> >> Apache Hive PMC is pleased to announce that Ayush Saxena has accepted
>> the
>> >> Apache Hive PMC's invitation to become PMC Member, and is now our
>> newest
>> >> PMC member. Many thanks to Ayush for all the contributions he has made
>> and
>> >> looking forward to many more future contributions in the expanded role.
>> >>
>> >> Please join me in congratulating Ayush !!!
>> >>
>> >> Cheers,
>> >> Naveen (on behalf of Hive PMC)
>> >>
>> >>
>> >
>>
>


[jira] [Created] (HIVE-26883) Revert "HIVE-21872 Bucketed tables that load data from data/files/auto_sortmerge_join should be tagged as 'bucketing_version'='1'"

2022-12-21 Thread Aman Raj (Jira)
Aman Raj created HIVE-26883:
---

 Summary: Revert "HIVE-21872 Bucketed tables that load data from 
data/files/auto_sortmerge_join should be tagged as 'bucketing_version'='1'"
 Key: HIVE-26883
 URL: https://issues.apache.org/jira/browse/HIVE-26883
 Project: Hive
  Issue Type: Sub-task
Reporter: Aman Raj
Assignee: Aman Raj


Tests are failing with the following errors:
Client Execution succeeded but contained differences (error code = 1) after 
executing auto_sortmerge_join_12.q 
3d2
< TBLPROPERTIES('bucketing_version'='1')
9d7
< TBLPROPERTIES('bucketing_version'='1')
31d28
< TBLPROPERTIES('bucketing_version'='1')
36d32
< TBLPROPERTIES('bucketing_version'='1')
108d103
< TBLPROPERTIES('bucketing_version'='1')
114d108
< TBLPROPERTIES('bucketing_version'='1')
249c243
< bucketing_version 1
---
> bucketing_version 2
299c293
< bucketing_version 1
---
> bucketing_version 2
378c372
< bucketing_version 1
---
> bucketing_version 2
456c450
< bucketing_version 1
---
> bucketing_version 2
 
 
Since this commit was already reverted in branch-3.1 we can revert this in 
branch-3 to make it 3.1 compatible.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (HIVE-26882) Allow transactional check of Table parameter before altering the Table

2022-12-21 Thread Peter Vary (Jira)
Peter Vary created HIVE-26882:
-

 Summary: Allow transactional check of Table parameter before 
altering the Table
 Key: HIVE-26882
 URL: https://issues.apache.org/jira/browse/HIVE-26882
 Project: Hive
  Issue Type: Improvement
  Components: Standalone Metastore
Reporter: Peter Vary


We should add the possibility to transactionally check if a Table parameter is 
changed before altering the table in the HMS.

This would provide an alternative, less error-prone and faster way to commit an 
Iceberg table, as the Iceberg table currently needs to:
- Create an exclusive lock
- Get the table metadata to check if the current snapshot is not changed
- Update the table metadata
- Release the lock

After the change these 4 HMS calls could be substituted with a single alter 
table call.
Also we could avoid cases where the locks are left hanging by failed processes



--
This message was sent by Atlassian Jira
(v8.20.10#820010)