[jira] [Created] (HIVE-25545) Add/Drop constraints events on table should be authorized in HS2

[Sai Hemanth Gantasala (Jira)]

Sai Hemanth Gantasala created HIVE-25545:


 Summary: Add/Drop constraints events on table should be authorized 
in HS2
 Key: HIVE-25545
 URL: https://issues.apache.org/jira/browse/HIVE-25545
 Project: Hive
  Issue Type: Bug
  Components: HiveServer2
Reporter: Sai Hemanth Gantasala
Assignee: Sai Hemanth Gantasala


Alter table foo_tbl ADD constraint c1_unique UNIQUE(id1) disable novalidate;
Alter table foo_tbl DROP constraint c1_unique;

The above statements are currently not being authorized in Ranger/Sentry. These 
should be authorized by creating authorizable events in Hive.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-25544) Remove Dependency of hive-meta-common From hive-common

[David Mollitor (Jira)]

David Mollitor created HIVE-25544:
-

 Summary: Remove Dependency of hive-meta-common From hive-common
 Key: HIVE-25544
 URL: https://issues.apache.org/jira/browse/HIVE-25544
 Project: Hive
  Issue Type: Improvement
Reporter: David Mollitor
Assignee: David Mollitor


These two things should not be linked and it means any HS2 client libraries 
pulling in hive-common library also has to pull in a ton of metastore code as 
well.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: hive-exec vs. hive-exec:core

[Edward Capriolo]

recommendation from the Hive team is to use the hive-exec.jar artifact.

You know about 10 years ago. I mentioned that oozie should just use
hive-service or hive jdbc. After a big fight where folks kept bringing up
concurrency bugs in hive-server-1 my prs were rejected (even though hive
server2 would not have these bugs). I still cannot fathom why someone using
oozie would want a fat jar of hive (as opposed to hive server or hivejdbc)
. If I had to do that, i would just use shell action. You all must like
enjoy shading jars.

Edward

On Thu, Sep 16, 2021 at 2:30 PM Chao Sun  wrote:

> I'm not sure whether it is a good idea to remove `hive-exec-core`
> completely - it is still being used today by some other popular projects
> including Spark and Trino/Presto. By sticking to `hive-exec-core` it gives
> more flexibility to the other projects to shade & relocate those classes
> according to their need, without waiting for new Hive releases. Hive also
> needs to make sure it relocate everything properly. Otherwise, if some
> classes are shaded & included in `hive-exec` but not relocated, there is no
> way for the other projects to exclude them and avoid potential conflicts.
>
> Chao
>
> On Thu, Sep 16, 2021 at 8:03 AM Zoltan Haindrich  wrote:
>
> > Hey
> >
> > On 9/6/21 12:48 PM, Stamatis Zampetakis wrote:
> > > Indeed this may lead to binary incompatibility problems as the one you
> > > mentioned. If I understood correctly the problem you cite comes up if
> > > library B in this case is not relocated. If Hive systematically
> relocates
> > > shaded deps do you think there will still be binary incompatibility
> > issues?
> > >
> > > If the relocating solution works, I would personally prefer going down
> > this
> > > path instead of introducing an entirely new module just for the sake of
> > > dependency management. Most of the time when there are problems with
> > > shading the answer comes from relocating the problematic dependencies
> and
> > > people are more or less accustomed with this route.
> >
> > I totally agree with you Stamatis - with the addition that we should work
> > together with the owners of other projects to help them use the correct
> > artifact to gain access to
> > Hive's internal parts.
> > I've opened HIVE-25531 to remove the core classified artifact - and
> ensure
> > that we will be uncovering and fixing future issues with the hive-exec
> > artifact.
> >
> > cheers,
> > Zoltan
> >
> >
> > >
> > > Best,
> > > Stamatis
> > >
> > > On Mon, Aug 30, 2021 at 9:49 PM Daniel Fritsi
> > 
> > > wrote:
> > >
> > >> Dear Hive developers,
> > >>
> > >> I am Dan from the Oozie team and I would like to bring up the
> > >> hive-exec.jar vs. hive-exec-core.jar topic.
> > >> The reason for that is because as far as we understand the official
> > >> recommendation from the Hive team is to use the hive-exec.jar
> artifact.
> > >>
> > >> However in Oozie that can end-up in a binary incompatibility.
> > >>
> > >> The reason for that is:
> > >>
> > >>* Let's say library A is included in the fat Jar.
> > >>
> > >>* And library B which is using library A is also included in the
> fat
> > Jar.
> > >>
> > >>* Let's also say that library A's com.library.alib package is
> > >>  relocated to org.apache.hive.com.library.alib,
> > >>  meaning the com.library.alib.SomeClass becomes
> > >>  org.apache.hive.com.library.alib.SomeClass
> > >>
> > >>* So if B has a method like public void
> > >>  someMethod(com.library.alib.SomeClass) then the signature of this
> > >>  method will be changed to:
> > >>  public void
> someMethod(org.apache.hive.com.library.alib.SomeClass)
> > >>
> > >>* If Oozie is also using B directly meaning we'll have b.jar on our
> > >>  classpath, but with the unchanged signature,
> > >>  so when hive-exec tries to invoke someMethod then depending on
> > >>  whether b.jar coming from us will be loaded first or hive-exec
> > will,
> > >>  we can end-up with a NoSuchMethodError is hive-exec tries to pass
> > an
> > >>  org.apache.hive.com.library.alib.SomeClass instance to the
> > >>  someMethod which was loaded from the original b.jar.
> > >>
> > >> Hence in Oozie a long time ago (OOZIE-2621
> > >> ) the decision was
> > >> made to use the hive-exec-core Jar.
> > >>
> > >> Now since the shading process actually removes those dependencies from
> > >> the hive-exec pom which are included in the fat Jar, we manually had
> to
> > >> add some dependencies to Oozie to compensate this.
> > >> However these dependencies are not used by Oozie directly and with the
> > >> growing features of hive-exec we had to repeat the same process
> > >> over-and-over which is a bit unmaintainable.
> > >>
> > >> Today I'm writing to you to propose a long-term solution where
> basically
> > >> nothing would change in the generated hive artifacts, poms and the
> same
> > >> time we wouldn't have to 

[jira] [Created] (HIVE-25543) Add Read-Only Capability to ObjectStore

[David Mollitor (Jira)]

David Mollitor created HIVE-25543:
-

 Summary: Add Read-Only Capability to ObjectStore
 Key: HIVE-25543
 URL: https://issues.apache.org/jira/browse/HIVE-25543
 Project: Hive
  Issue Type: Improvement
Reporter: David Mollitor
Assignee: David Mollitor


Recently saw some stack-traces that shows that calling "commit" triggers quite 
a bit of work within DataNucleus, as I understand it, to look for changes in 
the transaction and to commit those changes.

Given that many of the RPCs within the Metastore are look-ups, Hive can avoid 
all these needless work by making transaction read-only (rollbackOnly).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-25542) Remove References to hive.optimize.index.filter

[David Mollitor (Jira)]

David Mollitor created HIVE-25542:
-

 Summary: Remove References to hive.optimize.index.filter
 Key: HIVE-25542
 URL: https://issues.apache.org/jira/browse/HIVE-25542
 Project: Hive
  Issue Type: Improvement
Reporter: David Mollitor


Hive indexes were removed from 4.x series.

Please remove all references to the Index configurations
For example: hive.optimize.index.filter

Also update the docs:
https://cwiki.apache.org/confluence/display/hive/configuration+properties



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Delays in precommit runs

[Zoltan Haindrich]

Hey All,

I've merged a change to enable branch indexing on the ci job - this will enable 
it to autocleanup old builds and it will also make sure it will start runs even 
in case the github event is lost.
As a side effect of this it rediscovered almost all PRs - i've aborted all of 
the runs which already had a green run...but left the others running.
So, right now it is busy running those tests...if I count it correctly; it 
still has around 25 to go...
it would have been better to wait until the weekend with this...sorry for the 
holdup; I think it will get better by tomorrow.

cheers,
Zoltan