Thanks Ayush and Zsolt. >From what I have seen, Hive Atlas integration is mainly in replication metadata (optional as hive.repl.include.atlas.metadata is false by default).
I will investigate the usage, dependencies and impact and follow up with more information. Thanks, Kokila On Thu, Oct 1, 2026 at 1:24 PM Ayush Saxena <[email protected]> wrote: > Hi Zsolt, > > I think I vaguely recall an atlas-hive JAR. I’m not sure whether that > is purely a downstream artifact or if something similar exists > upstream as well. > > From a process perspective, though, if it doesn’t exist in the Hive > repository, I don’t think it should block us. It would be an Atlas > issue to resolve. If they don’t upgrade or fix their integration, they > may remain on an older Hive release while we move forward. > > That’s one of the advantages of the upstream vs. downstream model: we > need to focus on our own project rather than trying to account for the > entire stack. Of course, we should avoid unnecessarily breaking > downstream consumers and maintain compatibility where reasonably > possible. But for major milestones, I think everything is open for > discussion as long as we have consensus within the project. > > -Ayush > > On Thu, 1 Oct 2026 at 12:44, Zsolt Miskolczi <[email protected]> > wrote: > > > > Hi Ayush, > > > > > Regarding Atlas: is it used only by replication? I remember that > > replication had a dependency on Atlas for the Atlas replication > > feature, but that feature never made it to GA. > > > > I'm not sure about it. Atlas has a post exec hook that is not visible in > the Hive repository itself but when you use atlas, it is registered into > Hive. That means they provide a code that Hive runs. We had problems at > clusters back then when Atlas upgraded Jersey without notifying Hive about > it. I suppose it goes both ways. > > > > Thanks, > > Zsolt > > > > Ayush Saxena <[email protected]> ezt írta (időpont: 2026. okt. 1., Cs, > 8:38): > >> > >> Hi Kokila, > >> Thanks for sharing the details. > >> > >> Regarding Atlas: is it used only by replication? I remember that > >> replication had a dependency on Atlas for the Atlas replication > >> feature, but that feature never made it to GA. > >> > >> If that is the only major thing currently blocking us, I was wondering > >> if we could start a new thread and discuss dropping support for that > >> feature. We could also mention that the Atlas community has not been > >> very active, and the dependency is now blocking some of our > >> initiatives as well. > >> > >> Moreover, I don't think anyone is actively using the Atlas replication > >> feature at this point. > >> > >> Feel free to reach out if I can help or if you're stuck; I will be > >> happy to help move this forward :-) > >> > >> -Ayush > >> > >> On Wed, 30 Sept 2026 at 11:17, kokila narayanan > >> <[email protected]> wrote: > >> > > >> > Hi Ayush, > >> > I was working on the Hadoop upgrade to version 3.5.0. > Currently, we are blocked on few items. > >> > > >> > Hive requires Tez to also be on Hadoop 3.5.0. The necessary work is > done (https://github.com/apache/tez/pull/474) but not released yet. > >> > Atlas is currently using Jersey 1 which causes compilation issues and > is likely to lead to runtime errors. It's in progress: > https://issues.apache.org/jira/browse/ATLAS-5190. I have reached out to > get info on when it can be completed. > >> > Ranger has already completed the Jersey upgrade (Not released), but > not for Hadoop yet. > >> > > >> > In the meantime, I have been working on hive test failures which are > now reduced to 7 from over 140 + failures. > >> > > >> > Regards, > >> > Kokila > >> > > >> > On Wed, Sep 30, 2026 at 10:58 AM Ayush Saxena <[email protected]> > wrote: > >> >> > >> >> Hi Everyone, > >> >> Trying to revive this thread. It looks like quite a few projects have > >> >> either moved to JDK 25 already or are actively working towards it. If > >> >> I’m not mistaken, the latest Hadoop release also supports JDK 25 at > >> >> runtime. > >> >> Has anyone started looking into this on our side? I noticed Kokila > was > >> >> experimenting with something along these lines—any pointers on the > >> >> current status or work that has already been done? > >> >> > >> >> Also, if anyone has thoughts, plans, or would like to volunteer to > >> >> help drive this effort, please let me know. > >> >> > >> >> -Ayush > >> >> > >> >> On Thu, 22 Jan 2026 at 20:16, Ayush Saxena <[email protected]> > wrote: > >> >> > > >> >> > I’ve created a ticket to track the JDK 25–related work, assuming > there are no strong objections to this move: > https://issues.apache.org/jira/browse/HIVE-29412 > >> >> > > >> >> > -Ayush > >> >> > > >> >> > On Thu, 22 Jan 2026 at 08:18, Edward Capriolo < > [email protected]> wrote: > >> >> >> > >> >> >> Long Term Support 17 and 25. > >> >> >> 21 was popular for a moment, but I think it didn't have enough > shine to be LTS (especially with the vector support not appearing till 22 > and 23.) > >> >> >> > >> >> >> In general if I have to build something that I know is a pure > model for broad "sharing" I will target 17. But now with the "cool stuff" > like the pattern matching the only viable target is 25. > >> >> >> > >> >> >> 17 and 25 > >> >> >> > >> >> >> hadoop core just pushed to 17 minimum 3.5.0 > >> >> >> > >> >> >> Go 25 the future is here. > >> >> >> > >> >> >> On Wed, Jan 21, 2026 at 8:48 PM Shohei Okumiya <[email protected]> > wrote: > >> >> >>> > >> >> >>> +1. I also think it can be a good signal to promote our momentum. > >> >> >>> > >> >> >>> Besides Hadoop, I remember Trino dropped supports to some > connectors (if I remember correctly, they were Kudu and something) because > their client libraries didn't work with JDK 24. We may need collaborations > with some projects. > >> >> >>> > >> >> >>> Best, > >> >> >>> Okumin > >> >> >>> > >> >> >>> On Thu, Jan 22, 2026 at 1:27 László Bodor < > [email protected]> wrote: > >> >> >>>> > >> >> >>>> The corresponding Tez Jira is: > https://issues.apache.org/jira/browse/TEZ-4675 > >> >> >>>> Feel free to link Hive's one to it if any. > >> >> >>>> > >> >> >>>> On Tue, 20 Jan 2026 at 15:49, Ayush Saxena <[email protected]> > wrote: > >> >> >>>>> > >> >> >>>>> The Hadoop-related issue is already well known. There are > several tickets addressing it [1], with more currently in progress. > >> >> >>>>> > >> >> >>>>> Given this, I’ve been assuming that we would need a Hadoop > 3.5.0 release anyway. Based on the current pulse in the Hadoop community, > it seems that the release is expected fairly soon—there’s already > significant momentum from multiple contributors, so it may not require any > additional push from our side. > >> >> >>>>> > >> >> >>>>> That said, let’s wait and see if others have any opinions or > thoughts on this. > >> >> >>>>> > >> >> >>>>> -Ayush > >> >> >>>>> > >> >> >>>>> [1] https://issues.apache.org/jira/browse/HADOOP-19486 > >> >> >>>>> > >> >> >>>>> On Tue, 20 Jan 2026 at 08:37, Butao Zhang < > [email protected]> wrote: > >> >> >>>>>> > >> >> >>>>>> Adding a big +1. > >> >> >>>>>> Hive heavily relies on Hadoop, and I believe as long as > Hadoop is compatible with JDK25, we can upgrade Hive to JDK25 without > hesitation. Based on Kokila’s testing, we may need to wait for the official > release of Hadoop 3.5.0 before we can officially move to JDK25. > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> Thanks, > >> >> >>>>>> Butao Zhang > >> >> >>>>>> > >> >> >>>>>> > >> >> >>>>>> On 2026/01/19 18:27:45 kokila narayanan wrote: > >> >> >>>>>> > Hi all, > >> >> >>>>>> > > >> >> >>>>>> > I did some initial hands-on exploration to see what it > would take to > >> >> >>>>>> > compile Hive with JDK 25. > >> >> >>>>>> > > >> >> >>>>>> > *Compilation:* > >> >> >>>>>> > > >> >> >>>>>> > To compile with *JDK 25* (class file major version 69), > following upgrades > >> >> >>>>>> > are required: > >> >> >>>>>> > > >> >> >>>>>> > - *datanucleus-core*: 6.0.10 → 6.0.11 (includes ASM 9.8 > for Java 25 > >> >> >>>>>> > bytecode support) > >> >> >>>>>> > - *maven-shade-plugin*: 3.6.0 → 3.6.1 (fixes shading > phase failures) > >> >> >>>>>> > - *Error Prone*: Updated to latest version > >> >> >>>>>> > > >> >> >>>>>> > With these changes, compilation succeeds, although there > are still multiple > >> >> >>>>>> > warnings (only errors blocking compilation were addressed). > >> >> >>>>>> > > >> >> >>>>>> > *Testing: * > >> >> >>>>>> > > >> >> >>>>>> > I ran TestDriver.java > >> >> >>>>>> > < > https://github.com/apache/hive/blob/master/ql/src/test/org/apache/hadoop/hive/ql/TestDriver.java > > > >> >> >>>>>> > and > >> >> >>>>>> > encountered the following runtime failure: > >> >> >>>>>> > > >> >> >>>>>> > java.lang.UnsupportedOperationException: getSubject is not > supported > >> >> >>>>>> > > >> >> >>>>>> > at > java.base/javax.security.auth.Subject.getSubject(Subject.java:277) > >> >> >>>>>> > at > org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588) > >> >> >>>>>> > at > org.apache.hadoop.conf.Configuration$Resource.getRestrictParserDefault(Configuration.java:294) > >> >> >>>>>> > at > org.apache.hadoop.conf.Configuration$Resource.<init>(Configuration.java:262) > >> >> >>>>>> > at > org.apache.hadoop.conf.Configuration.addResource(Configuration.java:999) > >> >> >>>>>> > at > org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:6494) > >> >> >>>>>> > at > org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:6438) > >> >> >>>>>> > at > org.apache.hadoop.hive.conf.HiveConfForTest.<init>(HiveConfForTest.java:42) > >> >> >>>>>> > at > org.apache.hadoop.hive.ql.TestDriver.beforeTest(TestDriver.java:38) > >> >> >>>>>> > > >> >> >>>>>> > > >> >> >>>>>> > *Root Cause:* > >> >> >>>>>> > Hadoop 3.4.2 relies on* javax.security.auth.Subject* APIs > that are no > >> >> >>>>>> > longer supported as of Java 23+ due to *JEP 486: > Permanently Disable the > >> >> >>>>>> > Security Manager*. These APIs now throw > *UnsupportedOperationException*. > >> >> >>>>>> > > >> >> >>>>>> > 🔴* Critical Blocker:* Hadoop Incompatibility > >> >> >>>>>> > > >> >> >>>>>> > Specifically, Hadoop 3.4.2 uses the following removed APIs > in > >> >> >>>>>> > UserGroupInformation: > >> >> >>>>>> > - Subject.getSubject(context) > >> >> >>>>>> > < > https://github.com/apache/hadoop/blob/603cd61a56d884baca0f0ee91462f42721d2dd9d/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L588 > >in > >> >> >>>>>> > getCurrentUser() > >> >> >>>>>> > - Subject.doAs(subject, action) > >> >> >>>>>> > < > https://github.com/apache/hadoop/blob/603cd61a56d884baca0f0ee91462f42721d2dd9d/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L1930 > > > >> >> >>>>>> > in > >> >> >>>>>> > the doAs() methods > >> >> >>>>>> > > >> >> >>>>>> > *Impact:* > >> >> >>>>>> > This breaks at runtime during basic configuration > initialization not just > >> >> >>>>>> > in tests but likely in any Hive operation that initializes > HiveConf. > >> >> >>>>>> > > >> >> >>>>>> > *Solution:* > >> >> >>>>>> > The fix is in Hadoop 3.4.3 and 3.5.0, which aren't released > yet. > >> >> >>>>>> > HADOOP JIRA: > https://issues.apache.org/jira/browse/HADOOP-19212 > >> >> >>>>>> > <https://issues.apache.org/jira/browse/HADOOP-19212> > >> >> >>>>>> > > >> >> >>>>>> > We cannot move Hive to JDK 25 until a Hadoop release > including this fix is > >> >> >>>>>> > available. > >> >> >>>>>> > > >> >> >>>>>> > Happy to create a JIRA to track this and address it once > Hadoop has a > >> >> >>>>>> > release that includes this fix. > >> >> >>>>>> > > >> >> >>>>>> > - Kokila > >> >> >>>>>> > > >> >> >>>>>> > On Mon, Jan 19, 2026 at 3:45 PM lisoda <[email protected]> > wrote: > >> >> >>>>>> > > >> >> >>>>>> > > Actually, I think as long as hadoopClient supports JDK > 25+, we can just > >> >> >>>>>> > > reuse the compatibility layer we built for the older > Hadoop versions—it > >> >> >>>>>> > > should work just as well. > >> >> >>>>>> > > > >> >> >>>>>> > > > >> >> >>>>>> > > ---- Replied Message ---- > >> >> >>>>>> > > From Ayush Saxena<[email protected]> <[email protected] > > > >> >> >>>>>> > > Date 01/19/2026 17:19 > >> >> >>>>>> > > To dev<[email protected]> <[email protected]> > >> >> >>>>>> > > Cc > >> >> >>>>>> > > Subject [DISCUSS] Thoughts on JDK 25 (LTS) adoption for > Hive > >> >> >>>>>> > > Hi folks, > >> >> >>>>>> > > > >> >> >>>>>> > > As we know, JDK 25 has been released and is now the > latest LTS. I > >> >> >>>>>> > > wanted to start a discussion on whether and when it makes > sense for > >> >> >>>>>> > > Hive to start chasing it. > >> >> >>>>>> > > > >> >> >>>>>> > > From what I’ve read so far, the release looks solid and > there are > >> >> >>>>>> > > generally positive signals around it, which makes it an > interesting > >> >> >>>>>> > > option to consider. I also had a few offline discussions > last week, > >> >> >>>>>> > > and a common sentiment was that it might be a bit early > to move > >> >> >>>>>> > > aggressively. Our last major shift was to JDK 21, and > much of the > >> >> >>>>>> > > Hadoop ecosystem isn’t moving at the same pace. Given > that, an > >> >> >>>>>> > > immediate jump may be ambitious. > >> >> >>>>>> > > > >> >> >>>>>> > > One possible middle ground could be to acknowledge JDK 25 > as a target, > >> >> >>>>>> > > but not aim for it in the very next release—perhaps > instead in the one > >> >> >>>>>> > > after that, once the ecosystem has had more time to catch > up. > >> >> >>>>>> > > > >> >> >>>>>> > > I haven’t done any hands-on validation yet, so I can’t > comment > >> >> >>>>>> > > concretely on what might break or the level of effort > involved. That > >> >> >>>>>> > > said, from some initial looking around, a potential > prerequisite could > >> >> >>>>>> > > be moving to Hadoop 3.5.0+ (or beyond), which in itself > could be a > >> >> >>>>>> > > blocker. On top of that, there’s the usual question of > how third-party > >> >> >>>>>> > > dependencies—and our own code—would behave. > >> >> >>>>>> > > > >> >> >>>>>> > > Would be good to hear what others think: whether this is > something we > >> >> >>>>>> > > should start planning for now, or keep on the radar and > revisit after > >> >> >>>>>> > > some more ecosystem movement. > >> >> >>>>>> > > > >> >> >>>>>> > > -Ayush > >> >> >>>>>> > > > >> >> >>>>>> > >
