Gian, If I were to supply you with a link to a snapshot or a Release Candidate version of the datasketches jar (and Memory jar) could you test in your environment under JDK 11 and see if it works?
Lee On Mon, Oct 14, 2019 at 9:19 PM leerho <lee...@gmail.com> wrote: > Holy Cow! I have never seen such a complex travis config! > > On Mon, Oct 14, 2019 at 6:54 PM Gian Merlino <g...@apache.org> wrote: > >> I should have linked Druid's Travis config file for reference. Here it is: >> https://github.com/apache/incubator-druid/blob/master/.travis.yml >> >> On Mon, Oct 14, 2019 at 6:53 PM Gian Merlino <g...@apache.org> wrote: >> >> > > So what exactly is the requirement? Our DataSketches code does >> compile >> > > under JDK8 and we have lots of customers and users using JDK 8. >> > >> > It depends on who you ask, I guess :) >> > >> > From Druid's perspective, we aren't going to be compiling your code, >> only >> > running it. So the only requirement we would ask for is that the code >> > _runs_ under a Java 11 runtime. Being able to compile under Java 11 is >> good >> > for future-proofing your code, though. It is starting to become more >> > prevalent. >> > >> > > It sounds like (as far as Unsafe is concerned) converting all the >> direct >> > > unsafe.blah() calls to static MethodHandle.invoke calls is what is >> > required. >> > > >> > > Is this correct? >> > >> > That is certainly part of it, I'm not 100% sure if that's all that's >> > needed. Btw, CI can help verify. In Druid's Travis config, we run our >> tests >> > on both JDK 8 and JDK 11. In order to make this automated testing >> easier, >> > we chose to do the work needed to compile Druid on JDK 11, even though >> we >> > still compile release builds using JDK 8 only. (If we hadn't done this >> > work, then we'd have to compile on 8 and test on 11, which Travis >> doesn't >> > easily support.) >> > >> > On Mon, Oct 14, 2019 at 5:51 PM leerho <lee...@gmail.com> wrote: >> > >> >> Thanks, Gian, this is VERY helpful! >> >> >> >> From my cursory glance, it appears that UnsafeUtil acquires Unsafe and >> >> Unsafe-class the usual way using reflection (I didn't think this would >> >> work >> >> with 9+ ), and then defines some method-handles that the other classes >> >> use. >> >> >> >> Will this approach allow code to compile under JDK11 and JDK8? >> >> >> >> It also seems that compiling with JDK11 is still a work in progress as >> the >> >> pom.xml clearly specifies JDK8. >> >> >> >> So what exactly is the requirement? Our DataSketches code does compile >> >> under JDK8 and we have lots of customers and users using JDK 8. >> >> >> >> It sounds like (as far as Unsafe is concerned) converting all the >> direct >> >> unsafe.blah() calls to static MethodHandle.invoke calls is what is >> >> required. >> >> >> >> Is this correct? >> >> >> >> Lee. >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> On Mon, Oct 14, 2019 at 2:07 PM Gian Merlino <g...@apache.org> wrote: >> >> >> >> > And nobody should be using the Oracle JDK without a commercial >> >> relationship >> >> > with Oracle, its license is a nightmare. >> >> > >> >> > On Mon, Oct 14, 2019 at 2:04 PM Gian Merlino <g...@apache.org> >> wrote: >> >> > >> >> > > Oracle and OpenJDK builds behave the same as each other now. The >> >> > > differences are in terms of licensing, length of support, and >> release >> >> > > cadence. IMO most Java devs should be using one of the third-party >> >> > OpenJDK >> >> > > distributions (e.g. Corretto, Zulu, AdoptOpenJDK) rather than >> vanilla >> >> > > OpenJDK these days, because of the new OpenJDK policy that public >> >> updates >> >> > > will cease for major versions after 6 months. >> >> > > >> >> > > On Mon, Oct 14, 2019 at 11:07 AM leerho <lee...@gmail.com> wrote: >> >> > > >> >> > >> Thanks, Gian, >> >> > >> >> >> > >> Your references are helpful and I am studying them. >> >> > >> >> >> > >> I didn't see any references to OpenJDK vs Oracle's JDKs. >> >> > >> >> >> > >> Are there any differences in the way Unsafe calls (or other hidden >> >> > >> classes) >> >> > >> are handled by OpenJDK? Or is this a "don't care" as far as >> Druid is >> >> > >> concerned? Are the version numbers between Oracle and OpenJDK >> always >> >> > >> aligned? >> >> > >> >> >> > >> >> >> > >> Lee. >> >> > >> >> >> > >> >> >> > >> >> >> > >> On Fri, Oct 11, 2019 at 5:45 PM Gian Merlino <g...@apache.org> >> >> wrote: >> >> > >> >> >> > >> > You didn't address this message to me, but in Druid most of the >> >> work >> >> > for >> >> > >> > Java 9+ compatibility is mentioned in this master issue, which >> you >> >> > might >> >> > >> > find helpful: >> >> https://github.com/apache/incubator-druid/issues/5589. >> >> > >> > Skimming the list, these might be particularly relevant to >> >> > DataSketches: >> >> > >> > >> >> > >> > - https://github.com/apache/incubator-druid/pull/7487 (cleaner >> >> > >> operations) >> >> > >> > - https://github.com/apache/incubator-druid/pull/7466 >> (ByteBuffer >> >> > unmap >> >> > >> > operation) >> >> > >> > - https://github.com/apache/incubator-druid/pull/7576 (Remove >> >> direct >> >> > >> > references to Unsafe) >> >> > >> > >> >> > >> > Btw, it would be better, I think, to address emails to the list >> at >> >> > >> large. >> >> > >> > It encourages more people to participate. If Roman is a >> subscriber >> >> he >> >> > >> would >> >> > >> > get a copy anyway. If not, you could encourage him to subscribe. >> >> > >> > >> >> > >> > On Fri, Oct 11, 2019 at 3:13 PM leerho <lee...@gmail.com> >> wrote: >> >> > >> > >> >> > >> > > Roman, >> >> > >> > > >> >> > >> > > I hope you are doing well. >> >> > >> > > >> >> > >> > > I really appreciate the contributions you made to our Memory >> >> > >> component! >> >> > >> > > >> >> > >> > > Given that Druid has made its code base compatible with >> OpenJDK >> >> 11, >> >> > we >> >> > >> > > could use your help in what changes do we need to make to make >> >> that >> >> > >> > happen >> >> > >> > > for DataSketches. As I recall, our DataSketches library was >> not >> >> the >> >> > >> only >> >> > >> > > Druid dependency that took advantage of Unsafe. So I assume >> you >> >> > have >> >> > >> > been >> >> > >> > > down this path :) >> >> > >> > > >> >> > >> > > Currently, I have a check in the static initializer in >> >> > >> Memory/UnsafeUtil: >> >> > >> > > parseJavaVersion(...) that checks the string returned from >> >> > >> > > System.getProperty("java.version"). If that string does not >> >> contain >> >> > >> > "1.8" >> >> > >> > > or "8" it will throw an error. Given that we have not tested >> >> with >> >> > >> JDK 9, >> >> > >> > > 10, 11 or 12, I felt it was safer to explicitly throw rather >> than >> >> > >> having >> >> > >> > > the users experience some weird failure later that may be >> >> difficult >> >> > to >> >> > >> > > debug. >> >> > >> > > >> >> > >> > > With only one minor exception (which we could easily fix) the >> >> Memory >> >> > >> > > component is the only place where Unsafe is used. However, >> we do >> >> > use >> >> > >> > > reflection to gain internal access to a number of other >> classes >> >> > >> including >> >> > >> > > ByteBuffer, Cleaner, and Bits. >> >> > >> > > >> >> > >> > > Looking forward to hearing from you. >> >> > >> > > >> >> > >> > > Cheers, >> >> > >> > > >> >> > >> > > Lee. >> >> > >> > > >> >> > >> > >> >> > >> >> >> > > >> >> > >> >> >> > >> >
