Might be worth sort of cheking that, right? So if you open a 32bit TsFile on a 64bit VM, possibly detect that, provide a migration and then open it.
Chris Von: Yuan Tian <[email protected]> Datum: Freitag, 5. Januar 2024 um 13:19 An: [email protected] <[email protected]> Betreff: Re: Problem with sun.misc.Unsafe Hi, I'm not very sure.... If we have different word length for same type in 32 bit and 64 bit, TsFiles written with the 32 bit VM won't be read successfully on the 64 bit version. Best regard, ---------------------- Yuan Tian On Fri, Jan 5, 2024 at 7:22 PM Christofer Dutz <[email protected]> wrote: > Yeah … your version is a lot better :-) > > I just wanted to get the build working and you took care of making it a > good fix. > > However, one question … if someone builds TsFile on a 32 bit VM and then > wants to update that to 64 bit … can the same TsFiles written with the > 32bit VM be read on the 64bit version? > > Chris > > Von: Jialin Qiao <[email protected]> > Datum: Freitag, 5. Januar 2024 um 03:28 > An: [email protected] <[email protected]> > Betreff: Re: Problem with sun.misc.Unsafe > Hi, > > We need to support Java8. > +1 for reusing the constants in RamUsageEstimator > > Jialin Qiao > > Yuan Tian <[email protected]> 于2024年1月5日周五 09:25写道： > > > > Hi Chris, > > > > I think I've got another way to do this, you can see in my pr( > > https://github.com/apache/tsfile/pull/3). I found that all the constants > > have already been calculated and defined in the `RamUsageEstimator` of > > lucene and we can directly use that instead of defining it again. > > > > > > Best regards, > > -------------------------- > > Yuan Tian > > > > On Thu, Jan 4, 2024 at 11:59 PM Christofer Dutz < > [email protected]> > > wrote: > > > > > Hi all, > > > > > > so today we stumbled over a new type of problem (and I thought I had > seen > > > all types). > > > In order to replace the GPL licensed library, direct usages of > > > sun.misc.Unsafe were added for getting the sizes of types and arrays > in JMV > > > memory. > > > > > > The main problem is: If we want to compile true Java 8 libraries with > any > > > java version greater than 8, we need to set the “release” config > option in > > > the maven plugin. This tells the compiler to cross compile for that > java > > > version. It does this by checking an index file for that given java > > > version. Problem seems to be that for java 8 the sun.misc.Unsafe is not > > > included in this index and the compiler therefore fails. > > > > > > Now there are multiple options to resolve this issue: > > > > > > We could not use the “release” option, but that could produce Jars that > > > have the right java class version, but can’t work on Java 8 because of > > > missing types. (Sub-Ideal) > > > > > > We could create a library, that wraps these calls and which is > compiled on > > > Java 8. So Instead of using the direct calls, we’d use that library > > > instead. However this is just tricking the compiler and if in any > future > > > version of Java this file is moved/removed, things will explode. > > > > > > As a third option, I see that we could simply define the constants to > > > fixed values. These values are really not going to change and I will > do a > > > check with what the values are on various JDKs and OSes. This option > has > > > the advantage, that if any of these values would change in future > versions, > > > the old data-files would not be corrupt. And if we want to Provide > TsFiles > > > on other systems than Java, we probably need constants anyway. > > > > > > I have therefore defined these constants in order to get the build > working > > > again and we’ll continue this discussion to which path we want to go. > > > > > > > > > Chris > > > >
