You are not wrong, it is built from source, every night. And every TC run. I don't understand why numa allocator cannot be treated the same. Moreover, it is built using maven, with maven plugin and just needs gcc and libnuma-dev. All of theese are already on TC agents and build are ready. I didn't see any difficulties in building.
But I still don't understand how to organize Anton's proposal. It seems that if we follow this way, we cannot release allocator in 2.13 пн, 6 дек. 2021 г., 23:18 Ilya Kasnacheev <ilya.kasnach...@gmail.com>: > Hello! > > Maybe I am wrong, but ODBC installer is built from source and may be > improved from release to release. > > Regards, > -- > Ilya Kasnacheev > > > пн, 6 дек. 2021 г. в 20:41, Ivan Daschinsky <ivanda...@gmail.com>: > > > Only one reason -- nowadays amost all hardware platforms uses NUMA > > > > Another reason -- there is no any release process of extensions. > > > > > > BTW, apache ignite release is shipped with odbc binary installer for > > windows. And nobody complains about it. > > > > But may be listen to others? > > > > пн, 6 дек. 2021 г., 19:49 Anton Vinogradov <a...@apache.org>: > > > > > Any reason to release the same cpp sources for each release? > > > Any reason to increase the requirements amount for each new release? > > > Any reason to increase release complexity and duration? > > > All answers are "definitely no" > > > > > > What we should do is to release cpp part once and use it as a > dependency. > > > Extensions are a good location. > > > > > > On Mon, Dec 6, 2021 at 3:11 PM Zhenya Stanilovsky > > > <arzamas...@mail.ru.invalid> wrote: > > > > > > > > > > > > > > > +1 with Ivan, let`s store it in core product just because it looks > > > > like inalienable functionality and release cycle of extensions a > little > > > bit > > > > different. > > > > > > > > > > > > > > > > >Anton, I disagree. > > > > > > > > > >1. This should be released with main distro. > > > > >2. This should not be abandoned. > > > > >3. There is not any release process in ignite-extensions. > > > > >4. Everything is working now and working good. > > > > > > > > > > > > > > >So lets do not do this :) > > > > > > > > > >пн, 6 дек. 2021 г. в 14:49, Anton Vinogradov < a...@apache.org >: > > > > > > > > > >> Let's move all GCC-related parts to ignite-extensions, release, > and > > > use > > > > >> them as a maven dependency. > > > > >> > > > > >> > > > > >> On Fri, Dec 3, 2021 at 1:08 PM Ivan Daschinsky < > > ivanda...@gmail.com > > > > > > > > >> wrote: > > > > >> > > > > >> > Ok, TC suite is ready [1]. > > > > >> > If there is no objections, I will merge it soon. > > > > >> > > > > > >> > Possible concerns -- now it is required to install > > build-essentials > > > > and > > > > >> > libnuma-dev in order to build ignite on 64 bit linux. > > > > >> > I suppose that this is not a big deal, but maybe someone will > > > > contradict? > > > > >> > > > > > >> > > > > > >> > [1] -- > > > > >> > > > > > >> > > > > > >> > > > > > > > > > > https://ci.ignite.apache.org/buildConfiguration/IgniteTests24Java8_NumaAllocator/?mode=builds > > > > >> > > > > > >> > чт, 2 дек. 2021 г. в 12:03, Ivan Daschinsky < > ivanda...@gmail.com > > > >: > > > > >> > > > > > >> > > >> Our runs show about 7-10 speedup, > > > > >> > > Sorry, typo 7-10% speedup > > > > >> > > > > > > >> > > чт, 2 дек. 2021 г. в 12:01, Ivan Daschinsky < > > ivanda...@gmail.com > > > > >: > > > > >> > > > > > > >> > >> Andrey, thanks! > > > > >> > >> > > > > >> > >> This allocator can be tested on every NUMA system. > > > > >> > >> Our runs show about 7-10 speedup, if we use allocattor with > > > > >> interleaved > > > > >> > >> strategy + -XX:+UseNUMA. > > > > >> > >> But unfortunately our yardstick benches doesn't use offheap a > > > lot, > > > > >> > >> usually above one Gb. > > > > >> > >> We trying to do more benches with real data and share them, > > > > possibly > > > > >> in > > > > >> > >> meetup. > > > > >> > >> > > > > >> > >> AFAIK, GG lab servers are two-sockets machines, aren't they? > So > > > it > > > > is > > > > >> > >> worth to run benches with a lot data on them, using > > > > >> > >> allocator with interleaved strategy (you can skip specifying > > numa > > > > >> nodes, > > > > >> > >> by default it will use all available) and use -XX:+UseNUMA > jvm > > > > >> > >> flag. > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> чт, 2 дек. 2021 г. в 11:48, Andrey Mashenkov < > > > > >> > andrey.mashen...@gmail.com > > > > >> > >> >: > > > > >> > >> > > > > >> > >>> Ivan, > > > > >> > >>> > > > > >> > >>> Great job. PR looks good. > > > > >> > >>> > > > > >> > >>> This allocator in interleaved mode and passing > `-XX:+UseNUMA` > > > > flag to > > > > >> > jvm > > > > >> > >>> > show promising results on yardstick benches. Technically, > G1 > > > is > > > > >> not a > > > > >> > >>> numa > > > > >> > >>> > aware collector for java versions less than 14, but > > allocation > > > > of > > > > >> > heap > > > > >> > >>> in > > > > >> > >>> > interleaved mode shows good results even on java 11. > > > > >> > >>> > > > > >> > >>> Can you share benchmark results? > > > > >> > >>> I'm not sure I'll have an Optane on my notebook in a > > reasonable > > > > time > > > > >> ;) > > > > >> > >>> > > > > >> > >>> > > > > >> > >>> On Thu, Dec 2, 2021 at 10:41 AM Ivan Daschinsky < > > > > ivanda...@gmail.com > > > > >> > > > > > >> > >>> wrote: > > > > >> > >>> > > > > >> > >>> > Semyon D. and Maks T. -- thanks a lot for review. > > > > >> > >>> > > > > > >> > >>> > BTW, Igniters, I will appreciate all opinions and > feedback. > > > > >> > >>> > > > > > >> > >>> > пн, 29 нояб. 2021 г. в 10:13, Ivan Daschinsky < > > > > >> ivanda...@apache.org > > > > >> > >: > > > > >> > >>> > > > > > >> > >>> > > Hi, igniters! > > > > >> > >>> > > > > > > >> > >>> > > There is not a big secret that nowadays NUMA is quite > > common > > > > in > > > > >> > >>> > > multiprocessor systems. > > > > >> > >>> > > And this memory architecture should be treated in > specific > > > > ways. > > > > >> > >>> > > > > > > >> > >>> > > Support for NUMA is present in many commercial and > > > open-source > > > > >> > >>> products. > > > > >> > >>> > > > > > > >> > >>> > > I've implemented a NUMA aware allocator for Apache > Ignite > > > [1] > > > > >> > >>> > > It is a JNI wrapper around `libnuma` and supports > > different > > > > >> > >>> allocation > > > > >> > >>> > > options. > > > > >> > >>> > > I.e. interleaved, local, interleved_mask and so on. For > > more > > > > >> > >>> information, > > > > >> > >>> > > see > > > > >> > >>> > > [2], [3]. > > > > >> > >>> > > This allocator in interleaved mode and passing > > > `-XX:+UseNUMA` > > > > >> flag > > > > >> > >>> to jvm > > > > >> > >>> > > show promising results on yardstick benches. > Technically, > > G1 > > > > is > > > > >> > not a > > > > >> > >>> > numa > > > > >> > >>> > > aware collector for java versions less than 14, but > > > > allocation of > > > > >> > >>> heap in > > > > >> > >>> > > interleaved mode shows good results even on java 11. > > > > >> > >>> > > > > > > >> > >>> > > Currently, all needed libraries and tools for building > > this > > > > >> module > > > > >> > >>> are > > > > >> > >>> > > available on TC agents > > > > >> > >>> > > setup of specific test suite is in progress [4] > > > > >> > >>> > > > > > > >> > >>> > > So I am asking for a review of my patch. > > > > >> > >>> > > > > > > >> > >>> > > [1] -- > > https://issues.apache.org/jira/browse/IGNITE-15922 > > > > >> > >>> > > [2] -- > https://man7.org/linux/man-pages/man3/numa.3.html > > > > >> > >>> > > [3] -- > > https://man7.org/linux/man-pages/man2/mbind.2.html > > > > >> > >>> > > [4] -- > > https://issues.apache.org/jira/browse/IGNITE-15994 > > > > >> > >>> > > > > > > >> > >>> > > > > > >> > >>> > > > > > >> > >>> > -- > > > > >> > >>> > Sincerely yours, Ivan Daschinskiy > > > > >> > >>> > > > > > >> > >>> > > > > >> > >>> > > > > >> > >>> -- > > > > >> > >>> Best regards, > > > > >> > >>> Andrey V. Mashenkov > > > > >> > >>> > > > > >> > >> > > > > >> > >> > > > > >> > >> -- > > > > >> > >> Sincerely yours, Ivan Daschinskiy > > > > >> > >> > > > > >> > > > > > > >> > > > > > > >> > > -- > > > > >> > > Sincerely yours, Ivan Daschinskiy > > > > >> > > > > > > >> > > > > > >> > > > > > >> > -- > > > > >> > Sincerely yours, Ivan Daschinskiy > > > > >> > > > > > >> > > > > > > > > > >-- > > > > >Sincerely yours, Ivan Daschinskiy > > > > > > > > > > > > > > > > > > > > > >
