> I dispute the Eigen "seems effectively abandoned" claim. If you visit the repo, there are commits listed as recently as 6 days ago. In my opinion the core devs do sterling work.
Sure, that's why I said "seems". But if you take number of releases as the most visible proxy metric, then that claim isn't overly far-fetched. I didn't want to imply that [the core] developers don't do great work -- I'm sure they do. But the amount of maintenance that currently goes into Eigen is not enough, in my view. Ecosystem health and participation outside of a core group of developers has a lot to do with that. Cheers, Michael On Wed, May 6, 2020 at 9:36 AM Wood, Tobias <[email protected]> wrote: > Hello, > > > > I dispute the Eigen "seems effectively abandoned" claim. If you visit the > repo, there are commits listed as recently as 6 days ago. In my opinion the > core devs do sterling work. > > > > However, I'm already on record on this list as thinking Eigen should be > updated to a newer C++ standard. I agree with the central point below that > old versions do not disappear. ITK (https://itk.org) is a codebase of > similar age to Eigen, and has moved up to C++11 over the last couple of > years. They have seen big improvements in performance and usability. This > change happened fairly organically - they decided that version 5 would be > the first to require C++11, and then gradually over a year or two and a > couple of minor versions different parts of the codebase were slowly > updated to use lambdas etc. > > > > At the risk of being the annoying user who requests too much - I can't > find it right now but I have memories of a mailing list discussion from > several years ago where Gael proposed that the next major version change in > Eigen would merge the Tensor and main Eigen codebases, such that Array<> > and Matrix<> became specialisations of a rank 2 Tensor. I am sure that is > also a huge amount of work but that is what I would truly like to see in > Eigen 4, and would require the switch to C++11 as a minimum. The ability to > store a rank 3 Tensor and then natively chip slices out to an Array or > Matrix would be very helpful. > > > > Best wishes, > > Toby > > > > *From: *Michael Hofmann <[email protected]> > *Reply to: *"[email protected]" <[email protected]> > *Date: *Wednesday, 6 May 2020 at 08:19 > *To: *"[email protected]" <[email protected]> > *Subject: *Re: [eigen] Dropping C++14 compatibility discussion > > > > There are simple and established solutions to the problem of having an old > system compiler or standard libraries on a cluster-type computing system. > The Environment Modules system (http://modules.sourceforge.net/ > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmodules.sourceforge.net%2F&data=01%7C01%7Ctobias.wood%40kcl.ac.uk%7Cea96708cf04445cde91508d7f18de0a9%7C8370cf1416f34c16b83c724071654356%7C0&sdata=q4WFYFxcdtb2lov7k%2FTwNscBGworE2iXN67df69Ufu0%3D&reserved=0>) > is one of them. Local compiler installs or even Docker are others. Any > sysadmin worth their salt can solve these issues for users quite easily, if > they just wanted to. (They could additionally upgrade the system, if they > just wanted to.) > > > > However, conservatism certainly has value. Particularly when you > advertise yourself as a production library (not a research or proof of > concept toy). > > Eigen development is slow. The code has reached a very stable state. > The last major release was in 2018. These facts are probably good for most > users of Eigen. > > I heavily disagree. In my opinion, there is absolutely no value in this > kind of conservatism. Instead, it *only* harms. > > Conservatism in library development has little effect on "production > quality". In fact, I'd argue that libraries that choose to move faster are > often better, including for production use cases, since they have > judiciously cleaned up their cruft, thus attract a larger developer and > user base and have a healthier ecosystem. It is proven that in software > development, small lead and cycle times and frequent "deploys", be it > upstream or downstream, in libraries or services, are strongly beneficial. > At least part of the reason why Eigen development has slowed to a halt is > said refusal to move with the times, at the right time (i.e. not six years > too late). > > We have a classic case of "code rot" here, due to changing requirements > from the user base over time. Both the internals of Eigen have become > harder to maintain (things are not expressed as simply and elegantly as > they could be) and the externals harder to use (crufty interfaces, slow > compilation speeds, myriads of spurious warnings on newer compilers, etc.). > > Because nothing happened for years due to the conservative governance > model, modernizing the code has now accumulated to a huge task, and it's on > a developer base that has already shrunken quite a bit, judging from the > average level of activity on this mailing list. This greatly exacerbates > the problem. Continuous and incremental modernization in a timely(!) manner > is the best way to prevent this scenario from happening. This includes (but > is not limited to) moving to new C++ standards around the time they are > introduced. > > It doesn't really matter whether one calls a newer version 3.x or 4.x -- > these are just semantics in the end. No matter how a project is governed > (SemVer or not, level of promised API or ABI stability, etc.), the > fundamental insight has to be that *old versions do not go away* and can > still be used by anyone who wishes to do so. Conservative users may choose > to upgrade more rarely, at the cost of much larger pain when doing so. (A > model that I do not believe in.) Most users should be encouraged to migrate > to the latest released version quickly, and to be part of the continuous, > incremental maintenance cycle. > > I have been an Eigen user and defender for years, but I cannot recommend > this library anymore to others without giving huge caveats ("outdated", > "seems effectively abandoned"). A strong commitment to modernity and moving > with the times by the maintainers is the best and maybe only way to bring > back users and thus developers. As part of this, I do hope that Eigen > governance commits to moving to C++17 in the very near future. > > Cheers, > Michael > > > > On Wed, May 6, 2020 at 5:53 AM Rob McDonald <[email protected]> > wrote: > > I don't have access to the machine, so I can't answer the detailed > questions. > > > > My application isn't super intensive -- but it can be used to generate > input files for much more expensive computations and optimization studies. > Consequently, it sometimes gets run on big iron. > > > > My user who recently had problems because my app developed a dependency on > std::regexp -- which isn't really supported by gcc 4.8. The problem is > less the compiler and more the standard libraries installed on the > machine. He is not building on the big machine -- he builds on a local vm > on a laptop and then transfers the binary up. They also prefer to do their > local setup on images that are as similar as possible to the computation > environment. Perhaps they could set up local compilers and libraries, but > that is a much bigger hassle than using system installed standard libraries. > > > > The machine is not at TACC. It has nearly 250k cores and 1PB memory. It > was installed when RHEL 7 was brand new. I have no idea how many cores the > users are using -- a small fraction no doubt. > > > > Rob > > > > > > On Tue, May 5, 2020 at 8:01 PM Jeff Hammond <[email protected]> > wrote: > > Is the primary compiler on said supercomputer really /usr/bin/g++? How > many nodes are users running on when they build apps with this compiler? > > > > My guess is most of the users of that system use mpicxx or something > similar from modules that provides a trivial solution to this problem. If > you’re referring to TACC, I’m sure they’ll recommend a bunch of compilers > besides GCC-4.8. > > > > RHEL has devtoolset for a reason. All major HPC systems have modules for a > dozen or more compilers that don’t live in /usr. Eigen shouldn’t hold > itself back for people who refuse to help themselves. > > > > Jeff > > > > On Tue, May 5, 2020 at 3:21 PM Rob McDonald <[email protected]> > wrote: > > I certainly understand the need to move forward. However, conservatism > certainly has value. Particularly when you advertise yourself as a > production library (not a research or proof of concept toy). > > > > Just as an example, the main national supercomputer for a major US > government research agency is running RHEL 7. That means gcc 4.8.x and the > standard libraries that go with it. That means C++11 and most of the stuff > that goes with it -- but not everything. > > > > RHEL 8 was only released 364 days ago (depending on your time zone). It > is certainly not unreasonable for installed high value production computing > assets like the above machine to still be running RHEL 7. While it would > be great if RHEL moved compilers forward more quickly -- that isn't how it > works. > > > > The fact is -- Eigen development is slow. The code has reached a very > stable state. The last major release was in 2018. These facts are > probably good for most users of Eigen. > > > > If permission to jump to some advanced version means someone is going to > go through and wholesale clean up the code and move it forward -- who is > that someone? Where is this development energy coming from? Users making > a request doesn't mean there is anyone to get the job done. > > > > Eigen 3.x should remain pretty much the way it is. I hope/assume that any > big jump like this would start by creating an Eigen 4.x branch/fork on > GitHub and moving forward from there -- hopefully the energetic people > calling for this change are the ones who will step forward to do the work. > > > > Rob > > > > > > > > On Tue, May 5, 2020 at 3:01 PM Janek Kozicki (yade) < > [email protected]> wrote: > > I suggest to move forward, and use C++17. There is no reason to > stick with old libraries. > > If you prefer to keep for a year or two the ability to compile with > C++14, then place such obsolete code inside some #ifdef #else #endif, > so that more efficient code is generated with C++17 and less efficient > code when someone insists on compiling with C++14. > > In yade[1] we have moved to C++17 when building packages for ubuntu > focal 20.04 or debian buster or bullseye. The C++14 is used only when > building packages for older systems such as ubuntu xenial 16.04. > > Yade can compile with eigen using C++17 without any problems. Which > is good. For example the coinor library had some problems with C++17: > and older version worked only with C++17. > > best regards > Janek > > [1] https://yade-dem.org/doc/ > <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fyade-dem.org%2Fdoc%2F&data=01%7C01%7Ctobias.wood%40kcl.ac.uk%7Cea96708cf04445cde91508d7f18de0a9%7C8370cf1416f34c16b83c724071654356%7C0&sdata=UnCsj8Ztwsnp%2FwY5dwoSYGoJLK8wI7YeeTURe4xjd1Q%3D&reserved=0> > and https://gitlab.com/yade-dev/trunk > <https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.com%2Fyade-dev%2Ftrunk&data=01%7C01%7Ctobias.wood%40kcl.ac.uk%7Cea96708cf04445cde91508d7f18de0a9%7C8370cf1416f34c16b83c724071654356%7C0&sdata=5TmpODh2e%2B97dXtKAZFTzq3lFI%2FdzgibyTFupv7WXNQ%3D&reserved=0> > > > > Rasmus Munk Larsen said: (by the date of Tue, 5 May 2020 14:14:19 > -0700) > > > I agree with you, but it seems the maintainers are very busy with other > > projects, and I am guessing they do not have time for major initiatives > to > > clean up Eigen. We have dropped support for c++03 in the Tensor library, > > but a lot of things could be cleaned up if we more fully embraced c++11 > or > > c++14. At this point, I only see this happening slowly, or if somebody > from > > the open source community is willing to help. > > > > On Tue, May 5, 2020 at 2:05 PM Patrik Huber <[email protected]> > wrote: > > > > > Hi all, > > > > > > I'm a quite sad to see this post got no replies. I can only add that I > > > fully agree and think along the very same lines. I would love to see > Eigen > > > move forward, move to C++14 in the 3.5 release, and drop the old cruft. > > > It's 2020 now, 6 years after C++14. > > > > > > Best wishes, > > > Patrik > > > > > > On Wed, 8 Apr 2020 at 17:20, Martin Beeger <[email protected]> > > > wrote: > > > > > >> Hallo! > > >> > > >> This is continuation of the discussion from last year about > > >> compatibility, but I wanted to bring it up again, because I have some > > >> new data from my experience. > > >> > > >> Our company uses Eigen extensively and has a larger ecosystem built on > > >> it, as do many of the voices on this list. We are in the embedded > world, > > >> so we are often hindered to adapt quickly to new tools, very much like > > >> the HPC community has been. > > >> But we managed to move to C++14 with our codebase in 2019. When doing > > >> that, we somewhat monitored performance and compile time during the > > >> adaption. > > >> > > >> When we starting compiling our C++98 codebase as is with C+14, > > >> performance already went up slightly. C++14 does silently move or > elide > > >> copies, and modern compiler have better optimizers, and the quality of > > >> implementation of STL types got better. So if you are at all bound by > > >> performance of STL types, I would really recommend using a new > compiler > > >> in C++14 mode even with your old code. > > >> > > >> This was the obvious part. Another part was less obvious. We profiled > > >> compile time, which didn't change much after the switch and started to > > >> look into stuff which was expensive to compile. And then selectively > in > > >> places, where the compile time was bad we simplified the code using > > >> C++14 features to improve compile time. This allowed us with very > > >> localized changes to cut our compile time by almost in half, while, at > > >> the same time, making the code in question often a lot simpler and new > > >> features (e.g. for performance improvements) easier to implement. > > >> > > >> This is also the experience which was observed with other template > heavy > > >> codebases like boost::mpl in comparison to hana and other stuff. The > > >> gains in simplicity and compile time, especially from constexpr and > > >> lambda features are not minor. They can often cut your code in half > and > > >> more than double the compile time. > > >> > > >> The problem is, I am at a point where its hard to do much more to > > >> improve the compile time of my codebase significantly, because most of > > >> the compile time is brought in by 2 libraries: boost.test and Eigen. > We > > >> will most likely at some point abandon boost.test due to this, like > many > > >> others have already (which slowed the development and improvement of > > >> boost.test further, while the alternatives got better and getting into > > >> this downwards spiral). I would be vary happy if I am not forced to > > >> abandon Eigen after the great 10 years we had with this library. In > > >> order to avoid this I pulled a lot of tricks like explicit template > > >> instatiations, tricks to reduce includes, even pimpl-like > encapsulation > > >> at performance cost to isolate from the problem, but that gets you > only > > >> so far. > > >> > > >> I may well be that my use case is special, but I strongly assume that > > >> new users which today want to adopt Eigen and have to look into its > > >> internals (as you inevitable need to do at some points), will see how > > >> its written and quickly run for alternatives. This amount of macros, > > >> boilerplate and similar stuff will be an argument against this great > > >> library some day and this day may already have come. > > >> > > >> The important part is here: Eigen compile time and internal expression > > >> template engine code readability it was great by 2010s standards, it > was > > >> ok by 2015 standards, it is borderline by 2020 standards, and unless > > >> something changes, it will be unacceptable by 2025 standards, unless > the > > >> Eigen library moves along. > > >> > > >> As C++ users, we do care about backwards compatibility greatly and > that > > >> is even good for me, but we should not go the C way and care about too > > >> ancient compilers. The C++ comittee doesn't (that why int is required > to > > >> be 2s complement in C++20), so Eigen library maintainers IMHO should > > >> follow. > > >> > > >> What are the chances to get Eigen 3.4 out of the door with C++98 > support > > >> and drop it on the devel branch afterwards and jump to C++14? > > >> What is the Eigen promises about how old yours compilers may be? > > >> Can we explicitly agree on a statement like: We vow to support up to 3 > > >> year old compilers (or 5 years)? > > >> > > >> If we could agree on clear and conservative rules like we will go 3 > > >> years back or 5 years back and state these on the Eigen front page, > > >> Eigen user may look at our codebase and be much more willing to accept > > >> older standards code, knowing that it will improve over time and that > > >> the user gets some useful guarantees about the future in return. > > >> There is agrument to be made that if you use a 5+ years old compiler, > > >> you really do not care about performance, something Eigen uses > generally > > >> care about. So you are not in a targeted user group of Eigen. Little > HPC > > >> clusters do not at all offer any way to install a more recent > compilers > > >> that 5 years (this has changed a lot from a decade ago) and even in > the > > >> embedded world, vendors tend to drop support or upgrade for platforms > > >> with more than 5 years old toolchains too (this has also very much > > >> changed from 10 years ago). > > >> > > >> Eigen users should be able to get a clear answer on the question when > we > > >> drop C++98 (of if). That belongs on the front page IMHO. > > >> > > >> Kind regards, > > >> Martin > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > >> > > > -- > -- > Janek Kozicki, PhD. DSc. Arch. Assoc. Prof. > Gdańsk University of Technology > Faculty of Applied Physics and Mathematics > Department of Theoretical Physics and Quantum Information > -- > http://yade-dem.org/ > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fyade-dem.org%2F&data=01%7C01%7Ctobias.wood%40kcl.ac.uk%7Cea96708cf04445cde91508d7f18de0a9%7C8370cf1416f34c16b83c724071654356%7C0&sdata=6UPRmF3HaF1Kuj8W87FcGXm%2F3QBpuAeZlB6IjxspAzU%3D&reserved=0> > http://pg.edu.pl/jkozicki > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpg.edu.pl%2Fjkozicki&data=01%7C01%7Ctobias.wood%40kcl.ac.uk%7Cea96708cf04445cde91508d7f18de0a9%7C8370cf1416f34c16b83c724071654356%7C0&sdata=t0646E5OQ8NcpJ4Qui7jQse%2BXdVyU4nUYhV1KZ6wjwE%3D&reserved=0> > (click English flag on top right) > > -- > > Jeff Hammond > [email protected] > http://jeffhammond.github.io/ > <https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fjeffhammond.github.io%2F&data=01%7C01%7Ctobias.wood%40kcl.ac.uk%7Cea96708cf04445cde91508d7f18de0a9%7C8370cf1416f34c16b83c724071654356%7C0&sdata=cxzngE78zwQo5ZDgr3r6QTNH3IA6KNNnk%2BG7jkc8OIQ%3D&reserved=0> > >
