Re: [Yade-dev] High precision Real and minieigen
Yes, I think that we can stay with boost::python. But we need to put minieigen sources back into yade. best regards Janek Anton Gladky said: (by the date of Sun, 23 Feb 2020 22:08:47 +0100) > Hi Janek, > > thanks a lot for this quick tests and results! Really impressive! > > As far as I see, there is not benefit for us to replace boost::python, > at least right now? > > Regards > > Anton > > Am So., 23. Feb. 2020 um 16:02 Uhr schrieb Janek Kozicki (yade) > : > > > > > Another note: I just realized that the compilation benchmarks might > > > have not been fair, because I think that I added few more > > > registrations (like Vector2c, or sth. like that) while doing pybind > > > migration. I will compare again without these extra registrations. > > > > OK, I have compared with the exact same registrations. And it seems > > that pybind has many advantages, but not the one that it is compiling > > faster. See: > > > > pybind without debug info; call `ccache --clear` between every invocation. > > -O3 -j 1 -O3 -j 10 > > wall clock time: 2:41.14 wall clock time: 0:40.67 > > wall clock time: 2:46.56 wall clock time: 0:41.28 > > wall clock time: 2:47.42 wall clock time: 0:42.54 > > wall clock time: 2:43.79 wall clock time: 0:44.70 > > > > -O1 -j 1 -O1 -j 10 > > wall clock time: 2:09.88 wall clock time: 0:35.00 > > wall clock time: 2:10.78 wall clock time: 0:33.46 > > wall clock time: 2:08.19 wall clock time: 0:32.62 > > wall clock time: 2:10.10 wall clock time: 0:32.73 > > > > > > boost::python without debug info; call `ccache --clear` between every > > invocation. > > -O3 -j 1 -O3 -j 10 > > wall clock time: 2:16.83 wall clock time: 0:33.88 > > wall clock time: 2:15.96 wall clock time: 0:32.84 > > wall clock time: 2:16.00 wall clock time: 0:34.85 > > wall clock time: 2:16.52 wall clock time: 0:32.32 > > > > -O1 -j 1 -O1 -j 10 > > wall clock time: 2:01.48 wall clock time: 0:28.56 > > wall clock time: 2:00.70 wall clock time: 0:29.15 > > wall clock time: 2:01.29 wall clock time: 0:28.05 > > wall clock time: 2:00.56 wall clock time: 0:28.35 > > > > The main pybind disadvantage is unstable API. I don't like writing > > extra code to support older linux distributions. > > > > > > Also I think that if while copying minieigen into yade we did some > > extra rebalancing between the .cpp files, so that in each file there's > > about the same amount of registrations performed, then we could > > reduce (parallel) compilation time to maybe 25 or 20 seconds. > > > > > > If you wanted to compile and try yourself, the comparison was between > > branches master and tryPybind. To see how to compile see .gitlab-ci.yml, > > I'm sorry that it's a bit messy. I was never good at using build > > systems ;) I just wanted parallel build quickly. That's why I wrote > > such strange makefile. > > > > best regards > > Janek > > > > > > > > > Another comparison note: > > > > > > 8. boost::python is much more picky about the order of registered > > >functions, and sometimes does not work if the order is "wrong". > > >pybind always resolves the function overloads correctly. > > > > -- > > -- > > Janek Kozicki, PhD. DSc. Arch. Assoc. Prof. > > Gdańsk University of Technology > > Faculty of Applied Physics and Mathematics > > Department of Theoretical Physics and Quantum Information > > -- > > http://yade-dem.org/ > > http://pg.edu.pl/jkozicki (click English flag on top right) -- -- Janek Kozicki, PhD. DSc. Arch. Assoc. Prof. Gdańsk University of Technology Faculty of Applied Physics and Mathematics Department of Theoretical Physics and Quantum Information -- http://yade-dem.org/ http://pg.edu.pl/jkozicki (click English flag on top right) ___ Mailing list: https://launchpad.net/~yade-dev Post to : yade-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-dev More help : https://help.launchpad.net/ListHelp
Re: [Yade-dev] High precision Real and minieigen
Hi Janek, thanks for pybind feedback. It is intriguing. It was definitely after me mentionning compile times with boost for c++ / python, at a workshop, that someone reached me and advised about pybind. Rumor? Bruno Le dim. 23 févr. 2020 16:03, Janek Kozicki (yade) a écrit : > > Another note: I just realized that the compilation benchmarks might > > have not been fair, because I think that I added few more > > registrations (like Vector2c, or sth. like that) while doing pybind > > migration. I will compare again without these extra registrations. > > OK, I have compared with the exact same registrations. And it seems > that pybind has many advantages, but not the one that it is compiling > faster. See: > > pybind without debug info; call `ccache --clear` between every invocation. > -O3 -j 1 -O3 -j 10 > wall clock time: 2:41.14 wall clock time: 0:40.67 > wall clock time: 2:46.56 wall clock time: 0:41.28 > wall clock time: 2:47.42 wall clock time: 0:42.54 > wall clock time: 2:43.79 wall clock time: 0:44.70 > > -O1 -j 1 -O1 -j 10 > wall clock time: 2:09.88 wall clock time: 0:35.00 > wall clock time: 2:10.78 wall clock time: 0:33.46 > wall clock time: 2:08.19 wall clock time: 0:32.62 > wall clock time: 2:10.10 wall clock time: 0:32.73 > > > > boost::python without debug info; call `ccache --clear` between every > invocation. > -O3 -j 1 -O3 -j 10 > wall clock time: 2:16.83 wall clock time: 0:33.88 > wall clock time: 2:15.96 wall clock time: 0:32.84 > wall clock time: 2:16.00 wall clock time: 0:34.85 > wall clock time: 2:16.52 wall clock time: 0:32.32 > > -O1 -j 1 -O1 -j 10 > wall clock time: 2:01.48 wall clock time: 0:28.56 > wall clock time: 2:00.70 wall clock time: 0:29.15 > wall clock time: 2:01.29 wall clock time: 0:28.05 > wall clock time: 2:00.56 wall clock time: 0:28.35 > > The main pybind disadvantage is unstable API. I don't like writing > extra code to support older linux distributions. > > > Also I think that if while copying minieigen into yade we did some > extra rebalancing between the .cpp files, so that in each file there's > about the same amount of registrations performed, then we could > reduce (parallel) compilation time to maybe 25 or 20 seconds. > > > If you wanted to compile and try yourself, the comparison was between > branches master and tryPybind. To see how to compile see .gitlab-ci.yml, > I'm sorry that it's a bit messy. I was never good at using build > systems ;) I just wanted parallel build quickly. That's why I wrote > such strange makefile. > > best regards > Janek > > > > > Another comparison note: > > > > 8. boost::python is much more picky about the order of registered > >functions, and sometimes does not work if the order is "wrong". > >pybind always resolves the function overloads correctly. > > -- > -- > Janek Kozicki, PhD. DSc. Arch. Assoc. Prof. > Gdańsk University of Technology > Faculty of Applied Physics and Mathematics > Department of Theoretical Physics and Quantum Information > -- > http://yade-dem.org/ > http://pg.edu.pl/jkozicki (click English flag on top right) > > ___ > Mailing list: https://launchpad.net/~yade-dev > Post to : yade-dev@lists.launchpad.net > Unsubscribe : https://launchpad.net/~yade-dev > More help : https://help.launchpad.net/ListHelp > > > ___ Mailing list: https://launchpad.net/~yade-dev Post to : yade-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-dev More help : https://help.launchpad.net/ListHelp
Re: [Yade-dev] High precision Real and minieigen
> Another note: I just realized that the compilation benchmarks might > have not been fair, because I think that I added few more > registrations (like Vector2c, or sth. like that) while doing pybind > migration. I will compare again without these extra registrations. OK, I have compared with the exact same registrations. And it seems that pybind has many advantages, but not the one that it is compiling faster. See: pybind without debug info; call `ccache --clear` between every invocation. -O3 -j 1 -O3 -j 10 wall clock time: 2:41.14 wall clock time: 0:40.67 wall clock time: 2:46.56 wall clock time: 0:41.28 wall clock time: 2:47.42 wall clock time: 0:42.54 wall clock time: 2:43.79 wall clock time: 0:44.70 -O1 -j 1 -O1 -j 10 wall clock time: 2:09.88 wall clock time: 0:35.00 wall clock time: 2:10.78 wall clock time: 0:33.46 wall clock time: 2:08.19 wall clock time: 0:32.62 wall clock time: 2:10.10 wall clock time: 0:32.73 boost::python without debug info; call `ccache --clear` between every invocation. -O3 -j 1 -O3 -j 10 wall clock time: 2:16.83 wall clock time: 0:33.88 wall clock time: 2:15.96 wall clock time: 0:32.84 wall clock time: 2:16.00 wall clock time: 0:34.85 wall clock time: 2:16.52 wall clock time: 0:32.32 -O1 -j 1 -O1 -j 10 wall clock time: 2:01.48 wall clock time: 0:28.56 wall clock time: 2:00.70 wall clock time: 0:29.15 wall clock time: 2:01.29 wall clock time: 0:28.05 wall clock time: 2:00.56 wall clock time: 0:28.35 The main pybind disadvantage is unstable API. I don't like writing extra code to support older linux distributions. Also I think that if while copying minieigen into yade we did some extra rebalancing between the .cpp files, so that in each file there's about the same amount of registrations performed, then we could reduce (parallel) compilation time to maybe 25 or 20 seconds. If you wanted to compile and try yourself, the comparison was between branches master and tryPybind. To see how to compile see .gitlab-ci.yml, I'm sorry that it's a bit messy. I was never good at using build systems ;) I just wanted parallel build quickly. That's why I wrote such strange makefile. best regards Janek > Another comparison note: > > 8. boost::python is much more picky about the order of registered >functions, and sometimes does not work if the order is "wrong". >pybind always resolves the function overloads correctly. -- -- Janek Kozicki, PhD. DSc. Arch. Assoc. Prof. Gdańsk University of Technology Faculty of Applied Physics and Mathematics Department of Theoretical Physics and Quantum Information -- http://yade-dem.org/ http://pg.edu.pl/jkozicki (click English flag on top right) ___ Mailing list: https://launchpad.net/~yade-dev Post to : yade-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-dev More help : https://help.launchpad.net/ListHelp
Re: [Yade-dev] High precision Real and minieigen
Hi Anton > sorry for delays, "real" life is taking much more time when we get older :) please don't worry. I really appreciate all your hard work. I got a little carried away, I'm sorry. > > * I would rather avoid pybind11 switch in a rush, better give it a few > > months > Maybe I was not clear, pybind11 is just a possible option. If we > really decide to switch to it, we need to evaluate, whether it is > suitable for us, whether it really helps to improve compilation times > and what it costs for us (time). exactly. The quick migration which I have done yesterday passes the tests, and seems to work at first glance, but there were some tricky sections of the code. Which I converted but I didn't test. I feel more comfortable with old and well tested boost::python version. Also I see that Vaclav has done this already [1] > > * I would prefer minieigen-src package (no extra compilation time for > > people who use `double`) > > * minieigen-src in ubuntu 20.04 will let more people to test & use high > > precision. > Unfortunately, it is almost impossible for the time being […] this is clear. > We should probably consider alternative ways to help you to integrate > high-precision stuff. Thank you! :)) Another note: I just realized that the compilation benchmarks might have not been fair, because I think that I added few more registrations (like Vector2c, or sth. like that) while doing pybind migration. I will compare again without these extra registrations. Another comparison note: 8. boost::python is much more picky about the order of registered functions, and sometimes does not work if the order is "wrong". pybind always resolves the function overloads correctly. best regards Janek [1] https://github.com/woodem/woo/blob/master/lib/eigen/pybind11/visitors.hpp ___ Mailing list: https://launchpad.net/~yade-dev Post to : yade-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~yade-dev More help : https://help.launchpad.net/ListHelp
