Re: [yast-devel] Build Times Strikes Back
On 6/26/19 7:42 AM, Josef Reidinger wrote: V Tue, 25 Jun 2019 17:05:31 +0200 Stasiek Michalski napsáno: On wto, cze 25, 2019 at 5:02 PM, Josef Reidinger wrote: Hi, Any other idea that can help? I would be curious about how quick meson/ninja would as a replacement for configure/make all. It promises to be quite a bit faster compared to other tools ;) LCP [Stasiek] https://lcp.world In fact, my plan was to play with libstorage-ng + mason during this week, but I am still working on another thing :) Hi, sounds like interesting idea. Feel free to play with it and measure it. Ideally run on your machine ( so no side effects from different build host ) five times rake osc:build with old and same amount with new and try to compare average times with removal of extremes that can happen. I would be also curious, if we really spend time in g++ and swig or if it some ineffeciency in autotools respective its work distribution. Josef -- To unsubscribe, e-mail: yast-devel+unsubscr...@opensuse.org To contact the owner, e-mail: yast-devel+ow...@opensuse.org
Re: [yast-devel] Build Times Strikes Back
V Tue, 25 Jun 2019 17:05:31 +0200 Stasiek Michalski napsáno: > On wto, cze 25, 2019 at 5:02 PM, Josef Reidinger > wrote: > > Hi, > > > > > > Any other idea that can help? > > I would be curious about how quick meson/ninja would as a replacement > for configure/make all. It promises to be quite a bit faster compared to > other tools ;) > > LCP [Stasiek] > https://lcp.world > > Hi, sounds like interesting idea. Feel free to play with it and measure it. Ideally run on your machine ( so no side effects from different build host ) five times rake osc:build with old and same amount with new and try to compare average times with removal of extremes that can happen. I would be also curious, if we really spend time in g++ and swig or if it some ineffeciency in autotools respective its work distribution. Josef -- To unsubscribe, e-mail: yast-devel+unsubscr...@opensuse.org To contact the owner, e-mail: yast-devel+ow...@opensuse.org
[yast-devel] Pending Pull Requests
This email is automatic generated from yast CI node. It lists of pull requests that have no activity more then three working days. If your module is listed, please check all pull request, why they are not merged yet. Pending requests in repository yast-add-on: - Unify labels (bsc#1123679) (106 days) https://github.com/yast/yast-add-on/pull/69 Pending requests in repository yast-apparmor: - Port to rake (11 days) https://github.com/yast/yast-apparmor/pull/40 Pending requests in repository yast-autoinstallation: - Modernize specfile (77 days) https://github.com/yast/yast-autoinstallation/pull/499 Pending requests in repository yast-bootloader: - Fix resume with uuid partition (32 days) https://github.com/yast/yast-bootloader/pull/564 - patch to not show popup when editenv list failed (284 days) https://github.com/yast/yast-bootloader/pull/537 - [no merge] More logging sle12 (371 days) https://github.com/yast/yast-bootloader/pull/523 Pending requests in repository yast-cluster: - Port to rake (11 days) https://github.com/yast/yast-cluster/pull/52 Pending requests in repository yast-country: - Port to rake (8 days) https://github.com/yast/yast-country/pull/217 - Adding Persian (Farsi) (bsc#1092920) (74 days) https://github.com/yast/yast-country/pull/211 - Stop using obsolete XVersion (111 days) https://github.com/yast/yast-country/pull/205 - Keyboard rewrite: encapsulate systemd actions (178 days) https://github.com/yast/yast-country/pull/200 - Keyboard rewrite: add documentation and license information (215 days) https://github.com/yast/yast-country/pull/199 - Keyboard rewrite: fix error setting x11 keymap (215 days) https://github.com/yast/yast-country/pull/198 - Keyboard rewrite: add more layouts to the module (215 days) https://github.com/yast/yast-country/pull/197 - Keyboard rewrite: select current layout on startup (215 days) https://github.com/yast/yast-country/pull/196 - Keyboard rewrite: Use strategy pattern to avoid coupling (215 days) https://github.com/yast/yast-country/pull/195 - Keyboard rewrite: Add rubocop (215 days) https://github.com/yast/yast-country/pull/174 - Keyboard rewrite: Test keyboard layout before change it (215 days) https://github.com/yast/yast-country/pull/173 - Keyboard rewrite: Change keyboard layout (215 days) https://github.com/yast/yast-country/pull/168 - Keyboard rewrite: Load keyboard layouts (215 days) https://github.com/yast/yast-country/pull/144 Pending requests in repository yast-devtools: - Move desktop files (16 days) https://github.com/yast/yast-devtools/pull/143 - [RFC] Update the Rubocop config to version 0.59.2 (239 days) https://github.com/yast/yast-devtools/pull/134 Pending requests in repository yast-installation: - BSC#1132915 - Bootparameter url file:///PATH will not be evaluated co… (48 days) https://github.com/yast/yast-installation/pull/796 - Stop using obsolete XVersion (111 days) https://github.com/yast/yast-installation/pull/780 - Demo: eager mode of zombie-killer 0.5 (229 days) https://github.com/yast/yast-installation/pull/761 - Sp4 merge (391 days) https://github.com/yast/yast-installation/pull/708 Pending requests in repository yast-mail: - bsc#105491 setup of postfix with amavis DKIM signing does not work. (18 days) https://github.com/yast/yast-mail/pull/44 - bsc#105491 setup of postfix with amavis DKIM signing does not work at all (243 days) https://github.com/yast/yast-mail/pull/37 Pending requests in repository yast-network: - [WIP] Added bridge and infiniband connection configs (4 days) https://github.com/yast/yast-network/pull/850 - Routing default zero (42 days) https://github.com/yast/yast-network/pull/810 - Wrapping LanItems::Items into objects (39 days) https://github.com/yast/yast-network/pull/796 - [RFC] First impressions of the current network configuration status (106 days) https://github.com/yast/yast-network/pull/729 - [RFC] Create a backup of the modified interface and restore if canceled (121 days) https://github.com/yast/yast-network/pull/728 - [WIP] AutoYaST change the way the profile is intepreted (375 days) https://github.com/yast/yast-network/pull/646 - [RFC] Udev rule preparations (376 days) https://github.com/yast/yast-network/pull/639 - [WIP] Stop systemd socket associated services and ask for installing firewalld service definition (428 days) https://github.com/yast/yast-network/pull/624 ERROR: API query limit exceeded -- To unsubscribe, e-mail: yast-devel+unsubscr...@opensuse.org To contact the owner, e-mail: yast-devel+ow...@opensuse.org
Re: [yast-devel] Build Times Strikes Back
On Tue, Jun 25, 2019 at 05:02:43PM +0200, Josef Reidinger wrote: Hi. > today when I am waiting for OBS to build new ci container for testing new > rubocop I check how situation changes after our last effort to reduce build > time of yast stack. What actually caused a rebuild of libstorage-ng? libstorage-ng does not depend on rubocop. > To compare, old libstorage need in total 549 seconds. So there is really slow > down in build time. > Question is how to speed up building process? Any ideas? I think 20 minutes > for the initial building stone of all yast modules is too much. The build times look high to me. On my machine 'osc build' takes less than 500s (uses make -j8). > - create libstorage-ng-bootstrap that will be used for building > yast2-storage-ng. That bootstrap will skip tests and pythong bindings and > maybe even compile with less aggresive g++ options, which should help a lot. > And of course then proper package is build that will have all this. Both the Pyhton and Ruby bindings could be build in separate packages. But that would (likely) have to downside that the bindings cannot be build in parallel with the library. > - build python and ruby bindings in parallel. Is it doable? Or ideally do it > in parallel to other compilation tasks. ( not sure if it does not hit us back > with disk seeking ). Here non-recursive feature of autotools can help[7]. That should work. I have a hackish bash script that does so (even in parallel to the rest) and uses about 6m. ciao Arvin -- Arvin Schnell, Senior Software Engineer, Research & Development SUSE Linux GmbH Maxfeldstraße 5 90409 Nürnberg Germany GF: Felix Imendörffer, Mary Higgins, Sri Rasiah HRB 21284 (AG Nürnberg) -- To unsubscribe, e-mail: yast-devel+unsubscr...@opensuse.org To contact the owner, e-mail: yast-devel+ow...@opensuse.org
Re: [yast-devel] Build Times Strikes Back
On wto, cze 25, 2019 at 5:02 PM, Josef Reidinger wrote: Hi, today when I am waiting for OBS to build new ci container for testing new rubocop I check how situation changes after our last effort to reduce build time of yast stack. I use our old tool [1] and there is also old data[2]. So I can compare critical path ( which is the most interesting, as it count minimum time needed to build yast stack if we have same machines, but in unlimited amount ). So how it changes? critical path: - total time: 1780 + total time: 3452 path: - - yast2-schema [141s] - - autoyast2 [168s] - - yast2-update [143s] - - yast2-packager [75s] - - yast2-storage [226s] - - yast2-perl-bindings [236s] - - yast2-ruby-bindings [181s] - - yast2-ycp-ui-bindings [107s] - - yast2-core [426s] - - yast2-devtools [77s] + - yast2-schema [134s] + - yast2-ntp-client [93s] + - autoyast2 [723s] + - yast2-installation [721s] + - yast2-network [171s] + - yast2-packager [93s] + - yast2-storage-ng [186s] + - libstorage-ng [1331s] The first think that is visible immediatelly that it increase a lot. From cca 30 minutes to almost 60 minutes ( so no surprise I am waiting for new release :) So I check more deeply some values, as some times looks strange. Sometimes it looks like strange behavior of OBS because total time is much bigger then steps like for yast2-installation[3] or autoyast2[4], but for libstrorage-ng it looks like real data[5]. So I check if libstorage-ng is not unlucky and simply does not hit weak machine, but it builds with -j6 which is not bad from my POV. So I do quick analyse where it spends time ( from log [6]): total: 1327s unpacking: 1s compilation (configure + make all): 921s ( doxygen: 10s, translations: 0s, bindings: 542s, ) install: 8s deduplication: 2s tests: 260s To compare, old libstorage need in total 549 seconds. So there is really slow down in build time. Question is how to speed up building process? Any ideas? I think 20 minutes for the initial building stone of all yast modules is too much. Few wild ideas: - create libstorage-ng-bootstrap that will be used for building yast2-storage-ng. That bootstrap will skip tests and pythong bindings and maybe even compile with less aggresive g++ options, which should help a lot. And of course then proper package is build that will have all this. - build python and ruby bindings in parallel. Is it doable? Or ideally do it in parallel to other compilation tasks. ( not sure if it does not hit us back with disk seeking ). Here non-recursive feature of autotools can help[7]. Any other idea that can help? I would be curious about how quick meson/ninja would as a replacement for configure/make all. It promises to be quite a bit faster compared to other tools ;) LCP [Stasiek] https://lcp.world -- To unsubscribe, e-mail: yast-devel+unsubscr...@opensuse.org To contact the owner, e-mail: yast-devel+ow...@opensuse.org
[yast-devel] Build Times Strikes Back
Hi, today when I am waiting for OBS to build new ci container for testing new rubocop I check how situation changes after our last effort to reduce build time of yast stack. I use our old tool [1] and there is also old data[2]. So I can compare critical path ( which is the most interesting, as it count minimum time needed to build yast stack if we have same machines, but in unlimited amount ). So how it changes? critical path: - total time: 1780 + total time: 3452 path: - - yast2-schema [141s] - - autoyast2 [168s] - - yast2-update [143s] - - yast2-packager [75s] - - yast2-storage [226s] - - yast2-perl-bindings [236s] - - yast2-ruby-bindings [181s] - - yast2-ycp-ui-bindings [107s] - - yast2-core [426s] - - yast2-devtools [77s] + - yast2-schema [134s] + - yast2-ntp-client [93s] + - autoyast2 [723s] + - yast2-installation [721s] + - yast2-network [171s] + - yast2-packager [93s] + - yast2-storage-ng [186s] + - libstorage-ng [1331s] The first think that is visible immediatelly that it increase a lot. From cca 30 minutes to almost 60 minutes ( so no surprise I am waiting for new release :) So I check more deeply some values, as some times looks strange. Sometimes it looks like strange behavior of OBS because total time is much bigger then steps like for yast2-installation[3] or autoyast2[4], but for libstrorage-ng it looks like real data[5]. So I check if libstorage-ng is not unlucky and simply does not hit weak machine, but it builds with -j6 which is not bad from my POV. So I do quick analyse where it spends time ( from log [6]): total: 1327s unpacking: 1s compilation (configure + make all): 921s ( doxygen: 10s, translations: 0s, bindings: 542s, ) install: 8s deduplication: 2s tests: 260s To compare, old libstorage need in total 549 seconds. So there is really slow down in build time. Question is how to speed up building process? Any ideas? I think 20 minutes for the initial building stone of all yast modules is too much. Few wild ideas: - create libstorage-ng-bootstrap that will be used for building yast2-storage-ng. That bootstrap will skip tests and pythong bindings and maybe even compile with less aggresive g++ options, which should help a lot. And of course then proper package is build that will have all this. - build python and ruby bindings in parallel. Is it doable? Or ideally do it in parallel to other compilation tasks. ( not sure if it does not hit us back with disk seeking ). Here non-recursive feature of autotools can help[7]. Any other idea that can help? Josef [1] https://github.com/mvidner/rpm-build-dependencies [2] https://github.com/mvidner/rpm-build-dependencies/blob/master/yast_deps.yaml [3] https://build.opensuse.org/package/statistics/YaST:Head/yast2-installation/openSUSE_Factory/x86_64 [4] https://build.opensuse.org/package/statistics/YaST:Head/autoyast2/openSUSE_Factory/x86_64 [5] https://build.opensuse.org/package/statistics/YaST:Head/libstorage-ng/openSUSE_Factory/x86_64 [6] https://build.opensuse.org/build/YaST:Head/openSUSE_Factory/x86_64/libstorage-ng/_log [7] https://autotools.io/automake/parallel.html -- To unsubscribe, e-mail: yast-devel+unsubscr...@opensuse.org To contact the owner, e-mail: yast-devel+ow...@opensuse.org
Re: [yast-devel] YaST: highlights of development sprint 77, 78 and 79
On 6/25/19 11:52 AM, Ancor Gonzalez Sosa wrote: > three development sprints, namely the 77th, 78th and 79th. Which are the correct sprint numbers. The subject of the original email is wrong (the perils of copy&paste). Cheers. -- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscr...@opensuse.org To contact the owner, e-mail: yast-devel+ow...@opensuse.org
[yast-devel] YaST: highlights of development sprint 71 and 72
May and June have been, so far, interesting months for the YaST Team. Many things happening... which had the drawback of keeping us too busy to publish our usual sprint reports. But we are back with a looong report trying to summarize the latest three development sprints, namely the 77th, 78th and 79th. So take a cup of your favorite drink and dive into... https://lizards.opensuse.org/2019/06/25/yast-sprints-77-79/ As special bonus, it includes a link to the our recent blog post titled "Getting Further with Btrfs in YaST", in case you missed that announcement. Cheers -- Ancor González Sosa YaST Team at SUSE Linux GmbH -- To unsubscribe, e-mail: yast-devel+unsubscr...@opensuse.org To contact the owner, e-mail: yast-devel+ow...@opensuse.org
