Re: [OpenIndiana-discuss] ICU update and rebuilds
Rebuilds are done... let us hope everything republished fine... There was a problem in qt5, sorry about the delay. On Mon, Jun 1, 2020 at 8:47 PM Tim Mooney wrote: > In regard to: [OpenIndiana-discuss] ICU update and rebuilds, Aur?lien...: > > > ICU is being updated to a non-ABI compatible version and quite a few > > components need a rebuild...so the server will be quite busy and > > unavailable at times in the coming 10 hours. > > I would suggest postponing any update... > > I looked at updating ICU about 6 months ago, saw the list of dependencies, > and decided to tackle other packaging updates that didn't seem so > insurmountable. > > OpenSSL is another place where I "gave up", though it's even more > complicated than ICU because of the API changes and potential dependent > component updates required. > > Do you have any words of wisdom or suggestions for how to approach > a component with so many dependencies? Were there any "tricks" that > you used to make the ICU updates easier? It looks like for many of the > "easy" rebuilds, you grouped them into one PR (#5829). How did you decide > which ones could all be handled in one branch, and which ones needed > their own separate git branch and PR? > > Has there ever been any discussion about splitting just the library > into a separate package so that it's possible to have multiple versions > of the shared library installed at once, to make it possible to migrate > dependencies in "phases" to the latest version, rather than having to > coordinate rebuilding dozens of packages? > > Thanks for your work on this! I can only imagine how tedious this > was to accomplish... > > Tim > -- > Tim Mooney tim.moo...@ndsu.edu > Enterprise Computing & Infrastructure / > Division of Information Technology/701-231-1076 (Voice) > North Dakota State University, Fargo, ND 58105-5164 > > ___ > openindiana-discuss mailing list > openindiana-discuss@openindiana.org > https://openindiana.org/mailman/listinfo/openindiana-discuss > -- --- Praise the Caffeine embeddings ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org https://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ICU update and rebuilds
On Mon, 1 Jun 2020, Tim Mooney wrote: In regard to: [OpenIndiana-discuss] ICU update and rebuilds, Aur?lien...: ICU is being updated to a non-ABI compatible version and quite a few components need a rebuild...so the server will be quite busy and unavailable at times in the coming 10 hours. I would suggest postponing any update... I looked at updating ICU about 6 months ago, saw the list of dependencies, and decided to tackle other packaging updates that didn't seem so insurmountable. GraphicsMagick has had this to say about ICU in its release notes for some time: * It has been discovered that the 'ICU' library (a perhaps 30MB C++ library) which is now often a libxml2 dependendency causes huge process initialization overhead. This is noticed as unexpected slowness when GraphicsMagick utilities are used to process small to medium sized files. The time to initialize the 'ICU' library is often longer than the time that GraphicsMagick would otherwise require to read the input file, process the image, and write the output file. If the 'ICU' dependency can not be avoided, then make sure to use the modules build so there is only impact for file formats which require libxml2. Please lobby the 'ICU' library developers to change their implementation to avoid long start-up times due to merely linking with the library. Besides the build issues, I suggest taking care about how ICU is deployed and linked in the system. It appears that ICU expects that applications using it are long-lived, have a lot of resources, and can afford to take a long time to start up. An incidental linkage of ICU to satisfy a few apps requirements may have the unintended consequence of making many other apps slow and memory-hungry. I don't think that we want OpenIndiana to feel sluggish and slow. It is my guess that the ICU shared libraries use C++ static constructors (or a shared library equivalent) to perform expensive/protracted initialization when the shared library is loaded, regardless of if its functionality will ever actually be used! Just linking with the library (without invoking any API which might use it) afflicts the dependent application. An optional libxml2 dependency often brings in this steaming pile and thus afflicts the many applications which link with libxml2. My mind was boggled when I saw apparently trivial commands taking a substantial portion of a second under Linux. I expect that the situation would be worse with Illumos. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org https://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ICU update and rebuilds
On Mon, Jun 1, 2020 at 8:47 PM Tim Mooney wrote: > In regard to: [OpenIndiana-discuss] ICU update and rebuilds, Aur?lien...: > > > ICU is being updated to a non-ABI compatible version and quite a few > > components need a rebuild...so the server will be quite busy and > > unavailable at times in the coming 10 hours. > > I would suggest postponing any update... > > I looked at updating ICU about 6 months ago, saw the list of dependencies, > and decided to tackle other packaging updates that didn't seem so > insurmountable. > > OpenSSL is another place where I "gave up", though it's even more > complicated than ICU because of the API changes and potential dependent > component updates required. > > Do you have any words of wisdom or suggestions for how to approach > a component with so many dependencies? Were there any "tricks" that > you used to make the ICU updates easier? It looks like for many of the > "easy" rebuilds, you grouped them into one PR (#5829). How did you decide > which ones could all be handled in one branch, and which ones needed > their own separate git branch and PR? > I looked at dependencies using pkg search -r -o pkg.name "depend:require:library/icu" and then iterated through the dependencies. > > Has there ever been any discussion about splitting just the library > into a separate package so that it's possible to have multiple versions > of the shared library installed at once, to make it possible to migrate > dependencies in "phases" to the latest version, rather than having to > coordinate rebuilding dozens of packages? > > Thanks for your work on this! I can only imagine how tedious this > was to accomplish... > Unfortunately it did not go so well as the build server decided to become unavailable at the last round of builds... For now the build server is down and the latest packages have not been published... I am very sorry about that... :( Currently the procedure is partly manual and fragile but there is some work by Adam to automating rebuilds of dependent packages by iteratiing on the graph (and breaking cycles). It requires more integration but will be crucial to deal with OpenSSL. Another difficulty is packages that may fail to build because our build server is either on an older verson of osnet-incorporation or because OpenGL code builds differently in zones... > > Tim > -- > Tim Mooney tim.moo...@ndsu.edu > Enterprise Computing & Infrastructure / > Division of Information Technology/701-231-1076 (Voice) > North Dakota State University, Fargo, ND 58105-5164 > > ___ > openindiana-discuss mailing list > openindiana-discuss@openindiana.org > https://openindiana.org/mailman/listinfo/openindiana-discuss > -- --- Praise the Caffeine embeddings ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org https://openindiana.org/mailman/listinfo/openindiana-discuss
Re: [OpenIndiana-discuss] ICU update and rebuilds
In regard to: [OpenIndiana-discuss] ICU update and rebuilds, Aur?lien...: ICU is being updated to a non-ABI compatible version and quite a few components need a rebuild...so the server will be quite busy and unavailable at times in the coming 10 hours. I would suggest postponing any update... I looked at updating ICU about 6 months ago, saw the list of dependencies, and decided to tackle other packaging updates that didn't seem so insurmountable. OpenSSL is another place where I "gave up", though it's even more complicated than ICU because of the API changes and potential dependent component updates required. Do you have any words of wisdom or suggestions for how to approach a component with so many dependencies? Were there any "tricks" that you used to make the ICU updates easier? It looks like for many of the "easy" rebuilds, you grouped them into one PR (#5829). How did you decide which ones could all be handled in one branch, and which ones needed their own separate git branch and PR? Has there ever been any discussion about splitting just the library into a separate package so that it's possible to have multiple versions of the shared library installed at once, to make it possible to migrate dependencies in "phases" to the latest version, rather than having to coordinate rebuilding dozens of packages? Thanks for your work on this! I can only imagine how tedious this was to accomplish... Tim -- Tim Mooney tim.moo...@ndsu.edu Enterprise Computing & Infrastructure / Division of Information Technology/701-231-1076 (Voice) North Dakota State University, Fargo, ND 58105-5164 ___ openindiana-discuss mailing list openindiana-discuss@openindiana.org https://openindiana.org/mailman/listinfo/openindiana-discuss
