Re: [Rpm-maint] [rpm-software-management/rpm] Add --dwz-single-file-mode argument for find-debuginfo.sh. (#1579)

2021-03-19 Thread marxin
@mlschroe : May I please ping this? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/1579#issuecomment-802919085___ Rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Add --dwz-single-file-mode argument for find-debuginfo.sh. (#1579)

2021-03-16 Thread marxin
> Why are we doing this? We do it in order to reduce dependencies in-between `debuginfo` sub-packages of a package that uses baselibs mechanism: https://en.opensuse.org/openSUSE:Build_Service_baselibs.conf#Quickstart -- You are receiving this because you are subscribed to this thread. Reply to

Re: [Rpm-maint] [rpm-software-management/rpm] Add --dwz-single-file-mode argument for find-debuginfo.sh. (#1579)

2021-03-16 Thread marxin
That sounds good. I guess we can fix possible issues incrementally, right? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Add --dwz-single-file-mode argument for find-debuginfo.sh. (#1579)

2021-03-16 Thread marxin
> Hmm, I don't know. Are the extra requires gone if there is nore than one > debuginfo package? What do you mean by "extra requires"? Please provide an example you have concerns about. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it

Re: [Rpm-maint] [rpm-software-management/rpm] Add --dwz-single-file-mode argument for find-debuginfo.sh. (#1579)

2021-03-16 Thread marxin
Ok, second part of my attempt is here: https://build.opensuse.org/package/rdiff/home:marxin:branches:Base:System/rpm?opackage=rpm=Base%3ASystem=2 Do I miss something else? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Add --dwz-single-file-mode argument for find-debuginfo.sh. (#1579)

2021-03-16 Thread marxin
In the single file mode, no new files are created so any new dependency cannot be created by `dwz`. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Add --dwz-single-file-mode argument for find-debuginfo.sh. (#1579)

2021-03-15 Thread marxin
Adding @mlschroe to CC. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/1579#issuecomment-799538451___ Rpm-maint mailing list

[Rpm-maint] [rpm-software-management/rpm] Add --dwz-single-file-mode argument for find-debuginfo.sh. (#1579)

2021-03-15 Thread marxin
Sometimes its handy to disable multi-file mode and the patch adds option for that: `--dwz-single-file-mode`. It will be used in openSUSE for packages that use baselibs.conf mechanism. You can view, comment on, or merge this pull request online at:

Re: [Rpm-maint] [rpm-software-management/rpm] zstd: use thread pool (#1466)

2021-01-29 Thread marxin
I see your concerns and the pull request is hack a bit. On the other hand, `zstd` provides a nice thread pool functionality and package build really benefits from it. Right now, packages like Firefox also mostly compressed in one thread as compression of the biggest `.rpm` takes minutes. --

Re: [Rpm-maint] [rpm-software-management/rpm] zstd: use thread pool (#1466)

2021-01-28 Thread marxin
> Sorry but I can't see us accepting this. It's a layering violation, the build > code cannot have any special knowledge of what goes on inside librpmio and > vice versa. Hm, that's a pity. > > Rpm really needs a resource manager (see #804) ... which hopefully can > somehow integrate with

Re: [Rpm-maint] [rpm-software-management/rpm] zstd: use thread pool (#1466)

2021-01-09 Thread marxin
May I please ping this pull request, adding people who helped me in previous zstd-related PR: @ffesti @ignatenkobrain -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[Rpm-maint] [rpm-software-management/rpm] zstd: use thread pool (#1466)

2020-12-22 Thread marxin
Starting from `zstd` version `1.4.7`, the library provides an experimental usage of thread pools (thats why `-DZSTD_STATIC_LINKING_ONLY=1` is needed). Using them can fully utilize CPU on one side and maintain memory utilization at the same time. Such an approach fixes #1324. You can view,

Re: [Rpm-maint] [rpm-software-management/rpm] [WIP] Use semaphore for compression in ZSTD. (#1349)

2020-12-22 Thread marxin
Closed #1349. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/1349#event-4139307712___ Rpm-maint mailing list

Re: [Rpm-maint] [rpm-software-management/rpm] [WIP] Use semaphore for compression in ZSTD. (#1349)

2020-08-27 Thread marxin
@marxin pushed 1 commit. 10aeb351460a764806c558703db3c21d335ccd72 Use semaphore for compression in ZSTD. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1349/files

[Rpm-maint] [rpm-software-management/rpm] [WIP] Use semaphore for compression in ZSTD. (#1349)

2020-08-27 Thread marxin
This proof-of-concept patch can handle when parallel compression happens in the context of parallel `.rpm` file creation. Its mentioned here: #1324 and compression semaphore can handle that. `zstd` counterpart is here: facebook/zstd#2290. You can view, comment on, or merge this pull request

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-08-26 Thread marxin
> We'll need some sort of resource manager (no matter how crude) in rpm sooner > than later to handle this sort of stuff, but in the One simple possible solution can be using a semaphore that will be respected by a compression library. `zstd` issue about it lives here:

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-08-19 Thread marxin
@marxin commented on this pull request. > @@ -523,6 +523,27 @@ fprintf(stderr, "*** ufdOpen(%s,0x%x,0%o)\n", url, > (unsigned)flags, (unsigned)mo return fd; } +/* Return number of threads ought to be used for compression based + on a parsed value threads (e.g.

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-08-19 Thread marxin
@marxin pushed 2 commits. 2c659f7ec7dea6fb1a4a25d13a91592c0acb7d97 Add get_compression_threads and refactor code. c771b3b43d05d76d0b19df9a2c25eba3c9f1e6a1 Enable thread autodetection for a parallel compression. -- You are receiving this because you are subscribed to this thread. View

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-08-19 Thread marxin
@marxin pushed 2 commits. 854bf2775fbaaa7b0621de46f3e42a165a5fd60c Add get_compression_threads and refactor code. b490f59832a056a897cd1835d7faed4a41585327 Enable thread autodetection for a parallel compression. -- You are receiving this because you are subscribed to this thread. View

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-08-12 Thread marxin
@pmatilai I hope I addressed all your comments. Feel free to look at it now.. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-08-08 Thread marxin
@marxin pushed 2 commits. 9ab2a82e5e0663a86d1d489a7a6f8825da395b54 Disable threading for xz on 32-bit systems. 47ca1e51e5e113763cb13223f915b45b9bf894b6 Enable thread autodetection for a parallel compression. -- You are receiving this because you are subscribed to this thread. View

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-08-08 Thread marxin
> I think it should at least follow the same convention as XZ: T with no > numbers mean autodetection (its okay if 0 means that too). This seems to > silently change XZ behavior too which is not okay. I'm going to update the documentation of my change. Yes, it was intentional to sync behavior

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-08-04 Thread marxin
> Right, this isn't any worse than what we already have for XZ, but then again > what is there for XZ is simply wrong in the face of concurrent package > generation as we have now. The lowly IO stream has no way of knowing how many > threads it can legitimately consume. That said, would it be

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-07-31 Thread marxin
Thank you for your thoughts. Let's first summarize what tricks are done for `xz`: https://github.com/rpm-software-management/rpm/blob/28b3704034a9c772959485a130b918f414da9796/rpmio/rpmio.c#L811-L859 - the compression supports `%{_xz_memlimit}` limit - for 32-bit it calculates how much memory

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-07-30 Thread marxin
@marxin pushed 1 commit. 34a78ee61a67a9ffe11142c045fd1fa01e44160e Enable thread autodetection for a parallel compression. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1324/files

Re: [Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-07-30 Thread marxin
It's a follow up of #1303, so adding @ignatenkobrain as a reviewer ;) -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

[Rpm-maint] [rpm-software-management/rpm] Enable thread autodetection for a parallel compression. (#1324)

2020-07-30 Thread marxin
When using xz or zstd, it can be handy to automatically detect number of threads. We can use `%{getncpus}` for that. You can view, comment on, or merge this pull request online at: https://github.com/rpm-software-management/rpm/pull/1324 -- Commit Summary -- * Enable thread autodetection

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-30 Thread marxin
CI seems happy now, can please anybody merge it? I've got another patch based on this one that I would like to propose.. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-29 Thread marxin
> Looks like it fails to detect the right commit to checkout and build. Can you > force push a new version, please, to test if that get's it unstuck? Just do > some white space changes to the commit message. Done, good point! -- You are receiving this because you are subscribed to this

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-29 Thread marxin
@marxin pushed 1 commit. 790bd81bc215b7aea17cbc5eb2e7a53e15b99ded Support threading for zstd compression. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1303/files

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-28 Thread marxin
Can please anybody unblock it? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/1303#issuecomment-665012491___ Rpm-maint mailing

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-27 Thread marxin
> Looks like there's a hick-up in the CI. Re-running the semaphore tests. > > Well, trying... Thanks, apparently it's still stuck.. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-27 Thread marxin
Thank you for the review. Having 2 approvals can please anybody merge it? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-26 Thread marxin
@marxin pushed 1 commit. 11358ce35d0c98d716d68bb4824e86c44f78491b Support threading for zstd compression. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1303/files

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-26 Thread marxin
@marxin commented on this pull request. > @@ -350,7 +350,7 @@ package or when debugging this package.\ # "w9.gzdio" gzip level 9 (default). # "w9.bzdio" bzip2 level 9. # "w6.xzdio" xz level 6, xz's defa

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-24 Thread marxin
> I'll implement support in deltarpm and you can port it to drpm, ok? Sure, appreciate that! -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-24 Thread marxin
> Looks good to me, thanks. (Of course the deltarpm and drpm need to be updated > now so that they support a ZSTD_THREADED compression type as well.) Thanks. I can start working on the support for the mentioned tools. Can you please guide me a bit what would be needed? -- You are receiving

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-24 Thread marxin
> I've just asked about this explicitly here: > [facebook/zstd#2238 > (comment)](https://github.com/facebook/zstd/issues/2238#issuecomment-662906821) And `zstd` folks confirmed that a number of threads does not influence the stability of the "threaded" compression mode. -- You are receiving

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-23 Thread marxin
> From my POV it is good to go and we can follow up with improvements like > auto-detection of cores. Thank you. Yes, I agree with that. I can imagine doing what `zstd` or `xz` do for auto-detection of cores: `xz -T0 `. -- You are receiving this because you are subscribed to this thread.

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-23 Thread marxin
> I'm also not so sure about the "the multi-threaded output produces the same > compressed data no matter how many threads you use" statement, because the > block size seems to depend on the number of workers (see > ZSTDMT_compress_advanced_internal). I've just asked about this explicitly

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-23 Thread marxin
All right, I changed the logic so that threaded mode is used only when a user set a `T` value in the compression algorithm. And I've just removed the automatic deduction of number of cores for now. @mlschroe Hope it's useful now? -- You are receiving this because you are subscribed to this

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-23 Thread marxin
@marxin pushed 1 commit. 0281c07a8ba82cb73abdb5488a1d8a784db91f8e Support threading for zstd compression. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1303/files

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-11 Thread marxin
@marxin pushed 1 commit. 14af110e70cc1f63e462eb11dab1230028fb8ce7 Support threading for zstd compression. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1303/files

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-11 Thread marxin
> With my suggestions, and a proper build of libzstd in Fedora it works as > expected, it fully utilizes my system. Great to hear! -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-11 Thread marxin
@marxin commented on this pull request. > goto err; } + + if (threads > 0) Good point! I changed that to `> 1`, same what lzma does. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitH

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-11 Thread marxin
@marxin commented on this pull request. > + if (c >= (int)'0' && c <= (int)'9') + threads = strtol(s-1, (char **), 10); Well, it's just a small piece of code. I would not put it to a separate function. -- You are receiving this because you are subscrib

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-11 Thread marxin
@marxin pushed 1 commit. d2465252ae96f05b874d453e7a245c2a47a4bd50 Support threading for zstd compression. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1303/files

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-11 Thread marxin
@marxin commented on this pull request. >AS_IF([test "$enable_zstd" = "yes"], [ if test "$have_zstd" = "no"; then AC_MSG_ERROR([--enable-zstd specified, but not available]) fi ]) + PKG_CHECK_MODULES([ZSTD], [libzstd], [have_zst

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-11 Thread marxin
@marxin pushed 2 commits. 077e20cb996777e8b3e918c4e520005c73df37bc zstd compression: port to the new API. 749bf9ecbadbbde5076bf722e1b9883e4df345ba Support threading for zstd compression. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-10 Thread marxin
> This change looks good and straight forward. Thanks! > > I am missing some information on what version of zstd supports the new API - > and if necessary a check in configure.ac. Updated `configure.ac` (one needs at least release `1.3.8`). -- You are receiving this because you are

Re: [Rpm-maint] [rpm-software-management/rpm] Support threading for zstd compression. (#1303)

2020-07-10 Thread marxin
@marxin pushed 1 commit. 1d965759205bf5f072724c397004ba1a433f7c7f Support threading for zstd compression. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1303/files

Re: [Rpm-maint] [rpm-software-management/rpm] zstd compression: port to the new API. (#1303)

2020-07-10 Thread marxin
@marxin pushed 1 commit. fff307dd41294192560a33cca09e3675a0af925a Support threading for zstd compression. -- You are receiving this because you are subscribed to this thread. View it on GitHub: https://github.com/rpm-software-management/rpm/pull/1303/files

Re: [Rpm-maint] [rpm-software-management/rpm] zstd compression: port to the new API. (#1303)

2020-07-10 Thread marxin
So the answer is that it's stable and reproducible. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] zstd compression: port to the new API. (#1303)

2020-07-09 Thread marxin
> I haven't checked the zstd implementation, I admit, I just know that this was > the problem with parallel bzip2/xz compression. I've made some experiments for a 100MB big file and the compressed output is still the same. For being sure, I asked the question in

Re: [Rpm-maint] [rpm-software-management/rpm] zstd compression: port to the new API. (#1303)

2020-07-08 Thread marxin
No. How does it break it? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/1303#issuecomment-655620102___ Rpm-maint mailing list

Re: [Rpm-maint] [rpm-software-management/rpm] zstd compression: port to the new API. (#1303)

2020-07-08 Thread marxin
Demo of the used parallel compression for `cvise` package: https://gist.github.com/marxin/5560256dbf0af4c749bd4d56c0e83626 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull

[Rpm-maint] [rpm-software-management/rpm] zstd compression: port to the new API. (#1303)

2020-07-08 Thread marxin
Port zstd compression to the new API (ZSTD_compressStream2). My motivation is to eventually enable multi-threaded compression which can help packages like Firefox where theres a single huge rpm that is being compressed. Once the change is accepted, the following one-liner will enable parallel

Re: [Rpm-maint] [rpm-software-management/rpm] Question: multi-threaded zstd compression (#1300)

2020-07-07 Thread marxin
The answer is that threads are not supported right now. Adding the author of the port (@n3npq), he may be interested in? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Question: multi-threaded zstd compression (#1300)

2020-07-06 Thread marxin
Related to #256. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/1300#issuecomment-654130790___ Rpm-maint mailing list

[Rpm-maint] [rpm-software-management/rpm] Question: multi-threaded zstd compression (#1300)

2020-07-06 Thread marxin
I'm curious if `zstd` compression can utilize more threads for a single `.rpm` file? Is there any control of the threading level? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Come up with replace_optflags (#814). (#815)

2019-08-14 Thread marxin
Closed #815. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/815#event-2556286671___ Rpm-maint mailing list

Re: [Rpm-maint] [rpm-software-management/rpm] Come up with replace_optflags (#814). (#815)

2019-08-14 Thread marxin
> I'm really reluctant to add one-trick-pony utility macros like this because > over time they tend to become liabilities one way or the other, and have > other issues. Good. > > Manipulating compiler flags is a very common need indeed and could use some > help from rpm, but that need is not

[Rpm-maint] [rpm-software-management/rpm] Come up with replace_optflags (#814). (#815)

2019-08-14 Thread marxin
The macro function is intended for %{optflags} manipulation (replacement, conditional addition). You can view, comment on, or merge this pull request online at: https://github.com/rpm-software-management/rpm/pull/815 -- Commit Summary -- * Come up with replace_optflags (#814). -- File

Re: [Rpm-maint] [rpm-software-management/rpm] Provide function for $optflags manipulation (#814)

2019-08-13 Thread marxin
> `%remove_optflags 'undesirable'` > `%replace_optflags 'oldval' 'newval'` Then we can have just the simple one. Following works for me: ``` $ rpm --define "%optflags -flto=auto -O2" --define "%replace_optflags() %global optflags %(echo %optflags | sed s/%{?1}/%{?2}/)" --eval '%replace_optflags

Re: [Rpm-maint] [rpm-software-management/rpm] Provide function for $optflags manipulation (#814)

2019-08-13 Thread marxin
and @DimStar77 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/814#issuecomment-520847014___ Rpm-maint mailing list

Re: [Rpm-maint] [rpm-software-management/rpm] Provide function for $optflags manipulation (#814)

2019-08-13 Thread marxin
Adding @scarabeusiv -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/814#issuecomment-520846697___ Rpm-maint mailing list

[Rpm-maint] [rpm-software-management/rpm] Provide function for $optflags manipulation (#814)

2019-08-13 Thread marxin
As the follow up of https://github.com/rpm-software-management/rpm/pull/813 I would like to have rpm macros that will do operations on `%optflags`: Example: - `%filter-out-optflags '-flto=auto'` - `%append-if-to-optflags '-flto=auto' -ffat-lto-objects` I found the example how currently we do it

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-13 Thread marxin
> I'd think so. Good. I've just done that in: https://github.com/rpm-software-management/rpm/pull/812 -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-13 Thread marxin
Closed #809. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#event-2553371170___ Rpm-maint mailing list

[Rpm-maint] [rpm-software-management/rpm] Remove %_lto_cflags. (#812)

2019-08-13 Thread marxin
You can view, comment on, or merge this pull request online at: https://github.com/rpm-software-management/rpm/pull/812 -- Commit Summary -- * Remove %_lto_cflags. -- File Changes -- M platform.in (3) -- Patch Links -- https://github.com/rpm-software-management/rpm/pull/812.patch

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-13 Thread marxin
> It's also a fine example how basing anything on unreleased development work > comes back to bite you. Sure. So is the conclusion that we want to rip it off? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-13 Thread marxin
> In the meanwhile I learned that flto=auto is not supported by any released > gcc. We can't really go with such a value in rpm. Yep, it will be supported first in GCC 10.1 release. For openSUSE, we're using a patched GCC compiler with the support. So I'm going to push the package to our `rpm`

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-13 Thread marxin
> Sure. It just needs to grow an existence outside GH to be relevant :) That said I would like to keep the macro in `rpm` as suggested in the patch for this pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-13 Thread marxin
> The remaining question I have is whether it actually is worth it to have a > macro for this at all when we don't default to it at all and even the default > we ship is static so it's not something rpm would need to (or want to) be > aware of. Ie, how does this differ from the average other

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-13 Thread marxin
> In that case, would not it make sense to rename `%_lto_cflags` to > `%_lto_flags`? Well, we already have quite some usage of the name in our distribution. E.g. we use `%define _lto_cflags %{nil}` to disable LTO for a package. -- You are receiving this because you are subscribed to this

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-13 Thread marxin
> One slightly unrelated question to this PR (but to the LTO flags), should not > it be also part of %optflags? I guess it's up to distributions. We as openSUSE are using that in `$optflags` and we add it in so-called project config in OBS. -- You are receiving this because you are subscribed

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-12 Thread marxin
> Ah... now I get it. I somehow got the impression that it was the linking that > produced different results with different number of cpus (as often happens > with parallel compression etc) That's good we understand each other. No, our parallel linking in LTO is stable. Right now we divide the

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-12 Thread marxin
> Maybe I'm missing something fundamental here, but I don't understand how is > -flto=auto supposed to help with making builds more reproducable. Because if you are given a builder with 8 cores: ``` [ 40s] + export 'CFLAGS=-O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector-strong -funwind-tables

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-12 Thread marxin
> I can see -flto=auto being useful for the average upstream makefile defaults > but I'm not convinced this is the right thing to do in rpm context: we'd want > our parallel activities controlled via the same central tunables > ($RPM_BUILD_NCPUS / %{_smp_ncpus_max}). Which is way you want to

Re: [Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-12 Thread marxin
Hope @pmatilai can help me with this pull request? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/809#issuecomment-520366815___

[Rpm-maint] [rpm-software-management/rpm] Use -flto=auto as a default for _lto_cflags. (#809)

2019-08-09 Thread marxin
After a long discussion, we as GCC community newly introduced -flto=auto option. Doing that, LTO can automatically detect make's jobserver or fall back to # of cores. You can view, comment on, or merge this pull request online at: https://github.com/rpm-software-management/rpm/pull/809 --

Re: [Rpm-maint] [rpm-software-management/rpm] Adopt compiler flags related enhancements from Fedora (#692)

2019-05-13 Thread marxin
Any estimation when this will be merged? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/692#issuecomment-491786233___ Rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Use $ldflags in ./configure macro in order to set LDFLAGS. (#660)

2019-05-06 Thread marxin
Closed #660. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/660#event-2320800907___ Rpm-maint mailing list

Re: [Rpm-maint] [rpm-software-management/rpm] Use $ldflags in ./configure macro in order to set LDFLAGS. (#660)

2019-05-06 Thread marxin
Ok, I'm closing the pull request. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/660#issuecomment-489600068___ Rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Use $ldflags in ./configure macro in order to set LDFLAGS. (#660)

2019-05-02 Thread marxin
May I please ping this.. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/660#issuecomment-488596568___ Rpm-maint mailing list

Re: [Rpm-maint] [rpm-software-management/rpm] Use $ldflags in ./configure macro in order to set LDFLAGS. (#660)

2019-04-25 Thread marxin
OK, I've updated the pull request based on the discussion that we've got so far. -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Use $ldflags in ./configure macro in order to set LDFLAGS. (#660)

2019-04-25 Thread marxin
> I think switching to `%{build_…}` convention makes much more sense. In the > old times where `optflags` meant everything, nowadays you want to have > different CFLAGS, CXXFLAGS. And having `%{build_cflags}` but `%{ldflags}` > sounds inconsistent. I'm fine with the `%build_ldflags` name which

Re: [Rpm-maint] [rpm-software-management/rpm] Use $ldflags in ./configure macro in order to set LDFLAGS. (#660)

2019-04-25 Thread marxin
> No objection to adding LDFLAGS to %configure, it's actually long overdue. > Fedora has had its own version of this for a quite some time now, but nobody > got around to upstreaming it. This seems like as good time as any to take a > look at that and consider which parts would be upstreamable.

Re: [Rpm-maint] [rpm-software-management/rpm] Use $ldflags in ./configure macro in order to set LDFLAGS. (#660)

2019-04-23 Thread marxin
Motivation for the change: I'm planning to enable LTO in `openSUSE:Factory` and I need to add `-flto=N` to LDFLAGS in order to make LTO linking phase parallel. That speeds up package build significantly. -- You are receiving this because you are subscribed to this thread. Reply to this email

Re: [Rpm-maint] [rpm-software-management/rpm] Use $ldflags in ./configure macro in order to set LDFLAGS. (#660)

2019-04-15 Thread marxin
marxin commented on this pull request. > @@ -1021,6 +1021,7 @@ package or when debugging this package.\ CFLAGS="${CFLAGS:-%optflags}" ; export CFLAGS ; \ CXXFLAGS="${CXXFLAGS:-%optflags}" ; export CXXFLAGS ; \ FFLAGS="${FFLAGS:-%optflags}" ; export

[Rpm-maint] [rpm-software-management/rpm] Use $ldflags in ./configure macro in order to set LDFLAGS. (#660)

2019-04-11 Thread marxin
You can view, comment on, or merge this pull request online at: https://github.com/rpm-software-management/rpm/pull/660 -- Commit Summary -- * Use $ldflags in ./configure macro in order to set LDFLAGS. -- File Changes -- M macros.in (1) -- Patch Links --

Re: [Rpm-maint] [rpm-software-management/rpm] Come up with %_smp_flto macro that expands in a similar way to _smp_mflags. (#606)

2019-01-15 Thread marxin
> I edited the %_lto_cflags commit message a bit and merged this manually. > > Thanks for the patch! Thank you very much for help. Btw. when is planned another rpm release? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub:

Re: [Rpm-maint] [rpm-software-management/rpm] Come up with %_smp_flto macro that expands in a similar way to _smp_mflags. (#606)

2019-01-14 Thread marxin
> I guess we're happy with the macro itself now, but the commit message doesn't > match the action even remotely. Ideally this would be two commits, one for > the _smp_mflags refactoring and a second one to add the new macro, and a > proper rationale + explanations. For example, _smp_mflags

Re: [Rpm-maint] [rpm-software-management/rpm] Come up with %_smp_flto macro that expands in a similar way to _smp_mflags. (#606)

2019-01-14 Thread marxin
> The question is whether that's sufficient or if there should be some sort of > more user-friendly "disable lto" macro. OTOH those "user-friendly" disablers > tend to suffer from various other side-effects... That works fine for me! Then I won't need the openSUSE/rpm-config-SUSE#5 Can the SR

Re: [Rpm-maint] [rpm-software-management/rpm] Come up with %_smp_flto macro that expands in a similar way to _smp_mflags. (#606)

2019-01-14 Thread marxin
> Right. But the place in %optflags points out that this a not _make_ flag like > -j is but compiler flag. So maybe it really should be _lto_cflags, or > something like that, instead of _mflags. Done in updated version of the patch. > > The other thing I'm wondering if this is something that

Re: [Rpm-maint] [rpm-software-management/rpm] Come up with %_smp_flto macro that expands in a similar way to _smp_mflags. (#606)

2019-01-14 Thread marxin
> That's indeed much better, thanks. There's still the naming mismatch with the > concept though: %_smp_flto enables LTO regardless of number of cpus, and flto > is the name of the compiler switch, the feature is simply LTO. Maybe it > should be %_lto_mflags instead? Yep, sounds better to me

Re: [Rpm-maint] [rpm-software-management/rpm] Come up with %_smp_flto macro that expands in a similar way to _smp_mflags. (#606)

2019-01-14 Thread marxin
> NAK as it is now. Copy-paste approach is not going to cut it. We need a > separate macro that just figures out the number of cpu's we're allowed to > use, which can be then used by any macros needing that info. > > Also this is a bit different from -j as AFAICS this is used for enabling LTO

Re: [Rpm-maint] [rpm-software-management/rpm] Make %make_build to provide verbose output of make command. (#617)

2019-01-09 Thread marxin
> NAK. If someone actually wants all the output, they can append it themselves, > especially since different Makefiles handle verbosity differently. I see. What about a new ```%make_build_verbose``` that will do that? -- You are receiving this because you are subscribed to this thread. Reply

[Rpm-maint] [rpm-software-management/rpm] Make %make_build to provide verbose output of make command. (#617)

2019-01-09 Thread marxin
I consider it handy to print commands executed by make command. You can view, comment on, or merge this pull request online at: https://github.com/rpm-software-management/rpm/pull/617 -- Commit Summary -- * Make %make_build to provide verbose output of make command. -- File Changes --

Re: [Rpm-maint] [rpm-software-management/rpm] Come up with %_smp_flto macro that expands in a similar way to _smp_mflags. (#606)

2019-01-09 Thread marxin
Hello. May I please remind this patch review? -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/pull/606#issuecomment-452615568___

  1   2   >