[gentoo-dev] Last-rites: sys-kernel/xnu-headers, sys-libs/darwin-libc-headers, dev-libs/libmissing

2020-11-23 Thread Fabian Groffen
# Fabian Groffen  (2020-11-23)
# No longer used, not really functional either, noone should be using
# this, removal in 30 days.
sys-kernel/xnu-headers
sys-libs/darwin-libc-headers
dev-libs/libmissing

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?

2020-11-10 Thread Fabian Groffen
On 10-11-2020 09:21:53 +, Alexey Sokolov wrote:
> > What instructed you to perform the migration?  Was it the news-item?  I
> > don't think it should apply for Prefix profiles, and perhaps we should
> > be happy the tool won't work.
> 
> It was the big scary warning about the deprecation whenever I run
> emerge. It contains list of steps.

Ok.  I don't know if we can do anything about that.

> > Did it do anything to your system like creating a lib64 directory?  Does
> > anything work (because I have doubts on whether your system can still
> > find the libs in there now).
> 
> Yes. Attaching logs.

The logs seem to indicate that it thinks all libs on your systems do not
belong to any package.  This suggests the tool cannot locate the VDB or
something, as most of the things in the list are obviously owned by
packages.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?

2020-11-10 Thread Fabian Groffen
On 10-11-2020 09:49:27 +0100, Michał Górny wrote:
> > > So what you're saying is that you've had the wrong value of SYMLINK_LIB
> > > for ages, and now you've created a meaningless 17.1 profile that chnages
> > > it but isn't actually supposed to change anything, correct?
> > 
> > I guess, because the amd64 17.0 profile is deprecated with force, and I
> > had to do something ...
> 
> Now that's a lie.  Only the regular amd64 profiles are deprecated.
> There are no deprecation notices e.g. in the x32 profile or prefix
> profiles.

Our profiles either directly depend on the amd64/17.0 profile, or we use
a sub-profile from amd64/17.0 profile, so if it's going to get removed,
we are having a problem, don't we?


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?

2020-11-10 Thread Fabian Groffen
On 10-11-2020 09:34:52 +0100, Michał Górny wrote:
> On Tue, 2020-11-10 at 08:55 +0100, Fabian Groffen wrote:
> > On 09-11-2020 19:38:28 +, Alexey Sokolov wrote:
> > > Hi Fabian
> > > I tried to migrate my prefix to 17.1, and there are issues.
> > > 
> > > 1) unsymlink-lib requires "--root ~/gentoo" and otherwise produces an
> > > error "/usr/lib is a real directory! was the migration done already?"
> > 
> > I think unsymlink-lib doesn't have Prefix support, but in addition,
> > what unsymlink-lib is trying to achieve, is not a thing perhaps on
> > Prefix.
> > 
> > A prefix system (at least all of mine) doesn't have libXX or lib/XX
> > (a.k.a.  multilib) directories.  The /usr-split was long ago removed,
> > and thus what we have is:
> > 
> >   lib -> usr/lib
> > 
> > Now, SYMLINK_LIB=no seems to split into lib and lib64, but lib64 does
> > not exist on Prefix systems.
> > 
> 
> So what you're saying is that you've had the wrong value of SYMLINK_LIB
> for ages, and now you've created a meaningless 17.1 profile that chnages
> it but isn't actually supposed to change anything, correct?

I guess, because the amd64 17.0 profile is deprecated with force, and I
had to do something ...


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?

2020-11-09 Thread Fabian Groffen
On 09-11-2020 19:38:28 +, Alexey Sokolov wrote:
> Hi Fabian
> I tried to migrate my prefix to 17.1, and there are issues.
> 
> 1) unsymlink-lib requires "--root ~/gentoo" and otherwise produces an
> error "/usr/lib is a real directory! was the migration done already?"

I think unsymlink-lib doesn't have Prefix support, but in addition,
what unsymlink-lib is trying to achieve, is not a thing perhaps on
Prefix.

A prefix system (at least all of mine) doesn't have libXX or lib/XX
(a.k.a.  multilib) directories.  The /usr-split was long ago removed,
and thus what we have is:

  lib -> usr/lib

Now, SYMLINK_LIB=no seems to split into lib and lib64, but lib64 does
not exist on Prefix systems.

Since Prefix is non-multilib by design*, I wonder if unsymlink-lib is
necessary in the best case, but going to break the Prefix system in the
worst case.

What instructed you to perform the migration?  Was it the news-item?  I
don't think it should apply for Prefix profiles, and perhaps we should
be happy the tool won't work.

* non-multilib is a decision dating back a decade or so, which means
  effectively any Prefix install you encounter should be non-multilib


> 2) $ unsymlink-lib --root ~/gentoo --migrate --pretend
> usage: unsymlink-lib [-h] [-p] [--root ROOT] [--analyze] [--migrate]
>  [--rollback] [--finish] [--force-rollback]
>  [--resume-finish] [-P PREFIX] [--hardlink]
> unsymlink-lib: error: Requested action requires root privileges
> 
> Well, I worked it around by adding "is_root = True" to unsymlink-lib

Did it do anything to your system like creating a lib64 directory?  Does
anything work (because I have doubts on whether your system can still
find the libs in there now).

> 
> 3) Step 9 (Rebuild gcc) fails:
> configure:4372: checking whether the C compiler works
> 
> 
> 
> configure:4394: x86_64-pc-linux-gnu-gccconftest.c  >&5
> 
> 
> 
> /home/user/gentoo/usr/lib/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../x86_64-pc-linux-gnu/bin/as:
> error while loading shared libraries:
> libopcodes-2.34.0.gentoo-sys-devel-binutils-st.so: cannot open shared

Something like this I was suspecting.  Can you still rollback?  If you
can, I'd try that and hope it restores your system in working order.

For any Prefix user, DO NOT run unsymlink-lib, I believe you should NOT
NEED IT!

Thanks,
Fabian

> object file: No such file or directory
> configure:4398: $? = 1
> 
> 
> 
> configure:4436: result: no
> 
> The file exists:
> $ find ~ -name libopcodes-2.34.0.gentoo-sys-devel-binutils-st.so
> /home/user/gentoo/usr/lib/binutils/x86_64-pc-linux-gnu/2.34/libopcodes-2.34.0.gentoo-sys-devel-binutils-st.so
> 
> -- 
> Best regards,
> Alexey "DarthGandalf" Sokolov
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecating AMD64 17.0 profiles?

2020-11-07 Thread Fabian Groffen
On 07-11-2020 11:42:44 +, Alexey Sokolov wrote:
> 22.10.2020 20:16, Andreas K. Hüttel пишет:
> > Am Donnerstag, 22. Oktober 2020, 18:44:48 EEST schrieb Michał Górny:
> >> On Thu, 2020-10-22 at 11:17 -0400, Brian Evans wrote:
> >>> Users frequently are choosing the wrong profile versions in new installs
> >>> and accidentally downgrading to 17.0 with some saying they see it first.
> >>>
> >>> A simple reordering could help new installs.
> > 
> > Independent of this useful change, it's probably time to deprecate the 
> > amd64 
> > 17.0 profiles!
> > 
> 
> Prefix bootstrap script still makes new installations to use it

This should be solved with

b0445c0a8dd6d2f792c5bb088b154aca53868353
a9c478dc881ee18fefc7342da994b00e60eaad8e

on gentoo.git and

0d7f6b6eb00d0f51f35019846b8f79048b30be93

on prefix.git.

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] net-misc/asterisk fails to compile: clang/LLVM: bug 731280

2020-08-28 Thread Fabian Groffen
On 28-08-2020 08:52:09 +0100, Sergei Trofimovich wrote:
> On Fri, 28 Aug 2020 07:37:54 +0100
> Sergei Trofimovich  wrote:
> 
> > On Fri, 28 Aug 2020 08:15:47 +0200
> > Jaco Kroon  wrote:
> > 
> > > Hi All,
> > > 
> > > https://bugs.gentoo.org/731280
> > > 
> > > Summary:
> > > 
> > > This machine uses a clang/LLVM toolchain.
> > > Asterisk fails to compile, ./configure fails with:
> > > 
> > > checking for RAII support... checking for clang -fblocks...
> > > configure: error: BlocksRuntime is required for clang, please install
> > > libblocksruntime
> > > 
> > > I suspect this explains the issue:
> > > 
> > > https://github.com/mackyle/blocksruntime
> > > 
> > > I have no idea how to actually solve this.  
> > 
> > honggfuzz also needs it as it happens to use -fblocks on clang:
> > https://bugs.gentoo.org/729256
> > 
> > We need to package the library. I can give it a try.
> 
> Packaged library as sys-libs/blocksruntime:
> 
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ae376c73ef197d6c7aa619e821c436ccab0cd77e
> 
> Usage example for app-forensics/honggfuzz:
> 
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=fd841336dfdefbc14907e2d9b1eb1a1a3f5f8b8e

This is cool, but shouldn't it be something like openmp?  (e.g. blocks)

There is no reason blocks have to be used if not on macOS (where system
headers use blocks features).

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] euses(1) Reimplementation

2020-07-09 Thread Fabian Groffen
706#c4
> [4] https://dl.acm.org/doi/abs/10.1145/116825.116845
> [5] 
> https://sourceware.org/git/?p=glibc.git;a=blob;f=string/strcasestr.c;h=d2964c5548b9ea7a68fc5b18b25ddfe7ddd6835c;hb=HEAD#l45
> [6] http://git.suugaku.co.uk/ash-euses/tree/
> [7] http://git.suugaku.co.uk/ash-euses/snapshot/ash-euses-0.3.tar.gz
> 
> -- 
> 
> Ashley Dixon
> suugaku.co.uk
> 
> 2A9A 4117
> DA96 D18A
> 8A7B B0D2
> A30E BF25
> F290 A8AA
> 



-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] Speeding up Tree Verification

2020-07-01 Thread Fabian Groffen
On 30-06-2020 13:13:29 -0500, Sid Spry wrote:
> On Tue, Jun 30, 2020, at 1:20 AM, Fabian Groffen wrote:
> > Hi,
> > 
> > On 29-06-2020 21:13:43 -0500, Sid Spry wrote:
> > > Hello,
> > > 
> > > I have some runnable pseudocode outlining a faster tree verification 
> > > algorithm.
> > > Before I create patches I'd like to see if there is any guidance on 
> > > making the
> > > changes as unobtrusive as possible. If the radical change in algorithm is
> > > acceptable I can work on adding the changes.
> > > 
> > > Instead of composing any kind of structured data out of the portage tree 
> > > my
> > > algorithm just lists all files and then optionally batches them out to 
> > > threads.
> > > There is a noticeable speedup by eliding the tree traversal operations 
> > > which
> > > can be seen when running the algorithm with a single thread and comparing 
> > > it to
> > > the current algorithm in gemato (which should still be discussed here?).
> > 
> > I remember something that gemato used to use multiple threads, but
> > because it totally saturated disk-IO, it was brought back to a single
> > thread.  People were complaining about unusable systems.
> > 
> 
> I think this is an argument for cgroups limits support on the portage process 
> or
> account as opposed to an argument against picking a better algorithm. That is
> something I have been working towards, but I am only one man.

But this requires a) cgroups support, and b) the privileges to use it.
Shouldn't be a problem in the normal case, but just saying.

> > In any case, can you share your performance results?  What speedup did
> > you see, on warm and hot FS caches?  Which type of disk do you use?
> > 
> 
> I ran all tests multiple times to make them warm off of a Samsung SSD, but
> nothing very precise yet.
> 
> % gemato verify --openpgp-key signkey.asc /var/db/repos/gentoo
> [...]
> INFO:root:Verifying /var/db/repos/gentoo...
> INFO:root:/var/db/repos/gentoo verified in 16.45 seconds
> 
> sometimes going higher, closer to 18s, vs.
> 
> % ./veriftree.py
> 4.763171965983929
> 
> So roughly an order of magnitude speedup without batching to threads.

That is kind of a change.  Makes one wonder if you really did the same
work.

> > You could compare against qmanifest, which uses OpenMP-based
> > paralllelism while verifying the tree.  On SSDs this does help.
> > 
> 
> I lost my notes -- how do I specify to either gemato or qmanifest the GnuPG
> directory? My code is partially structured as it is because I had problems 
> doing
> this. I rediscovered -K/--openpgp-key in gemato but am unsure for qmanifest.

qmanifest doesn't do much magic out of the standard gnupg practices.
(It is using gpgme.)  If you want it to use a different gnupg dir, you
may change HOME, or GNUPGHOME.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] Speeding up Tree Verification

2020-06-30 Thread Fabian Groffen
: str) -> bool:
> response = dns.resolver.query(domain, dns.rdatatype.NS)
> nsname = response.rrset[0]
> response = dns.resolver.query(str(nsname), dns.rdatatype.A)
> nsaddr = response.rrset[0].to_text()
> 
> # DNSKEY
> request = dns.message.make_query(domain,
> dns.rdatatype.DNSKEY, want_dnssec=True)
> response = dns.query.udp(request, nsaddr)
> if response.rcode() != 0:
> raise Exception('Unable to request dnskey.')
> 
> answer = response.answer
> if len(answer) != 2:
> raise Exception('Malformed answer to dnskey query.')
> 
> name = dns.name.from_text(domain)
> try:
> dns.dnssec.validate(answer[0], answer[1], {name: answer[0]})
> except dns.dnssec.ValidationFailure as exc:
> # Validation failed. The raise causes python to abort with status 1.
> #raise exc
> return False
> except AttributeError as exc:
> # Validation may have failed; DNSKEY missing signer attribute. dig 
> may report
> # domain as valid.
> #
> # TODO: Additional state where subdomain of valid domain may fail 
> with 3 nested
> # KeyErrors. Avoid temptation to wildcard catch. Safer to put in 
> process?
> #raise exc
> return False
> else:
> return True
> 
> def hash_localpart(incoming: bytes) -> str:
> '''Z-base32 the localpart of an e-mail address
> 
> 
> https://tools.ietf.org/html/draft-koch-openpgp-webkey-service-08#section-3.1
> describes why this is needed.
> 
> See https://tools.ietf.org/html/rfc6189#section-5.1.6 for a
> description of the z-base32 scheme.
> '''
> zb32 = "ybndrfg8ejkmcpqxot1uwisza345h769"
> 
> b = hashlib.sha1(incoming).digest()
> ret = ""
> assert(len(b) * 8 == 160)
> for i in range(0, 160, 5):
> byte = i // 8
> offset = i - byte * 8
> # offset | bits remaining in k+1 | right-shift k+1
> # 3 | 0 | x
> # 4 | 1 | 7
> # 5 | 2 | 6
> # 6 | 3 | 5
> # 7 | 4 | 4
> if offset < 4:
> n = (b[byte] >> (3 - offset))
> else:
> n = (b[byte] << (offset - 3)) + (b[byte + 1] >> (11 - offset))
> 
> ret += zb32[n & 0b1]
> return ret
> 
> def build_web_key_uri(address: str) -> str:
> local, remote = address.split('@')
> local = hash_localpart(local.encode('utf-8'))
> return 'https://' + remote + '/.well-known/openpgpkey/hu/' + \
> local
> 
> def stream_to_file(uri: str, fname: str) -> None:
> with requests.get(uri, verify=True, stream=True) as r:
> from pprint import pprint
> pprint(r.headers)
> with open(fname, 'wb') as f:
> shutil.copyfileobj(r.raw, f)
> ```
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] Add caching to a few commonly used functions

2020-06-27 Thread Fabian Groffen
Hi Chun-Yu,

On 26-06-2020 23:34:12 -0700, Chun-Yu Shei wrote:
> Hi,
> 
> I was recently interested in whether portage could be speed up, since
> dependency resolution can sometimes take a while on slower machines.
> After generating some flame graphs with cProfile and vmprof, I found 3
> functions which seem to be called extremely frequently with the same
> arguments: catpkgsplit, use_reduce, and match_from_list.  In the first
> two cases, it was simple to cache the results in dicts, while
> match_from_list was a bit trickier, since it seems to be a requirement
> that it return actual entries from the input "candidate_list".  I also
> ran into some test failures if I did the caching after the
> mydep.unevaluated_atom.use and mydep.repo checks towards the end of the
> function, so the caching is only done up to just before that point.
> 
> The catpkgsplit change seems to definitely be safe, and I'm pretty sure
> the use_reduce one is too, since anything that could possibly change the
> result is hashed.  I'm a bit less certain about the match_from_list one,
> although all tests are passing.
> 
> With all 3 patches together, "emerge -uDvpU --with-bdeps=y @world"
> speeds up from 43.53 seconds to 30.96 sec -- a 40.6% speedup.  "emerge
> -ep @world" is just a tiny bit faster, going from 18.69 to 18.22 sec
> (2.5% improvement).  Since the upgrade case is far more common, this
> would really help in daily use, and it shaves about 30 seconds off
> the time you have to wait to get to the [Yes/No] prompt (from ~90s to
> 60s) on my old Sandy Bridge laptop when performing normal upgrades.
> 
> Hopefully, at least some of these patches can be incorporated, and please
> let me know if any changes are necessary.

This sounds like a good job to me!  Do you have any idea what the added
memory pressure for these changes are?

Thanks,
Fabian
> 
> Thanks,
> Chun-Yu
> 
> 
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH] Use env to find python

2020-06-17 Thread Fabian Groffen
On 16-06-2020 23:10:28 -0500, Sid Spry wrote:
> On Tue, Jun 16, 2020, at 3:57 PM, Michał Górny wrote:
> > '/usr/bin/env python' (with no extra options) is the portable shebang.
> > 
> 
> I added `-S` to preserve the options passed via the shebang line. It seems 
> they can be left off, does anyone know otherwise?

-S is not supported on some platforms.

In any case, I think this patch is wrong.  In the Prefix case, the
shebangs are set during install, in the non-Prefix case I think there is
machinery in place to replace the shebangs during install.

It seems the problem is already solved, why do you need these shebangs
changed?

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] New USE=-native-symlinks for gcc-config and binutils-config

2020-05-24 Thread Fabian Groffen
On 24-05-2020 10:48:47 +0100, Sergei Trofimovich wrote:
> On Sat, 23 May 2020 22:41:02 -0400
> Mike Gilbert  wrote:
> > I don't think we
> > want to give people the impression that this is a well-supported
> > configuration. I would only expect people to disable this if they want
> > to break their system intentionally.
> 
> Yeah, today it's certainly a way to get your system in a miserable state.
> My hope is that USE=-native-symlinks can get you a working Gentoo in a
> near future by only fixing real package problems and limitations of their
> build systems.

Portage adds PREROOTPATH, ROOTPATH and a standard set of
/usr/{local/,}{s,}bin to PATH in _doebuild_path.  That means if you add
something like /usr/bin/native-toolchain to PATH only, users will get
gcc, ld, etc., while root and Portage will lack these.  One can wonder
if root should have direct access to compilers, but it's easy enough to
add the compilers to PATH if really necessary.

I guess what I'm trying to say is: you can hide effect of the setup for
users if you'd like.  That is, after we had buildbots point out the bulk
of packages that are wrong of course.

Thanks,
Fabian
 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [RFC] Anti-spam for goose

2020-05-23 Thread Fabian Groffen
On 22-05-2020 21:58:54 +0200, Michał Górny wrote:
> Let's put it like this.  This thing starts working.  Package X is
> broken, and we see that almost nobody is using it.  We remove that
> package.  Now somebody is angry.  He submits a lot of fake data to
> render the service useless so that we don't make any future decisions
> based on it.

I'm affraid that has a heroic flair to me.  The service should never be
used for decisions like that, because it's a biased sample at most.
Doing stuff like this simply destroys the soul of the distribution.

I hope this isn't one of your genuine objectives with the service.  If
it is, I can see why you fear spam so much.

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [RFC] Anti-spam for goose

2020-05-21 Thread Fabian Groffen
 alternative of using a proof-of-work algorithm was suggested to me
> yesterday.  The idea is that every submission has to be accompanied with
> the result of some cumbersome calculation that can't be trivially run
> in parallel or optimized out to dedicated hardware.
> 
> On the plus side, it would rely more on actual physical hardware than IP
> addresses provided by ISPs.  While it would be a waste of CPU time
> and memory, doing it just once a week wouldn't be that much harm.
> 
> On the minus side, it would penalize people with weak hardware.
> 
> For example, 'time hashcash -m -b 28 -r test' gives:
> 
> - 34 s (-s estimated 38 s) on Ryzen 5 3600
> 
> - 3 minutes (estimated 92 s) on some old 32-bit Celeron M
> 
> At the same time, it would still permit a lot of fake submissions.  For
> example, randomx [1] claims to require 2G of memory in fast mode.  This
> would still allow me to use 7 threads.  If we adjusted the algorithm to
> take ~30 seconds, that means 7 submissions every 30 s, i.e. 20k
> submissions a day.
> 
> So in the end, while this is interesting, it doesn't seem like
> a workable anti-spam measure.
> 
> 
> Option 3: explicit CAPTCHA
> ==
> A traditional way of dealing with spam -- require every new system
> identifier to be confirmed by solving a CAPTCHA (or a few identifiers
> for one CAPTCHA).
> 
> The advantage of this method is that it requires a real human work to be
> performed, effectively limiting the ability to submit spam.
> The disadvantage is that it is cumbersome to users, so many of them will
> just resign from participating.
> 
> 
> Other ideas
> ===
> Do you have any other ideas on how we could resolve this?
> 
> 
> [1] https://github.com/tevador/RandomX
> 
> 
> -- 
> Best regards,
> Michał Górny
> 



-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] RFC: Gentoo Identity Provider

2020-05-19 Thread Fabian Groffen
On 18-05-2020 18:42:24 -0700, Alec Warner wrote:
> TL;DR: What if we launched id.gentoo.org[1], an identity provider that 
> provides
> authentication for Gentoo properties? Basically, 1 username / password for 
> wiki,
> bugs, email, forums, and any other http service[0][1].

I'd be in favour of SSO for all http-, imap- and smtp-based Gentoo services.

Thanks,
Fabian

> 
> Today Gentoo has numerous systems that mostly work in a segmented way.
> 
>  - To connect to hosts, we use ssh keys.
>  - Git is authenticated via ssh keys.
>  - Email uses LDAP passwords.
>  - Bugzilla has its own identities, with their own passwords.
>  - Wiki is separate, with its own passwords.
>  - Forums are separate.
>  - Infra has an additional 4 systems that use separate credentials.
> 
> Some applications support 2FA (such as wiki.)
> Some applications do not support 2FA.
> Applications that require 2FA have a configuration for each app, so you have N
> configurations.
> 
> If we configured id.gentoo.org[2] you would have 1 identity across all gentoo
> properties.
> 
> Is this a thing people are interested in?
>  
> [0] It's unlikely operations for git via ssh would change in this rollout.
> [1] Its unclear if the scope is "gentoo developers" or "any community member."
> The former have LDAP accounts and @gentoo.org[3] email addresses and so we can
> manage them easily; managing 1000s of other accounts in the IDP remains to be
> seem.
> 
> 
> References
>1. http://id.gentoo.org
>2. http://id.gentoo.org
>3. http://gentoo.org

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Last standing Python 2.7 dependency

2020-05-03 Thread Fabian Groffen
On 02-05-2020 23:24:42 -0700, Brian Dolbec wrote:
> On Sun, 3 May 2020 07:28:50 +0200
> Viktar Patotski  wrote:
> 
> > Hi all,
> > 
> > I'd also like to clean my system and have it Python 2.7 free. Are
> > there any guidelines to check which packages are still using pyton2_7
> > in my system?
> > 
> > Thanks,
> > Viktar
> > 
> 
> There are both equery and enalyze commands in gentoolkit that can give
> you reports about what pkgs are installed.
> 
> equery hasuse
> enalyze analyze [use|pkguse]
> 
> for help on them:
> equery -h
> equery hasuse -h
> enalyze -h
> enalyze a -h

In addition to these great tools, portage-utils' quse might also be
useful:

% quse python2_7
...


Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

2020-04-26 Thread Fabian Groffen
On 26-04-2020 21:29:42 +0200, Michał Górny wrote:
> On Sun, 2020-04-26 at 09:55 -0700, Matt Turner wrote:
> > Bug: https://bugs.gentoo.org/715108
> > Signed-off-by: Matt Turner 
> > ---
> > Strawman patch. Bikeshed away.
> > 
> 
> xz is generally slow and doesn't do parallel good.  If we want to change
> this, we should go for something cool like zstd that scales better.

I'd go for zstd too.  It seems to be the best of both worlds, good
compression at a good speed.

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [PATCH] fcaps.eclass: disable fcaps() on Prefix.

2020-03-07 Thread Fabian Groffen
On 08-03-2020 15:26:50 +0800, hero...@gentoo.org wrote:
> From: Benda Xu 
> 
> Gentoo Prefix runs with a normal user and cannot grant extra
> capabilities.  Exit gracefully with a message.
> 
> Signed-off-by: Benda Xu 
> ---
>  eclass/fcaps.eclass | 5 +
>  1 file changed, 5 insertions(+)
> 
> diff --git a/eclass/fcaps.eclass b/eclass/fcaps.eclass
> index 467f955f5e9a..563d177c92d5 100644
> --- a/eclass/fcaps.eclass
> +++ b/eclass/fcaps.eclass
> @@ -78,6 +78,11 @@ DEPEND="filecaps? ( sys-libs/libcap )"
>  fcaps() {
>   debug-print-function ${FUNCNAME} "$@"
>  
> + if [[ ${EUID} != 0 ]] ; then
> + einfo "Insufficient privileges to execute ${FUNCNAME[0]}"
> + return 0

Perhaps add "ignoring" or something in the message here to make it clear
this isn't treated as an error?

Thanks,
Fabian

> + fi
> +
>   # Process the user options first.
>   local owner='root'
>   local group='0'
> -- 
> 2.25.0
> 
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Keywordreqs and slacking arch teams

2019-12-28 Thread Fabian Groffen
On 28-12-2019 22:27:02 +1300, Kent Fredric wrote:
> On Sat, 28 Dec 2019 08:09:33 +0100
> Michał Górny  wrote:
> 
> > How can we improve this?
> 
> Every time this kind of issue comes up, I can't help feeling we need
> some sort of tool more advanced than what we currently have.
> 
> Something that maintains persistence of keyword demands similar to how
> the current bot does, but more ... 
> 
> Its the sort of thing I get tempted to implement myself, but get too
> horrified about the prospect of working with portage internals to do it.

Hmmm, interested to hear what kind of things you're thinking about here.

I feel it would help if we would have the ability to at least
compile/test ebuilds automatically.  Not sure how helpful qemu could be
there, but I know things like compiling for things like arm (RPi) works
reasonably well.

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH] eapply: Drop -s option for patch.

2019-12-13 Thread Fabian Groffen
On 13-12-2019 14:24:33 -0500, Michael Orlitzky wrote:
> On 12/13/19 9:28 AM, Fabian Groffen wrote:
> > 
> > We are providing those patches, maybe.  In reality very often the
> > patches originate from somewhere else though.  And you don't want to
> > have to respin all of those just because.  At least that's what I feel.
> > 
> 
> Just because... the context changed? A new "!" in a line of context can
> be the difference between letting someone log in with the right password
> and letting them log in with the wrong password. You should at least
> have to stop and verify that the patch does what you think it does when
> it "gains" fuzz. And at that point, git diff will give you a clean
> version of it.

Counter argument is that we've been doing this for decades, and always
relied on maintainers to check the contents of their patches, without
problems.  We didn't introduce a predictable random number generator or
something.
Your very specific example just illustrates the niche this proposal is
targetting.

As with many of the proposals lately, they just seem to aim at more work
for individual maintainers, with a very low gain ratio.
Better, allow this to be a FEATURE, or whatever, that devs should enable,
but don't spit this in the user's face by default.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH] eapply: Drop -s option for patch.

2019-12-13 Thread Fabian Groffen
On 13-12-2019 15:24:40 +0100, Michał Górny wrote:
> > > ...and why do we consider it correct to apply patches when the context
> > > doesn't match?  If our only goal is to make things 'easier' for
> > > 'everyone', then we could just pass -F and ignore all the context.
> > > 
> > > Though I don't understand why include any context in the first place if
> > > you don't care about it matching.  Sounds like a waste of space to me!
> > 
> > The patch command defaults to -F2. If that makes no sense, why is it
> > the upstream default?
> > 
> 
> You should ask upstream, not me.  But if I were to guess, the answer
> would be because patch(1) is used by random people trying to apply
> random patches they've found somewhere.  We on the other hand are
> applying patches that *we* are supposed to provide.

We are providing those patches, maybe.  In reality very often the
patches originate from somewhere else though.  And you don't want to
have to respin all of those just because.  At least that's what I feel.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] New distfile mirror layout

2019-10-29 Thread Fabian Groffen
On 29-10-2019 15:45:34 +0100, Michał Górny wrote:
> On Tue, 2019-10-29 at 15:33 +0100, Fabian Groffen wrote:
> > In addition, there are currently files there that aren't referenced from
> > ebuilds.  Prefix uses these files during bootstrap, local mirrors are
> > often much faster than dev.g.o.
> > 
> > If the files don't get mirrored anymore, I guess I can create a dummy
> > ebuild that has the files in SRC_URI.
> 
> Ok, this is something I wasn't aware of.  I agree that dummy ebuild
> should not be necessary here.  However, I'm also not sure if distfiles-
> local is really the proper way either, especially that I don't see such
> files on woodpecker right now.

There should be /space/distfiles-local and
/space/distfiles-whitelist/prefix with a list of files to retain on the
mirror.

Thanks,
Fabian

> I don't think the matter is urgent right now, so let's ponder on it
> a bit.  In particular, I think we should have a clear indication of who
> added which files, when, what for and where they came from.  Those are
> precisely the things that the current distfiles-local approach misses.
> 
> > If the files get mirrored, but put in a subdir based on the filename
> > hash, the original query endpoint on distfiles.g.o changes, much like
> > the SRC_URI approach.
> > 
> > Now I can use distfiles.prefix.b.n which redirects to the distfiles.g.o
> > URL with subdir for most part I think, but it's sub-optimal from my
> > point of view.  Calculating the hash is not always feasible due to the
> > lack of b2sum or other means.  Hence my earlier request to have such
> > official translation service on Gentoo hardware.
> > 
> > (I just wrote a small wsgi script that calculates the hash and generates
> > the redirect from Python, served via uwsgi/nginx, but there should be
> > many ways to achieve the same goals, if and only if a blake2b
> > implementation were available for it.)
> 
> This is also something that needs thinking.  I personally don't mind
> having one but it would be nice if it was able to account for geodns
> and such.
> 
> -- 
> Best regards,
> Michał Górny
> 



-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] New distfile mirror layout

2019-10-29 Thread Fabian Groffen
On 29-10-2019 15:17:38 +0100, Ulrich Mueller wrote:
> >>>>> On Tue, 29 Oct 2019, Michał Górny wrote:
> 
> > On Tue, 2019-10-29 at 14:09 +0100, Ulrich Mueller wrote:
> >> > What if the file is hosted at a non-standard tcp port upstream
> >> > (like http://example.org:8080/)? The devmanual says that it _must_
> >> > be manually uploaded to /space/distfiles-local/ in such cases.
> 
> >> Or another example, app-emacs/vhdl-mode-3.38.1, where (incompetent,
> >> or nasty?) upstream blocks wget for some reason, but other methods
> >> (e.g., curl, firefox) work? How would I get the file onto the mirrors
> >> there?
> 
> > If I were you, I would've explicitly mirrored the file anyway.
> > If upstream blocks wget, then users who do not use GENTOO_MIRRORS will
> > also suffer due to it.
> 
> All what I'm saying is that there can be unusual circumstances where
> manual uploading of a file is useful. So please don't take that
> possibility away.

In addition, there are currently files there that aren't referenced from
ebuilds.  Prefix uses these files during bootstrap, local mirrors are
often much faster than dev.g.o.

If the files don't get mirrored anymore, I guess I can create a dummy
ebuild that has the files in SRC_URI.

If the files get mirrored, but put in a subdir based on the filename
hash, the original query endpoint on distfiles.g.o changes, much like
the SRC_URI approach.

Now I can use distfiles.prefix.b.n which redirects to the distfiles.g.o
URL with subdir for most part I think, but it's sub-optimal from my
point of view.  Calculating the hash is not always feasible due to the
lack of b2sum or other means.  Hence my earlier request to have such
official translation service on Gentoo hardware.

(I just wrote a small wsgi script that calculates the hash and generates
the redirect from Python, served via uwsgi/nginx, but there should be
many ways to achieve the same goals, if and only if a blake2b
implementation were available for it.)

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] New distfile mirror layout

2019-10-29 Thread Fabian Groffen
On 29-10-2019 05:27:37 +0100, Michał Górny wrote:
> On Tue, 2019-10-29 at 00:24 +0100, Chí-Thanh Christopher Nguyễn wrote:
> > Hi!
> > 
> > > Today you get chastised for using /space/distfiles-local and not
> > > following policy changes.  The devmanual states that it's deprecated
> > > since at least 2011, and talks of using d.g.o [1].
> >  > [1] 
> > https://devmanual.gentoo.org/general-concepts/mirrors/index.html#suitable-download-hosts
> > 
> > Sorry I'm late to the party, but I would like to enquire about what happens 
> > if a file with existing filename but different b2sum gets uploaded to 
> > /space/distfiles-local now?
> 
> The same as before.  It gets put in top-level disfiles directory. 
> Hashes are calculated from filenames, so this wouldn't affect it.  That
> is, if it put those files in subdirectories in the first place because
> it doesn't.

/space/distfiles-local is no longer copied to the mirrors? or just not
copied in the subdir-hierarchy?

> > Doing so and updating the Manifest used to be another (not necessarily 
> > preferred) method to address upstream remaking release packages.
> > 
> 
> It's no longer valid.

Just wondering.  Do you mean it isn't valid that some upstreams do this
(yes horror)?  We surely need a way to work around that ...

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] New distfile mirror layout

2019-10-19 Thread Fabian Groffen
Hi,

On 18-10-2019 15:41:32 +0200, Michał Górny wrote:
> 3. Directly fetching files from distfiles.gentoo.org will become
> a little harder.  To fetch a distfile named 'foo-1.tar.gz', you'd have
> to use something like:
> 
> $ printf '%s' foo-1.tar.gz | b2sum | cut -c1-2
> 1b
> $ wget http://distfiles.gentoo.org/distfiles/1b/foo-1.tar.gz
> ...
> 
> 
> Alternatively, you can:
> 
> $ wget http://distfiles.gentoo.org/distfiles/INDEX
> 
> and grep for the right path there.  This INDEX is also a more
> lightweight alternative to HTML indexes generated by the servers.

Would it be possible to run a service that sends a 302 for the
distfiles/foo-1.tar.gz to the appropriate bucket such that manual
fetching doesn't require to calculate the hash?

I prototyped this myself for distfiles.prefix, and seems like a nice
guesture for at least the transition period?

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Underscores in USE flags

2019-09-21 Thread Fabian Groffen
On 21-09-2019 09:06:01 +0200, Michał Górny wrote:
> On Sat, 2019-09-21 at 08:43 +0200, Fabian Groffen wrote:
> > Why not teach our tools (equery, quse, etc.) to print these USE-flags
> > like Portage does?  (looking them up to be valid expands)
> > Then users have nothing to be confused about (no distinction between
> > foo_bar and FOO="bar"), and new USE_EXPANDS cannot be
> > silently/accidentially introduced.
> 
> I don't see how that solves the problem.  More tools having distinct
> output don't change the fact that anyone with a bit of ebuild knowledge
> will say 'this looks like USE_EXPAND' while looking at it, independently
> of what some tools would say.

Well... someone with a bit of ebuild knowledge would see odd USE-flags.
USE_EXPAND is a (bad) hack, of having some USE-flags mean something
different, or resolve through something different, while in reality they
really don't do anything odd.  In fact, sometimes users have to use
FOO="bar" (make.conf), while other times foo_bar needs to be used
(e.g. use.mask, or IUSE=).

Consistency would be nice, the real question is, what does USE_EXPAND
actually try to achieve, and can we fix it properly in the next EAPI,
such that repoman can also do the proper complaints about USE-flag
(and USE_EXPAND-flag) naming by then.

Back to the thread, the point is, these flags exist today, and renaming
flags is not something to be considered harmless.  As much as the recent
renaming of lm_sensors to lm-sensors caused breakage (and still does,
apparently some tools keep caches, etc.) also renaming USE-flags goes by
problems, in particular for managed systems (Chef, Puppet).  It's not a
matter of just fixing the name for a USE-flag.  This is saying nothing
about whether or not we'd want to change the flag.  It's about the
impact of the change, and whether that is worth it for the noble aim of
consistency or correctness.  I believe this was the OPs point in this
thread.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Underscores in USE flags

2019-09-21 Thread Fabian Groffen
On 20-09-2019 22:53:53 +0200, Michał Górny wrote:
> On Fri, 2019-09-20 at 13:46 -0700, Zac Medico wrote:
> > 
> > If we take this underscore rule to its logical extreme, then we should
> > rename python_targets_python3_7 to python_targets_python3-7, yes?
> 
> Believe me, I would have done that already if not the fact that with all
> the dependency logic around here it would be totally destructive to all
> Gentoo systems.

Honestly, with this reasoning, why force other packages to go through
USE-flag renaming in that case?  A major consumer of USE_EXPAND isn't
sticking to the rule, which makes any benefit of it moot.  Tools cannot
assume the last underscore separates the USE_EXPAND var from its value,
users cannot see what is the value either, without knowledge.

Why not teach our tools (equery, quse, etc.) to print these USE-flags
like Portage does?  (looking them up to be valid expands)
Then users have nothing to be confused about (no distinction between
foo_bar and FOO="bar"), and new USE_EXPANDS cannot be
silently/accidentially introduced.

> But hey, expect hyphen on 3.8.

I honestly feel for consistency and not confusing users, we should
either do them all or stick to the current scheme.

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-dev] Last rites: app-portage/hashgen

2019-09-15 Thread Fabian Groffen
# Fabian Groffen  (2019-09-15)
# Incorporated in >=app-portage/portage-utils-0.80 as qmanifest
# Removal in 30 days.  Bug #694428
app-portage/hashgen


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [PATCH] use.desc: Introduce global USE=magic

2019-08-20 Thread Fabian Groffen
Hi Michał,

It looks like you missed two from your original set?

app-editors/nano[magic] Add magic file support (sys-apps/file) to
automatically detect appropriate syntax highlighting
www-servers/pshs[magic] Enable automatic detection of Content-Type using
libmagic (sys-apps/file)

Thanks,
Fabian

On 19-08-2019 22:04:12 +0200, Michał Górny wrote:
> On Sun, 2019-08-11 at 13:21 +0200, Michał Górny wrote:
> > USE=magic is currently used consistently by 12 packages:
> > 
> 
> Merged.
> 
> -- 
> Best regards,
> Michał Górny
> 



-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes

2019-07-26 Thread Fabian Groffen
Hi,

On 25-07-2019 14:20:50 +0200, Michał Górny wrote:
> Hi,
> 
> TL;DR: I'd like to make it possible for ebuilds to define additional
> variables that will be stored in md5-cache.  This would be useful for CI
> and other tooling that right now has to parse ebuilds for other data.

Only downside I can think of, is a diskspace increase for the md5-cache.
Not sure if this is going to be substantial, but given things like
PYTHON_COMPAT, perhaps a quick calculation of extra "cost" can be made.
Should diskspace become a problem, one could consider to use a separate
file/dir, that users could rsync-exclude, since Portage won't need it to
operate properly.

Thanks,
Fabian

> 
> 
> The idea is to add a new incremental ebuild/eclass variable (technical
> name: QA_EXTRA_CACHE_VARS) that would define additional data to be
> stored in cache.  For example, python*-r1 eclasses would define
> 'PYTHON_COMPAT', acct-user would define 'ACCT_USER_ID', etc.
> 
> When regenerating cache, the PM would read this variable, and store
> the values of all defined variables into md5-cache.  As a result,
> programs needing those variables can get them straight from cache
> without having to attempt to run or parse ebuilds (which is both slow
> and prone to bugs).
> 
> This would benefit e.g. gpyutils that right now need to attempt to parse
> PYTHON_COMPAT from ebuilds.  It would also benefit writing future
> pkgcheck checks for user/group ID collisions.
> 
> 
> Notes:
> 
> - since md5-cache uses key-value format and allows for future
> extensions, the new values can be added without breaking anything;
> 
> - md5-cache is not specified in the PMS, and the whole thing can be
> implemented without need for EAPI bump,
> 
> - I would like to have this implemented consistently both in Portage
> and pkgcore,
> 
> - we will need to clearly define how to dump arrays.
> 
> 
> What do you think?
> 
> -- 
> Best regards,
> Michał Górny
> 



-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [RFC] New QA policy: Packages must not disable installing manpages via USE flags

2019-07-21 Thread Fabian Groffen
On 21-07-2019 23:47:02 +1200, Kent Fredric wrote:
> Yes, yes, I'm suggesting something perverted like a build server or
> system for aggregating built man-pages on gentoo-servers automatically
> as part of CI, that end users can just trivially fetch. But that's just
> one approach, surely, there are others.

Right, or we go for some (official) form of binpkgs distribution.

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-dev] Re: RFC: isodate for packages.mask starting on 2019-07-01

2019-06-29 Thread Fabian Groffen
I think ISO 8601 date format is an improvement.

However, as you're suggesting to use a date at UTC time, this begs for
scripting, such as the old echangelog did.  Then, when scripted, the
actual date format is no longer an issue.  Somehow the current format
looks easier to read (for humans) to me, so no real need to change if
there was a script to just add/remove/update entries.  (And commit
without forgetting to add --signoff.)

Fabian

On 29-06-2019 13:23:10 +0200, Jonas Stein wrote:
> Dear all,
> 
> Situation:
> We have different date formats in packages.mask.
> 
> 
> Change:
> I suggest that we start using the date format -mm-dd
> for all dates in packages.mask
> starting with 2019-07-01
> 
> The following changes in packages.mask will introduce the date format,
> specify the timezone, and use Larry as example user.
> 
> 3,6c3,6
> < # When you add an entry to the top of this file, add your name, the
> date, and
> < # an explanation of why something is getting masked. Please be extremely
> < # careful not to commit atoms that are not valid, as it can cause
> large-scale
> < # breakage, especially if it ends up in the daily snapshot.
> ---
> > # When you add an entry to the top of this file, add your name, the date
> > # in the UTC timezone, and an explanation of why something is getting
> masked.
> > # Please be extremely careful not to commit atoms that are not valid,
> as it can
> > # cause large-scale breakage, especially if it ends up in the daily
> snapshot.
> 10c10
> < ## # Dev E. Loper  (28 Jun 2012)
> ---
> > ## # Larry the cow  (2019-07-01)
> 24,26c24,26
> < ## # Dev E. Loper  (23 May 2015)
> < ## # Masked for removal in 30 days.  Doesn't work
> < ## # with new libfoo. Upstream dead, gtk-1, smells
> ---
> > ## # Larry the cow  (2019-07-01)
> > ## # Masked for removal after 2019-08-01.
> > ## # Doesn't work with new libfoo. Upstream dead, gtk-1, smells
> 
> 
> Reason:
> * Larry is the Gentoo Example
> * 2019-01-01 + 30 days is unclear, if we do not use UTC time
> * The new date format is easy to read and write and easy to parse
> internationally.
> 
> Do you have any objections?
> 
> 
> By the way, you can get a formatted string of now in UTC with:
> date -u +"%Y-%m-%d"
> 
> -- 
> Best,
> Jonas
> 




-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Deprecation and removal of 13.0 profiles is imminent

2019-02-20 Thread Fabian Groffen
On 19-02-2019 22:52:10 +, Sergei Trofimovich wrote:
> A few months passed and we almost finished with a long tail.
> 
> To-be-deprecated profiles yet are:
>   https://bugs.gentoo.org/673278
> prefix/linux-standalone/ppc64
> 
> +prefix@, what is our plan here? Can we drop at least empty
> 'deprecated' file to notify users?

I cleaned it up, it isn't in use any more.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH v5] collision_protect: use dynamic report interval

2019-01-11 Thread Fabian Groffen
Pushed, thanks

On 10-01-2019 20:37:16 -0800, Zac Medico wrote:
> On 1/10/19 7:30 AM, Fabian Groffen wrote:
> > The reporting of files remaining can look somewhat odd since the report
> > interval is hardcoded to be per 1000 objects.  Adjust this interval to
> > be time based.  This means that modern (fast) machines likely will never
> > see the countdown messages at all.  On slow setups the message will be
> > informative that there is progress, albeit rather slowly.  While at it,
> > report percentage done.
> > 
> > Output before this patch:
> > 
> >  * checking 6158 files for package collisions
> > 5158 files remaining ...
> > 4158 files remaining ...
> > 3158 files remaining ...
> > 2158 files remaining ...
> > 1158 files remaining ...
> > 158 files remaining ...
> > 
> > Possible output after this patch on a slower machine:
> > 
> >  * checking 6158 files for package collisions
> >  48% done,  3145 files remaining ...
> >  96% done,   192 files remaining ...
> > 100% done
> > 
> > Signed-off-by: Fabian Groffen 
> > ---
> >  lib/portage/dbapi/vartree.py | 15 +--
> >  1 file changed, 13 insertions(+), 2 deletions(-)
> > 
> > diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
> > index 4b91caea8..909c0e473 100644
> > --- a/lib/portage/dbapi/vartree.py
> > +++ b/lib/portage/dbapi/vartree.py
> > @@ -35,6 +35,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
> > 'portage.util.install_mask:install_mask_dir,InstallMask',
> > 'portage.util.listdir:dircache,listdir',
> > 'portage.util.movefile:movefile',
> > +   'portage.util.monotonic:monotonic',
> > 'portage.util.path:first_existing,iter_parents',
> > 'portage.util.writeable_check:get_ro_checker',
> > 'portage.util._xattr:xattr',
> > @@ -3475,13 +3476,21 @@ class dblink(object):
> > symlink_collisions = []
> > destroot = self.settings['ROOT']
> > totfiles = len(file_list) + len(symlink_list)
> > +   previous = monotonic()
> > +   progress_shown = False
> > +   report_interval = 1.7  # seconds
> > +   falign = len("%d" % totfiles)
> > showMessage(_(" %s checking %d files for package 
> > collisions\n") % \
> > (colorize("GOOD", "*"), totfiles))
> > for i, (f, f_type) in enumerate(chain(
> > ((f, "reg") for f in file_list),
> > ((f, "sym") for f in symlink_list))):
> > -   if i % 1000 == 0 and i != 0:
> > -   showMessage(_("%d files remaining 
> > ...\n") % (totfiles - i))
> > +   current = monotonic()
> > +   if current - previous > report_interval:
> > +   showMessage(_("%3d%% done,  %*d files 
> > remaining ...\n") %
> > +   (i * 100 / totfiles, 
> > falign, totfiles - i))
> > +   previous = current
> > +   progress_shown = True
> >  
> > dest_path = normalize_path(
> > os.path.join(destroot, 
> > f.lstrip(os.path.sep)))
> > @@ -3570,6 +3579,8 @@ class dblink(object):
> > break
> > if stopmerge:
> > collisions.append(f)
> > +   if progress_shown:
> > +   showMessage(_("100% done\n"))
> > return collisions, dirs_ro, symlink_collisions, 
> > plib_collisions
> >  
> > def _lstat_inode_map(self, path_iter):
> > 
> 
> Looks good!
> -- 
> Thanks,
> Zac
> 




-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-portage-dev] [PATCH v5] collision_protect: use dynamic report interval

2019-01-10 Thread Fabian Groffen
The reporting of files remaining can look somewhat odd since the report
interval is hardcoded to be per 1000 objects.  Adjust this interval to
be time based.  This means that modern (fast) machines likely will never
see the countdown messages at all.  On slow setups the message will be
informative that there is progress, albeit rather slowly.  While at it,
report percentage done.

Output before this patch:

 * checking 6158 files for package collisions
5158 files remaining ...
4158 files remaining ...
3158 files remaining ...
2158 files remaining ...
1158 files remaining ...
158 files remaining ...

Possible output after this patch on a slower machine:

 * checking 6158 files for package collisions
 48% done,  3145 files remaining ...
 96% done,   192 files remaining ...
100% done

Signed-off-by: Fabian Groffen 
---
 lib/portage/dbapi/vartree.py | 15 +--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
index 4b91caea8..909c0e473 100644
--- a/lib/portage/dbapi/vartree.py
+++ b/lib/portage/dbapi/vartree.py
@@ -35,6 +35,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.install_mask:install_mask_dir,InstallMask',
'portage.util.listdir:dircache,listdir',
'portage.util.movefile:movefile',
+   'portage.util.monotonic:monotonic',
'portage.util.path:first_existing,iter_parents',
'portage.util.writeable_check:get_ro_checker',
'portage.util._xattr:xattr',
@@ -3475,13 +3476,21 @@ class dblink(object):
symlink_collisions = []
destroot = self.settings['ROOT']
totfiles = len(file_list) + len(symlink_list)
+   previous = monotonic()
+   progress_shown = False
+   report_interval = 1.7  # seconds
+   falign = len("%d" % totfiles)
showMessage(_(" %s checking %d files for package 
collisions\n") % \
(colorize("GOOD", "*"), totfiles))
for i, (f, f_type) in enumerate(chain(
((f, "reg") for f in file_list),
((f, "sym") for f in symlink_list))):
-   if i % 1000 == 0 and i != 0:
-   showMessage(_("%d files remaining 
...\n") % (totfiles - i))
+   current = monotonic()
+   if current - previous > report_interval:
+   showMessage(_("%3d%% done,  %*d files 
remaining ...\n") %
+   (i * 100 / totfiles, 
falign, totfiles - i))
+   previous = current
+   progress_shown = True
 
dest_path = normalize_path(
os.path.join(destroot, 
f.lstrip(os.path.sep)))
@@ -3570,6 +3579,8 @@ class dblink(object):
break
if stopmerge:
collisions.append(f)
+   if progress_shown:
+   showMessage(_("100% done\n"))
return collisions, dirs_ro, symlink_collisions, 
plib_collisions
 
def _lstat_inode_map(self, path_iter):
-- 
2.20.1




Re: [gentoo-portage-dev] [PATCH v4] collision_protect: use dynamic report interval

2019-01-10 Thread Fabian Groffen
On 09-01-2019 21:22:26 -0800, Zac Medico wrote:
> > +   if tnow + tinterv < ninterv:
> > +   showMessage(_("100% done\n"))
> 
> It took me a moment to understand the calculation here. How about if we
> do something like this instead:
> 
> 
> previous = monotonic()
> progress_shown = False

You're right.  I should've gone for readability in the first place.
There is no point in trying to save anything here.

Thanks,
Fabian

> for f in files:
>   current = monotonic()
>   if current - previous > tinterv:
>   showMessage(...)
>   previous = current
>   progress_shown = True
> 
> if progress_shown:
>   showMessage(_("100% done\n"))
> 
> 
> 
> > return collisions, dirs_ro, symlink_collisions, 
> > plib_collisions
> >  
> > def _lstat_inode_map(self, path_iter):
> > 
> 
> 
> -- 
> Thanks,
> Zac
> 




-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-portage-dev] [PATCH v4] collision_protect: use dynamic report interval

2019-01-09 Thread Fabian Groffen
The reporting of files remaining can look somewhat odd since the report
interval is hardcoded to be per 1000 objects.  Adjust this interval to
be time based.  This means that modern (fast) machines likely will never
see the countdown messages at all.  On slow setups the message will be
informative that there is progress, albeit rather slowly.  While at it,
report percentage done.

Output before this patch:

 * checking 6158 files for package collisions
5158 files remaining ...
4158 files remaining ...
3158 files remaining ...
2158 files remaining ...
1158 files remaining ...
158 files remaining ...

Possible output after this patch on a slower machine:

 * checking 6158 files for package collisions
 48% done,  3145 files remaining ...
 96% done,   192 files remaining ...
100% done

Signed-off-by: Fabian Groffen 
---
 lib/portage/dbapi/vartree.py | 13 +++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
index 4b91caea8..df192e6fc 100644
--- a/lib/portage/dbapi/vartree.py
+++ b/lib/portage/dbapi/vartree.py
@@ -35,6 +35,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
'portage.util.install_mask:install_mask_dir,InstallMask',
'portage.util.listdir:dircache,listdir',
'portage.util.movefile:movefile',
+   'portage.util.monotonic:monotonic',
'portage.util.path:first_existing,iter_parents',
'portage.util.writeable_check:get_ro_checker',
'portage.util._xattr:xattr',
@@ -3475,13 +3476,19 @@ class dblink(object):
symlink_collisions = []
destroot = self.settings['ROOT']
totfiles = len(file_list) + len(symlink_list)
+   tnow = monotonic()
+   tinterv = 2  # seconds
+   ninterv = tnow + tinterv
+   falign = len("%d" % totfiles)
showMessage(_(" %s checking %d files for package 
collisions\n") % \
(colorize("GOOD", "*"), totfiles))
for i, (f, f_type) in enumerate(chain(
((f, "reg") for f in file_list),
((f, "sym") for f in symlink_list))):
-   if i % 1000 == 0 and i != 0:
-   showMessage(_("%d files remaining 
...\n") % (totfiles - i))
+   if monotonic() > ninterv:
+   showMessage(_("%3d%% done,  %*d files 
remaining ...\n") %
+   (falign, i * 100 / 
totfiles, totfiles - i))
+   ninterv = monotonic() + tinterv
 
dest_path = normalize_path(
os.path.join(destroot, 
f.lstrip(os.path.sep)))
@@ -3570,6 +3577,8 @@ class dblink(object):
break
if stopmerge:
collisions.append(f)
+   if tnow + tinterv < ninterv:
+   showMessage(_("100% done\n"))
return collisions, dirs_ro, symlink_collisions, 
plib_collisions
 
def _lstat_inode_map(self, path_iter):
-- 
2.20.1




Re: [gentoo-portage-dev] [PATCH v3] collision_protect: use dynamic report interval

2019-01-09 Thread Fabian Groffen
On 08-01-2019 20:59:34 +, M. J. Everitt wrote:
> On 08/01/19 19:15, Zac Medico wrote:
> > On 1/8/19 5:42 AM, Fabian Groffen wrote:
> >> The reporting of files remaining can look somewhat odd since the report
> >> interval is hardcoded to be per 1000 objects.  Adjust this interval to
> >> be time based.  This means that modern (fast) machines likely will never
> >> see the countdown messages at all.  On slow setups the message will be
> >> informative that there is progress, albeit rather slowly.  While at it,
> >> report percentage done.
> >>
> >> Output before this patch:
> >>
> >>  * checking 6158 files for package collisions
> >> 5158 files remaining ...
> >> 4158 files remaining ...
> >> 3158 files remaining ...
> >> 2158 files remaining ...
> >> 1158 files remaining ...
> >> 158 files remaining ...
> >>
> >> Possible output after this patch on a slower machine:
> >>
> >>  * checking 6158 files for package collisions
> >>  48% done, 3145 files remaining ...
> >>  96% done, 192 files remaining ...
> >> 100% done
> >>
> >> Signed-off-by: Fabian Groffen 
> >> ---
> >>  lib/portage/dbapi/vartree.py | 11 +--
> >>  1 file changed, 9 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
> >> index 4b91caea8..244195fad 100644
> >> --- a/lib/portage/dbapi/vartree.py
> >> +++ b/lib/portage/dbapi/vartree.py
> >> @@ -3475,13 +3475,18 @@ class dblink(object):
> >>symlink_collisions = []
> >>destroot = self.settings['ROOT']
> >>totfiles = len(file_list) + len(symlink_list)
> >> +  tnow = time.time()
> >> +  tinterv = 2  # seconds
> >> +  ninterv = tnow + tinterv
> >>showMessage(_(" %s checking %d files for package 
> >> collisions\n") % \
> >>(colorize("GOOD", "*"), totfiles))
> >>for i, (f, f_type) in enumerate(chain(
> >>((f, "reg") for f in file_list),
> >>((f, "sym") for f in symlink_list))):
> >> -  if i % 1000 == 0 and i != 0:
> >> -  showMessage(_("%d files remaining 
> >> ...\n") % (totfiles - i))
> >> +  if time.time() > ninterv:
> >> +  showMessage(_("%3d%% done, %d files 
> >> remaining ...\n") %
> >> +  (i * 100 / totfiles, 
> >> totfiles - i))
> >> +  ninterv = time.time() + tinterv
> >>  
> >>dest_path = normalize_path(
> >>os.path.join(destroot, 
> >> f.lstrip(os.path.sep)))
> >> @@ -3570,6 +3575,8 @@ class dblink(object):
> >>break
> >>if stopmerge:
> >>collisions.append(f)
> >> +  if tnow + tinterv < ninterv:
> >> +  showMessage(_("100% done\n"))
> >>return collisions, dirs_ro, symlink_collisions, 
> >> plib_collisions
> >>  
> >>def _lstat_inode_map(self, path_iter):
> >>
> > Please replace time.time() with portage.util.monotonic.monotonic().

Will do.

> It's a bit of a nit-pick, granted, but can we ensure that the count-down's
> remain padded/justified such that the numbers line up for easy at-a-glance
> inspection ? The optimal standard looks somewhat like the pre-merge Sizes:
> 
>  * Final size of build directory: 2696 KiB (2.6 MiB)
>  * Final size of installed tree:  5372 KiB (5.2 MiB)
> 
> Otherwise, I think this will be quite helpful. Thanks.

I think I understand what you mean, let me see if I can do something
without too much complexity.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-portage-dev] [PATCH v3] collision_protect: use dynamic report interval

2019-01-08 Thread Fabian Groffen
The reporting of files remaining can look somewhat odd since the report
interval is hardcoded to be per 1000 objects.  Adjust this interval to
be time based.  This means that modern (fast) machines likely will never
see the countdown messages at all.  On slow setups the message will be
informative that there is progress, albeit rather slowly.  While at it,
report percentage done.

Output before this patch:

 * checking 6158 files for package collisions
5158 files remaining ...
4158 files remaining ...
3158 files remaining ...
2158 files remaining ...
1158 files remaining ...
158 files remaining ...

Possible output after this patch on a slower machine:

 * checking 6158 files for package collisions
 48% done, 3145 files remaining ...
 96% done, 192 files remaining ...
100% done

Signed-off-by: Fabian Groffen 
---
 lib/portage/dbapi/vartree.py | 11 +--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
index 4b91caea8..244195fad 100644
--- a/lib/portage/dbapi/vartree.py
+++ b/lib/portage/dbapi/vartree.py
@@ -3475,13 +3475,18 @@ class dblink(object):
symlink_collisions = []
destroot = self.settings['ROOT']
totfiles = len(file_list) + len(symlink_list)
+   tnow = time.time()
+   tinterv = 2  # seconds
+   ninterv = tnow + tinterv
showMessage(_(" %s checking %d files for package 
collisions\n") % \
(colorize("GOOD", "*"), totfiles))
for i, (f, f_type) in enumerate(chain(
((f, "reg") for f in file_list),
((f, "sym") for f in symlink_list))):
-   if i % 1000 == 0 and i != 0:
-   showMessage(_("%d files remaining 
...\n") % (totfiles - i))
+   if time.time() > ninterv:
+   showMessage(_("%3d%% done, %d files 
remaining ...\n") %
+   (i * 100 / totfiles, 
totfiles - i))
+   ninterv = time.time() + tinterv
 
dest_path = normalize_path(
os.path.join(destroot, 
f.lstrip(os.path.sep)))
@@ -3570,6 +3575,8 @@ class dblink(object):
break
if stopmerge:
collisions.append(f)
+   if tnow + tinterv < ninterv:
+   showMessage(_("100% done\n"))
return collisions, dirs_ro, symlink_collisions, 
plib_collisions
 
def _lstat_inode_map(self, path_iter):
-- 
2.20.1




Re: [gentoo-portage-dev] [PATCH] collision_protect: use dynamic report interval

2019-01-08 Thread Fabian Groffen
On 08-01-2019 09:17:04 +0100, Ulrich Mueller wrote:
> >>>>> On Tue, 08 Jan 2019, Fabian Groffen wrote:
> 
> > Output before this patch:
> 
> >  * checking 6111 files for package collisions
> >  [...]
> 
> > After:
> 
> >  * checking 6158 files for package collisions
> >  [...]
> 
> Why does the total number of files change?

First was python2, second python3.  The result was indicative only, the
number doesn't change of course.

Sorry for the confusion.

Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-portage-dev] [PATCH] collision_protect: use dynamic report interval

2019-01-08 Thread Fabian Groffen
The reporting of files remaining can look somewhat odd since the report
interval is hardcoded to be per 1000 objects.  Adjust this interval to
be regular towards the end.  While at it, report percentage done.

Output before this patch:

 * checking 6111 files for package collisions
5111 files remaining ...
4111 files remaining ...
3111 files remaining ...
2111 files remaining ...
 files remaining ...
111 files remaining ...

After:

 * checking 6158 files for package collisions
 16% done, 5131 files remaining ...
 33% done, 4104 files remaining ...
 50% done, 3077 files remaining ...
 66% done, 2050 files remaining ...
 83% done, 1023 files remaining ...
100% done

Signed-off-by: Fabian Groffen 
---
 lib/portage/dbapi/vartree.py | 14 +++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
index 4b91caea8..78f2b37f2 100644
--- a/lib/portage/dbapi/vartree.py
+++ b/lib/portage/dbapi/vartree.py
@@ -3475,13 +3475,19 @@ class dblink(object):
symlink_collisions = []
destroot = self.settings['ROOT']
totfiles = len(file_list) + len(symlink_list)
+   bucksize = 1000
+   buckcnt = int(totfiles / bucksize)
+   if buckcnt == 0 or totfiles % bucksize > int(bucksize / 
2):
+   buckcnt = buckcnt + 1
+   bucksize = int(totfiles / buckcnt) + 1
showMessage(_(" %s checking %d files for package 
collisions\n") % \
-   (colorize("GOOD", "*"), totfiles))
+   (colorize("GOOD", "*"), totfiles))
for i, (f, f_type) in enumerate(chain(
((f, "reg") for f in file_list),
((f, "sym") for f in symlink_list))):
-   if i % 1000 == 0 and i != 0:
-   showMessage(_("%d files remaining 
...\n") % (totfiles - i))
+   if i % bucksize == 0 and i != 0:
+   showMessage(_("%3d%% done, %d files 
remaining ...\n") %
+   (i * 100 / totfiles, 
totfiles - i))
 
dest_path = normalize_path(
os.path.join(destroot, 
f.lstrip(os.path.sep)))
@@ -3570,6 +3576,8 @@ class dblink(object):
break
if stopmerge:
collisions.append(f)
+   if bucksize < totfiles:
+   showMessage(_("100% done\n"))
return collisions, dirs_ro, symlink_collisions, 
plib_collisions
 
def _lstat_inode_map(self, path_iter):
-- 
2.20.1




[gentoo-portage-dev] [PATCH] lib/portage/dbapi/vartree: use dynamic report interval in _collision_protect

2019-01-07 Thread Fabian Groffen
The reporting of files remaining can look somewhat odd since the report
interval is hardcoded to be per 1000 objects.  Adjust this interval to
be regular towards the end.  While at it, report percentage done.

Output before this patch:

 * checking 6111 files for package collisions
5111 files remaining ...
4111 files remaining ...
3111 files remaining ...
2111 files remaining ...
 files remaining ...
111 files remaining ...

After:

 * checking 6158 files for package collisions
16% done, 5131 files remaining ...
33% done, 4104 files remaining ...
50% done, 3077 files remaining ...
66% done, 2050 files remaining ...
83% done, 1023 files remaining ...

Signed-off-by: Fabian Groffen 
---
 lib/portage/dbapi/vartree.py | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/lib/portage/dbapi/vartree.py b/lib/portage/dbapi/vartree.py
index 4b91caea8..4d0bf2789 100644
--- a/lib/portage/dbapi/vartree.py
+++ b/lib/portage/dbapi/vartree.py
@@ -3475,13 +3475,19 @@ class dblink(object):
symlink_collisions = []
destroot = self.settings['ROOT']
totfiles = len(file_list) + len(symlink_list)
+   bucksize = 1000
+   buckcnt = int(totfiles / bucksize)
+   if buckcnt == 0 or totfiles % bucksize > int(bucksize / 
2):
+   buckcnt = buckcnt + 1
+   bucksize = int(totfiles / buckcnt) + 1
showMessage(_(" %s checking %d files for package 
collisions\n") % \
(colorize("GOOD", "*"), totfiles))
for i, (f, f_type) in enumerate(chain(
((f, "reg") for f in file_list),
((f, "sym") for f in symlink_list))):
-   if i % 1000 == 0 and i != 0:
-   showMessage(_("%d files remaining 
...\n") % (totfiles - i))
+   if i % bucksize == 0 and i != 0:
+   showMessage(_("%2d%% done, %d files 
remaining ...\n") %
+   (i * 100 / totfiles, 
totfiles - i))
 
dest_path = normalize_path(
os.path.join(destroot, 
f.lstrip(os.path.sep)))
-- 
2.20.1




Re: [gentoo-dev] [pre-GLEP] Gentoo binary package container format

2018-11-27 Thread Fabian Groffen
On 26-11-2018 21:13:53 +, Andrey Utkin wrote:
> On Wed, Nov 21, 2018 at 11:45:54AM +0100, Fabian Groffen wrote:
> > We agree it is hackish, and we agree we can do without.  You simply
> > exaggerate the problem, IMO, which mostly isn't there, because it works
> > fine today.  It can also be solved today using shell tools.
> 
> I am sad that you don't see it as a productivity impediment that the
> user is required to know the custom tooling to do even such a trivial
> non-standard action as manual extraction.

Huh?  tar -jxf doesn't do the trick for you?

> Maybe I will make myself look bad by admitting this, but I'm not meeting
> your expectations. I use Gentoo for ~11 years, and for about one year I
> am using my private binpkgs distributed to all my machines (i.e. I have
> read binary package guide fair number of times, but I stopped rereading
> it when I satisfied my needs). When in need, I still reached to trusty
> tar, and I did not even know what are the names of special tools (a
> toolchain?) qtbz2 and qxpak.
> 
> Just few days ago I messed with binpkgs for investigation purpose. I
> just wanted to extract few to somewhere (definitely not into system
> root), and read a core dump with GDB asking it to use those extracted
> files for debug symbols.
> 
> Of course I used `tar xaf`, because what I know is that it's honest tbz2
> just with metadata appended.
> 
>  # tar xaf boost-1.65.0.tbz2
> 
> bzip2: (stdin): trailing garbage after EOF ignored
> 
> Exit code is 0.
> But the notice is annoying (on subconscious level), because Silence Is
> Golden - "when a program has nothing interesting or surprising to say,
> it should shut up".

You seem to contradict yourself.  You didn't know the tools, yet you say
you needed to, to unpack the files.  But you show here you just unpacked
the files without said knowledge.

> > % head -c `grep -abo 'XPAKPACK' 
> > $EPREFIX/usr/portage/packages/sys-apps/sed-4.5.tbz2 | sed 's/:.*$//'` 
> > $EPREFIX/usr/portage/packages/sys-apps/sed-4.5.tbz2 | tar -jxf -
> > 
> > results in no warnings/errors from bzip about trailing garbage, possible
> > thanks to the spec being smart enough about this.
> 
> Thanks, this is a very concise **custom tool** to handle current binpkg
> format.

As is tar followed by tar.  The obvious advantage of the latter is that
you don't get a warning which could trigger you into thinking something
is wrong.  So, in my opinion, that is a better way of doing it compared
to the current way.

> > Not having to do this, when under stress and pressure to restore a
> > system to get it back into production, is a plus.  Though, in that
> > scenario the trailing garbage warning wouldn't have been that bad
> > either.
> 
> When understress and pressure, the irrelevant warning is not bad?
> I am sure it is really bad for operator's attention.

I've been using Gentoo binpkgs for a long while, I think something like
~14 years ago when I used them extensively.  Perhaps I'm an exception,
but back then I knew already there was an extra bit attached to the
tars, as were all my collegues around me back then.  The fact it comes
up now (as a surprise?) maybe means the knowledge has gone.  So good
thing we're replacing it with something easier to infer from inspecting
it.

Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [pre-GLEP r2] Gentoo binary package container format

2018-11-21 Thread Fabian Groffen
On 20-11-2018 21:33:17 +0100, Michał Górny wrote:
> The volume label
> 
> 
> The volume label provides an easy way for users to identify the binary
> package without dedicated tooling or specific format knowledge.
> 
> The implementations should include a volume label consisting of fixed
> string ``gpkg:``, followed by a single space, followed by full package
> identifier.  However, the implementations must not rely on the volume
> label being present or attempt to parse its value when it is.
> 
> Furthermore, since the volume label is included in the .tar archive
> as the first member, it provides a magic string at a fixed location
> that can be used by tools such as file(1) to easily distinguish Gentoo
> binary packages from regular .tar archives.

Just for clarity on this point.
Are you proposing that we patch file(1) to print the Volume Header here?
file-5.35 seems to not say much but "tar archive" or "POSIX tar archive"
for tar-files containing a Volume Header as shown by tar -tv.

> Container and archive formats
> -
> 
> During the debate, the actual archive formats to use were considered.
> The .tar format seemed an obvious choice for the image archive since
> it is the only widely deployed archive format that stores all kinds
> of file metadata on POSIX systems.  However, multiple options for
> the outer format has been debated.

You mention POSIX, which triggered me.  I think it would be good to
specify which tar format to use.

POSIX.1-2001/pax format doesn't have a 100/256 char filename length
restriction, which is good but it is not (yet) used by default by GNU
tar.  busybox tar can read pax tars, it seems.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [pre-GLEP] Gentoo binary package container format

2018-11-21 Thread Fabian Groffen
reason.  And before you ask really
> > > silly questions, yes, I did fight binary packages over hex editor
> > > at some point.
> > 
> > Which I still don't understand, to be frank.  I think even Portage
> > exposes python APIs to get to the data.
> 
> Compare the time needed to make a trivial (but unforeseen) change
> on a format that's transparent vs a format that requires you to learn
> its spec and/or API, write a program and debug it.

I was under the impression you could unpack a tbz2 into data and xpak,
then unpack both, modify the contents with an editor or whatever, and
then pack the whole stuff back into a tbz2 again.  This can be done
worst case scenario by emerge -k , modifying the vdb and quickpkg
 afterwards.
I know that with portage-utils you can do this easily with the qtbz2 and
qxpak commands.  No need to do anything with a hex editor, or know
anything about how it's done.
Obvious advantage of your approach is that you don't need q* tools, but
can use tar instead.  The editting is as trivial though.  In your case
you need a special procedure to reconstruct the binpkg should you want
to keep your special properties (label, order) which equates to q* tools
somewhat.

> > > The most trivial case is an attempted recovery of a broken system.
> > > If you don't have Portage working and don't have portage-utils
> > > installed, do you really prefer a custom format which will require you
> > > to fetch and compile special tools?  Or is one that can be processed
> > > with tools you're quite likely to have on every system, like tar?
> > 
> > Well, I think the idea behind the original binpkg format was to use tar
> > directly on the files in emergency scenarios like these...
> > The assumption was bzip2 decompressor and tar being available.
> > I think it is an example of how you add something, while still allowing
> > to fallback on existing tools.
> 
> Except progress in compressors has made it work less and less reliably. 
> It's mostly an example how to be *clever*.  However, being clever
> usually doesn't pay off in the long term, compared to doing things *in a
> simple way*.

We agree it is hackish, and we agree we can do without.  You simply
exaggerate the problem, IMO, which mostly isn't there, because it works
fine today.  It can also be solved today using shell tools.

% head -c `grep -abo 'XPAKPACK' 
$EPREFIX/usr/portage/packages/sys-apps/sed-4.5.tbz2 | sed 's/:.*$//'` 
$EPREFIX/usr/portage/packages/sys-apps/sed-4.5.tbz2 | tar -jxf -

results in no warnings/errors from bzip about trailing garbage, possible
thanks to the spec being smart enough about this.

Not having to do this, when under stress and pressure to restore a
system to get it back into production, is a plus.  Though, in that
scenario the trailing garbage warning wouldn't have been that bad
either.

> > > > > 3. **The file format should provide for partial fetching of binary
> > > > >packages.**  It should be possible to easily fetch and read
> > > > >the package metadata without having to download the whole package.
> > > > 
> > > > Like above, what is the use-case here?  Why would you want this?  I
> > > > think I'm missing something here.
> > > 
> > > Does this harm anything?  Even if there's little real use for this, is
> > > there any harm in supporting it?  Are we supposed to do things the other
> > > way around with no benefit just because you don't see any real use for
> > > it?
> > 
> > Well, you make a huge point out of it.  And if it isn't used, then why
> > bother so much about it.  Then it just looks like you want to use it as
> > an argument to get rid of something you just don't like.
> > 
> > In my opinion you better just say "hey I would like to implement this
> > binpkg format, because I think it would be easier to support with
> > minimal tools since it doesn't have custom features".  I would have
> > nothing against that.  Simple and elegant is nice, you don't need to
> > invent arguments for that, in my opinion.
> 
> The spec is now more focused on that.

Thank you, much appreciated.

Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [pre-GLEP] Gentoo binary package container format

2018-11-18 Thread Fabian Groffen
On 18-11-2018 10:38:51 +0100, Michał Górny wrote:
> On Sun, 2018-11-18 at 10:16 +0100, Fabian Groffen wrote:
> > On 17-11-2018 12:21:40 +0100, Michał Górny wrote:
> > > Problems with the current binary package format
> > > ---
> > > 
> > > The following problems were identified with the package format currently
> > > in use:
> > > 
> > > 1. **The packages rely on custom binary archive format to store
> > >metadata.**  It is entirely Gentoo invented, and requires dedicated
> > >tooling to work with it.  In fact, the reference implementation
> > >in Portage does not even include a CLI tool to work with tbz2
> > >packages; an unofficial implementation is provided as part
> > >of portage-utils toolkit [#PORTAGE-UTILS]_.
> > 
> > I think you should rewrite this section to the argument that the
> > metadata is hard to edit, and that there is only one tool to do so
> > (except a python interface from Portage?).
> > On a separate note, I don't think portage-utils can be considered
> > "unofficial", it is a Gentoo official project as far as I am aware.
> 
> In this context, Portage is 'official'.  Portage-utils is a project
> that's developed entirely separately from Portage and doesn't use
> Portage APIs but instead reinvents everything.  As such, it is easy for
> the two to go out of sync.  Or for one of them to have bugs that
> the other one doesn't have (say, with endianness).

I'm not sure if it's actually true, I was under the impression the same
author(s) worked on the Portage as well as portage-utils code.  Anyway,
aren't quickpkg and emerge enough from a user's perspective?

> > > 2. **The format relies on obscure compressor feature of ignoring
> > >trailing garbage**.  While this behavior is traditionally implemented
> > >by many compressors, the original reasons for it have become long
> > >irrelevant and it is not surprising that new compressors do not
> > >support it.  In particular, Portage already hit this problem twice:
> > >once when users replaced bzip2 with parallel-capable pbzip2
> > >implementation [#PBZIP2]_, and the second time when support for zstd
> > >compressor was added [#ZSTD]_.
> > 
> > I think this is actually the result of a rather opportunistic
> > implementation.  The fault is that we chose to use an extension that
> > suggests the file is a regular compressed tarball.
> > When one detects that a file is xpak padded, it is trivial to feed the
> > decompressor just the relevant part of the datastream.  The format
> > itself isn't bad, and doesn't rely on obscure behaviour.
> 
> Except if you don't have the proper tools installed.  In which case
> the 'opportunistic' behavior made it possible to extract the contents
> without special tools... except when it actually happens not to work
> anymore.  Roy's reply indicates that there is actually interest in this
> design feature.

Your point is that the format is broken (== relies on obscure compressor
feature).  My point is that the format simply requires a special tool.
The fact that we prefer to use existing tools doesn't imply in any way
that the format is broken to me.
I think you should rewrite your point to mention that you don't want to
use a tool that doesn't exist in @system (?) to unpack a binpkg.  My
guess is that you could use some head/tail magic in a script if the
trailing block is upsetting the decompressor.

I'm not saying this may look ugly, I'm just saying that your point seems
biased.

> > > 3. **Placing metadata at the end of file makes partial fetches
> > >complex.**  While it is technically possible to obtain package
> > >metadata remotely without fetching the whole package, it usually
> > >requires e.g. 2-3 HTTP requests with rather complex driver.  For
> > >comparison, if metadata was placed at the beginning of the file,
> > >early-terminated pipeline with a single fetch request would suffice.
> > 
> > I think this point needs to be quantified somewhat why it is so
> > important.
> > I may be wrong, but the average binpkg is small, <1MiB, bigger packages
> > are <50MiB.
> > So what is the gain to be saved here?  A "few" MiBs for what operation
> > exactly?  I say "few" because I know for some users this is actually not
> > just a blib before it's downloaded.  So if this is possible to achieve,
> > in what scenarios is this going to be used (and is this often?).
> 
> Last I checked, Gentoo aimed to support more users than the 'majority'
> of peopl

Re: [gentoo-dev] [pre-GLEP] Gentoo binary package container format

2018-11-18 Thread Fabian Groffen
 metadata updates.**
>In particular, it should be possible to update the metadata without
>having to recompress package files.
> 
> 6. **The file format should account for easy recognition both through
>filename and through contents.**  Preferably, it should have distinct
>features making it possible to detect it via file(1).
> 
> 7. **The file format should allow for metadata compression.**
> 
> 8. **The file format should make future extensions easily possible
>without breaking backwards compatibility.**

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [RFC] gpkg format proposal v2 (was: Re: [gentoo-portage-dev] [RFC] Improving Gentoo package format)

2018-11-12 Thread Fabian Groffen
On 11-11-2018 21:53:33 +0100, Michał Górny wrote:
> Hi,
> 
> Ok, here's the second version integrating the feedback received.
> The format is much simpler, based on nested tarballs inspired by Debian.
> 
> The outer tarball is uncompressed and uses '.gpkg.tar' suffix.  It
> contains (preferably in order but PM should also handle packages with
> mismatched order):
> 
> 1. Optional (but recommended) "gpkg: ${PF}" package label that can be
> used to quickly distinguish Gentoo binpkgs from regular tarballs
> (for file(1)).
> 
> 2. "metadata.tar${comp}" tarball containing binary package metadata
> as files.
> 
> 3. Optional "metadata.tar${comp}.sig" containing detached signature
> for the metadata archive.
> 
> 4. "contents.tar${comp}" tarball containing files to be installed.
> 
> 5. Optional "contents.tar${comp}.sig" containing detached signature for
> the contents archive.
> 
> Notes:
> 
> a. ${comp} can be any compression format supported by binary packages. 
> Technically, metadata and content archives may use different
> compression.  Either or both may be uncompressed as well.

I'm wondering here, how much sense does it make to compress 2., 3.
and/or 4. if you compress the whole gpkg?  I have the impression
compression on compression isn't beneficial here.  Shouldn't just
compressing of the gpkg tar be sufficient?

As to allowing different compressors for a single gpkg, I think it would
be better to require all compressors to be the same, such that a PM or
tool can quickly see if it can "read" the file from the gpkg filename,
instead of having to fetch and open it first.  Obviously, if you drop
compression of the inner tars, this point goes away.

Thanks,
Fabian

> b. While signatures are optional, the PM should have a switch
> controlling whether to expect them, and fail hard if they're not present
> when expected.
> 
> 
> Advantages
> --
> Guaranteed:
> 
> + The binary package is still one file, so can be fetched easily.
> 
> + File format is trivial and can be extracted using tar(1) + compressor.
> 
> + The metadata and contents are compressed independently, and so can be
> easily extracted or modified independently.
> 
> + The package format provides for separate metadata and content
> signatures, so they can be verified independently.
> 
> + Metadata can be compressed now.
> 
> Achieved by regular archives (but might be broken if modified by user):
> 
> + Easy recognition by magic(1).
> 
> + The metadata archive (and its signature) is packed first, so it may be
> read without fetching the whole binpkg.
> 
> 
> Why not .ar format?
> ---
> The use of .ar format has been proposed, akin to Debian.  While
> the option is mostly feasible, and the simplicity of .ar format would
> reduce the outer size of binary packages, I think the format is simply
> too obscure.  It lives mostly as static library format, and the tooling
> for it is part of binutils.  LSB considers it deprecated.  While I don't
> see it going away anytime soon, I'd rather not rely on it in order to
> save a few KiB.
> 
> 
> Is there anything left to address?
> 
> -- 
> Best regards,
> Michał Górny



-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [PATCH] app-emulation/libvirt-snmp: version bump to 0.0.4

2018-10-27 Thread Fabian Groffen
On 27-10-2018 15:18:28 +0300, Joonas Niilola wrote:
> Why can't you use Github like _everyone_ else? It's really simple and fast.

Hrm ... given Github's not integrated with Gentoo, one can argue that
sending a patch to the ML is actually more simple/fast for developers.

I don't think sending patches to this ML targets the right audience, yet
I welcome the contribution.

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Changing policy about -Werror

2018-09-14 Thread Fabian Groffen
I think you misunderstood what I wrote, or I wasn't clear enough.
Richard summed up my intention nicely in his response.

Fabian

On 15-09-2018 00:46:24 +0300, Alon Bar-Lev wrote:
> On Sat, Sep 15, 2018 at 12:29 AM Fabian Groffen  wrote:
> >
> > On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote:
> > > >
> > > > Perhaps, if one persists on going this route, only do this for platforms
> > > > that upstream supports, such that arches which will suffer from this
> > > > (typically ppc, sparc, ...) don't have to be blocked by this.
> > >
> > > Exactly in these cases the -Werror is useful as if upstream expects no
> > > warnings then any warning should block installation and trigger bug
> > > report. In Gentoo in many cases we use packages on platform has no
> > > access to, our feedback to upstream is valuable. A great example is
> > > gnutls in which we collectively (maintainer, unstable users,
> > > architecture teams, stable users) found issues on architectures that
> > > almost nobody other than Gentoo has access to.
> > >
> >
> > I don't believe Gentoo users are (supposed to be) an extension of
> > upstreams.
> 
> This is exactly what I think that is special about Gentoo, and the
> reason I use Gentoo. Unlike other distribution Gentoo is the closest
> thing of using upstream. A maintainer in Gentoo who is not see himself
> part of the upstream packages he maintains has far less impact than a
> maintainer who does see himself as part of upstream or is upstream.
> 
> Per your statement, we should not allow any architecture or setup that
> upstream, such as exact versioning, architecture or toolchain.
> 
> > If upstreams insist on that, they should make their software
> > non-free, adding a non-modification clause or something.  In any case,
> > it is not Gentoo's job IMHO.
> 
> Then we cannot re-distribute or patch, how is it related to the
> discussion? We are talking about open source projects and I know it is
> cliche... the "greater good" and helping the "free open source
> movement" a a viable alternative. I thought this is what unite us
> here.
> 
> > In the end it is Gentoo who needs to care
> > for its users.  I prefer we do that by giving them an option to become
> > that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo
> > disables by default.
> 
> Do you think someone do not care about the users? Do you actually
> think upstream does not care about users? I do not understand this
> statement. If downstream maintainer believes that upstream is friendly
> for the Gentoo overhead (which is higher than binary distributions) or
> create the relationship in which Gentoo is 1st citizen at upstream,
> why do you think users cannot use vanilla upstream?
> 
> > As maintainer and/or enthusiastic user, like you wrote for gnutls, I
> > would be more than happy to provide build logs/errors for all the arches
> > I have access to.  So like I wrote before, I think we should consider
> > case-by-case basis to make it easy to do so.
> 
> This entire discussion is to allow case-by-case and not black and
> white approach recently enforced.
> 
> Regards,
> Alon
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Changing policy about -Werror

2018-09-14 Thread Fabian Groffen
On 15-09-2018 00:07:12 +0300, Alon Bar-Lev wrote:
> >
> > Perhaps, if one persists on going this route, only do this for platforms
> > that upstream supports, such that arches which will suffer from this
> > (typically ppc, sparc, ...) don't have to be blocked by this.
> 
> Exactly in these cases the -Werror is useful as if upstream expects no
> warnings then any warning should block installation and trigger bug
> report. In Gentoo in many cases we use packages on platform has no
> access to, our feedback to upstream is valuable. A great example is
> gnutls in which we collectively (maintainer, unstable users,
> architecture teams, stable users) found issues on architectures that
> almost nobody other than Gentoo has access to.
>

I don't believe Gentoo users are (supposed to be) an extension of
upstreams.  If upstreams insist on that, they should make their software
non-free, adding a non-modification clause or something.  In any case,
it is not Gentoo's job IMHO.  In the end it is Gentoo who needs to care
for its users.  I prefer we do that by giving them an option to become
that extension of upstream, e.g. by USE=upstream-cflags, which Gentoo
disables by default.

As maintainer and/or enthusiastic user, like you wrote for gnutls, I
would be more than happy to provide build logs/errors for all the arches
I have access to.  So like I wrote before, I think we should consider
case-by-case basis to make it easy to do so.

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Changing policy about -Werror

2018-09-14 Thread Fabian Groffen
On 14-09-2018 16:29:43 -0400, Rich Freeman wrote:
> On Fri, Sep 14, 2018 at 4:20 PM Michael Orlitzky  wrote:
> >
> > On 09/14/2018 03:58 PM, Richard Yao wrote:
> > >>
> > >> No one has answered the question: what do you do when a stable package
> > >> breaks because of a new warning?
> > >>
> > >> ...>
> > > Wouldn’t this be largely covered as part of GCC stabilization? We could 
> > > reserve the right to kill -Werror in a package where it blocks GCC 
> > > stabilization if the maintainer does not handle it in a timely manner.
> > >>
> >
> > They would be uncovered during GCC stabilization, but then you're right
> > back in the original situation: how do you fix the stable package? The
> > only answer that doesn't violate some other policy is to patch it in a
> > new revision and wait for it to stabilize again.
> >
> > Other questions arise: Do we block stabilization of clang et al.?
> >
> 
> Presumably we could make it a blocker, so then portage won't install
> the new stable toolchain.  That buys time and only affects users of
> that particular package.  But, as I pointed out before you can do that
> without using -Werror - just block installation with an unqualified
> toolchain.
> 
> You would only use an approach like this for packages where QA was
> fairly important, so the inconvenience would be worth it.

Perhaps, if one persists on going this route, only do this for platforms
that upstream supports, such that arches which will suffer from this
(typically ppc, sparc, ...) don't have to be blocked by this.

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Changing policy about -Werror

2018-09-13 Thread Fabian Groffen
On 13-09-2018 18:56:13 +0300, Alon Bar-Lev wrote:
> On Thu, Sep 13, 2018 at 6:51 PM Fabian Groffen  wrote:
> >
> > On 12-09-2018 17:46:03 -0700, Matt Turner wrote:
> > > On Wed, Sep 12, 2018 at 5:11 PM Rich Freeman  wrote:
> > > With new GCC comes new warnings, and harmless as the vast majority are
> > > they cause the build to break with Werror.
> >
> > To illustrate harmless:
> >   warning: this statement may fall through [-Wimplicit-fallthrough=]
> > The warning message already has it in it that it's just a pure guess.
> 
> One that exposed a lot of unintentional fallthoughs which were fixed
> when reporting to upstream.

Sure that's why the warning is there.  But you ignore the point that the
same code compiled fine and ran fine for years without problems.

> Once again... we should discuss to leave -Werror when policy of
> upstream to have no warnings and is maintaining that policy properly
> while we at downstream may cooperate and avoid patching upstream but
> discuss issues when found.

On a developer's system, that would be nice.

For ordinary users on the other hand:
Leaving -Werror is leaving our users alone in the dark.  Don't do that.

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Changing policy about -Werror

2018-09-13 Thread Fabian Groffen
On 12-09-2018 20:09:54 -0400, Rich Freeman wrote:
> On Wed, Sep 12, 2018 at 7:32 PM Chí-Thanh Christopher Nguyễn
>  wrote:
> >
> > Alon Bar-Lev schrieb:
> > > We
> > > are unique as permutations and architectures that are used by Gentoo
> > > users are so diverse that we find issues that nobody else finds.
> >
> > This needs to be highlighted more, as it is why suggestions that the
> > maintainer can simply put -Werror back on their own system are insufficient.
> >
> 
> Wouldn't running into new runtime issues be exactly the sort of reason
> why we'd want to build with -Werror on packages where these issues are
> unacceptable?

Can you think of a way in which a new runtime issue would occur that
-Werror would have guarded?  And that issue would also get through
maintainer and ~arch testing?

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Changing policy about -Werror

2018-09-13 Thread Fabian Groffen
On 13-09-2018 07:36:09 -0400, Richard Yao wrote:
> 
> 
> > On Sep 12, 2018, at 6:55 PM, Thomas Deutschmann  wrote:
> > 
> >> On 2018-09-12 16:50, Rich Freeman wrote:
> >> There is also the case where we want these warnings to block
> >> installation, because the risk of there being a problem is too great.
> > 
> > I really disagree with that. So many devs have already said multiple
> > times in this thread that "-Werror" is only turning existing warnings
> > into fatal errors but "-Werror" itself doesn't add any new checks and
> > more often requires "-O3" to be useful.
> The way that compilers work is that the warnings are generated in the front 
> end while the optimization level affects the backend. That means that -O3 has 
> no effect on the code that does error generation. This remark about -O3 being 
> needed to make -Werror useful is just plain wrong. 

Huh?  -O3 enables more checks, which can generate more warnings.  -O3
isn't "needed", but if upstream is so interested in clean and correct
code, they should've fixed all warnings in the first place and thus
enabled all of them.  In fact, I expect every sane upstream to use "-O3
-Wall -Werror" in one of their automated builds.  Not that this catches
anything useful on x86{,_64} when there is for instance use of signed
and unsigned char types, so it isn't conclusive.

The whole point in here is that -Werror doesn't add much if you care.
The whole point why it is not desired in Gentoo is that users don't
necessarily are developers, or even interested in fixing warnings --
regardless whether they point to real problems or not.

If there are real problems in a package (exposed by a compiler or not)
then this should ideally stand out during ~arch testing, or even before
when the Gentoo maintainer examines the build (might even use -Werror
for his own purposes).  If such code ends up in stable arch we just made
a stabilisation mistake, or got royally messed up by upstream, depending
how you look at it.

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Changing policy about -Werror

2018-09-13 Thread Fabian Groffen
On 12-09-2018 17:46:03 -0700, Matt Turner wrote:
> On Wed, Sep 12, 2018 at 5:11 PM Rich Freeman  wrote:
> >
> > On Wed, Sep 12, 2018 at 7:52 PM Matt Turner  wrote:
> > >
> > > On Wed, Sep 12, 2018 at 4:03 PM Rich Freeman  wrote:
> > > > Now, I could buy that -Werror turns NEW warnings into fatal errors,
> > > > due to the use of a newer toolchain, since upstream probably didn't
> > > > test with that toolchain and thus wouldn't have seen the warning.
> > >
> > > Yes, exactly. This is one of the major things people have said repeatedly.
> > >
> > > Werror makes moving gcc forward much more difficult.
> > >
> >
> > Sure, but wouldn't a block on newer versions of gcc cause similar headaches?
> 
> Sure...? Who is suggesting that? I'm not sure where you're going with this.
> 
> With new GCC comes new warnings, and harmless as the vast majority are
> they cause the build to break with Werror.

To illustrate harmless:
  warning: this statement may fall through [-Wimplicit-fallthrough=]
The warning message already has it in it that it's just a pure guess.


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Changing policy about -Werror

2018-09-13 Thread Fabian Groffen
On 13-09-2018 00:55:45 +0200, Thomas Deutschmann wrote:
> Unless you can do that we don't really need to discuss this. Simply
> because everyone interested in "-Werror" *can* set that flag via CFLAGS,
> even just per package, whereas the majority, not interested in this,
> cannot do the same to filter "-Werror". Nobody advocating for "-Werror"
> replied to that fact yet.

Unfortunately adding -Werror to CFLAGS is not just possible.  Many
configure checks rely on the compiler just generating something,
ignoring warnings for various reasons.  If you add -Werror to your
CFLAGS you basically introduce breakage for some autoconf-based
packages.

What we really should be having is an easy way for post-configure CFLAGS
addition.  Just to support devs/users who insist on this for their
setups.

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Changing policy about -Werror

2018-09-10 Thread Fabian Groffen
On 09-09-2018 11:22:41 -0400, Richard Yao wrote:
> -Werror has caught bugs that could have resulted in data loss in ZFS in the 
> past thanks to it being built in userspace as part of zdb. So it is useful 
> for integrity too, not just security (although arguably, integrity is part of 
> security).

This is a misconception, as jer already pointed out.  Instead:

-Werror has forced you to take notice of problems that could have
resulted in data loss in ZFS ...

Also, consider that for -Werror to be "better", you also need -O3 in
order to activate the "proper" compiler checks like "variable set but
never used" ones.

> Perhaps we could have another USE flag for -Werror where it is a security 
> feature. e.g. USE=strict-compile-checks

You better run a static code analyser, such as the one you can hook up
with Travis.  It usually points out real security problems such as
races, which GCC doesn't do yet, as far as I'm aware.  Let alone
trigger with -Werror.

Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [PATCH] multilib: allow specifying the subtype of library in get_libname

2018-08-03 Thread Fabian Groffen
Just wondering, we have get_libname, get_modname, shouldn't we just
introduce get_linklibname or something instead of trying to overload
get_libname?

The implementation of get_linklibname could be to just call get_libname
for anything but mingw of course.

Fabian

On 03-08-2018 00:09:41 -0500, Marty E. Plummer wrote:
> Signed-off-by: Marty E. Plummer 
> ---
> 
> On mingw-w64 (and perhaps cygwin and mingw.org), there are two forms of
> non-static libraries. Standard *.dll libraries are for runtime and are
> loaded from %PATH% on windows systems, and are typically stored in
> either /bin or /usr/bin on mingw-w64 cross-toolchain filesystems. Import
> libraries, *.dll.a, are used when linking and live in the ''normal''
> libdirs, eg, /lib, /usr/lib and so on.
> 
> A number of ebuilds which otherwise work on mingw-w64 crossdev
> toolchains exhibit failure due to usage of get_libname not being able to
> specify which of the two types are required.
> 
> For example, sys-libs/ncurses, uses the following snippet of code:
> ln -sf libncurses$(get_libname) 
> "${ED}"/usr/$(get_libdir)/libcurses$(get_libname) || die
> in order to create a 'libcurses.so -> libncurses.so' symlink.
> 
> However, on a crossdev-built mingw-w64 toolchain, one will end up with a
> broken 'libcurses.dll -> libncurses.dll' symlink, which should properly
> be a 'libcurses.dll.a -> libncurses.dll.a' symlink, as the symlink here
> is provided to allow linking with -lcurses instead of -lncurses.
> 
>  eclass/multilib.eclass | 52 ++
>  1 file changed, 48 insertions(+), 4 deletions(-)
> 
> diff --git a/eclass/multilib.eclass b/eclass/multilib.eclass
> index 350b6f949d1..6a99f5977ec 100644
> --- a/eclass/multilib.eclass
> +++ b/eclass/multilib.eclass
> @@ -239,26 +239,70 @@ get_exeext() {
>  }
>  
>  # @FUNCTION: get_libname
> -# @USAGE: [version]
> +# @USAGE: --link|--run [version]
>  # @DESCRIPTION:
>  # Returns libname with proper suffix {.so,.dylib,.dll,etc} and optionally
>  # supplied version for the current platform identified by CHOST.
>  #
> +# If '--link' argument is passed, the linktime library's suffix is returned,
> +# as in the file that must exist to let `gcc -lfoo foo.c -o foo` to work.
> +# If '--run' argument is passed, the runtime library's suffix is returned.
> +#
> +# In most unix-like platforms the two are identical, however on mingw-w64 the
> +# linktime library has the suffix of '.dll.a' and the runtime library '.dll'.
> +#
>  # Example:
>  # get_libname ${PV}
>  # Returns: .so.${PV} (ELF) || .${PV}.dylib (MACH) || ...
> +# get_libname --link
> +# Returns: .so (ELF) || .dylib (MACH) || .dll.a (PE32) || ...
>  get_libname() {
> - local libname
> - local ver=$1
> + local libtype="undefined"
> + local libname opt ver
> + for opt; do
> + case "${opt}" in
> + --link)
> + libtype="link"
> + shift
> + ;;
> + --run)
> + libtype="run"
> + shift
> + ;;
> + *)
> + ;;
> + esac
> + done
> + ver="$1"
> + # general unixy types
>   case ${CHOST} in
>   *-cygwin*)   libname="dll.a";; # import lib
> - mingw*|*-mingw*) libname="dll";;
>   *-darwin*)   libname="dylib";;
>   *-mint*) libname="irrelevant";;
>   hppa*-hpux*) libname="sl";;
>   *)   libname="so";;
>   esac
>  
> + # wierd mingw-w64 stuff, maybe even cygwin
> + case ${CHOST} in
> + mingw*|*-mingw*)
> + case ${libtype} in
> + link)
> + libname="dll.a" # import library
> + ;;
> + run)
> + libname="dll" # runtime library
> + ;;
> + undefined)
> + eerror "please specify either --link or 
> --run to get_libname"
> + eerror "for mingw builds, as there are 
> two types of libraries"
> + eerror "on this platform"
> + die
> + ;;
> + esac
> + ;;
> + esac
> +
>   if [[ -z $* ]] ; then
>   echo ".${libname}"
>   else
> -- 
> 2.18.0
> 
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] rfc: [QA] Ban policy introduction

2018-07-29 Thread Fabian Groffen
Completely agreeing with Sergei, with some additional suggestions:

On 28-07-2018 23:14:12 +0100, Sergei Trofimovich wrote:
> On Sun, 29 Jul 2018 00:40:18 +0300
> Mikle Kolyada  wrote:
> 
> > Hello,
> > 
> > The Gentoo QA team would like to introduce the following policy that
> > would be applied to individuals breaking the state and quality of the
> > main gentoo.git tree
> > 
> > ( as we do not have this strictly documented yet):
> > 
> > 
> > 
> > If recommended
> 
> It's not called "recommended" but "enforced".

I agree.  If you put penalties on these, they become hard rules.  I
think that change should be discussed by the council perhaps?

> > Gentoo workflow policies are not followed by an
> > individual developer
> > (e.g make major changes to the widely used eclasses without prior
> > discussion on the mailing list or
> > commit changes that lead to multiple CI checks failure)
> 
> Here should go exhaustive list of links to the policies to be enforced.

At least.  And they should be clear and concise.  No "common sense" or
anything involved for exceptions and the like.  In addition, new checks
should be introduced to the community and possibly approved by council
as to whether being enforced or not.

Fabian

> 
> > the standard QA
> > procedure is:
> > 
> > 1.) Two warnings granted by QA team, after two independent breakages
> > 2.) Revoking the commit access for 14 days
> > 
> > These violations will be evaluated individually by all QA team members.
> > Warnings can be revoked, if during 6 months period a developer makes at
> > least 20 non trivial changes not producing more breakages.
> > 
> > 
> 
> -- 
> 
>   Sergei
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH v2] rsync: quarantine data prior to verification (bug 660410)

2018-07-08 Thread Fabian Groffen
lg = False
>   is_synced = False
>   if timestamp != 0 and "--quiet" not in opts:
> @@ -686,6 +752,12 @@ class RsyncSync(NewBase):
>   elif (servertimestamp == 0) or (servertimestamp > 
> timestamp):
>   # actual sync
>   command = rsynccommand[:]
> +
> + if self.repo.location != download_dir:
> + # Use shared hardlinks for files that 
> are identical
> + # in the previous snapshot of the 
> repository.
> + command.append('--link-dest=%s' % 
> self.repo.location)
> +
>   submodule_paths = self._get_submodule_paths()
>   if submodule_paths:
>   # The only way to select multiple 
> directories to
> @@ -696,9 +768,10 @@ class RsyncSync(NewBase):
>   # /./ is special syntax 
> supported with the
>   # rsync --relative option.
>   command.append(syncuri + "/./" 
> + path)
> - command.append(self.repo.location)
>   else:
> - command.extend([syncuri + "/", 
> self.repo.location])
> + command.append(syncuri + "/")
> +
> + command.append(download_dir)
>  
>   exitcode = None
>   try:
> -- 
> 2.13.6
> 
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] [PATCH v2 09/11] glep-0063: Make recommended expiration terms mandatory

2018-07-06 Thread Fabian Groffen
On 06-07-2018 13:34:21 +0200, Ulrich Mueller wrote:
> - Make creation of a revocation certificate (and storing it in a place
>   separate from the key) mandatory.

What does this really achieve?  Or require?  Am I supposed to buy or
hire a vault now?  --  I'm assuming the word "safe" is missing from
above sentence.

Side observation:
You can't check I have the revocation cert, let alone that you can
check where it is stored, or if I lost it.

Unless it is stored in a Gentoo owned vault or something, such that
infra can invoke it on retirement scripts, this seems like unnecessary
bureaucracy.

Of course we want to encourage anyone to have a revocation cert, and to
store it in a safe place somewhere.  This is at best subject to means
and opportunities of the person in question.  In reality it is quite
hard to store secrets securely, even more when they don't fit well in
the human SSD.

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Developer commit timeline updates

2018-06-25 Thread Fabian Groffen
On 24-06-2018 00:36:28 +0200, Michał Górny wrote:
> I'd like to just make a short announcement that I've updated
> the developer commit timeline [1].

Very nice stats, thanks for that.

Suggestion: perhaps this can be cron-jobbed and updated every month or
so?

In any case, fun to look at. Thanks :)


> [1]:https://dev.gentoo.org/~mgorny/dev-timeline.html
> [2]:https://dev.gentoo.org/~mgorny/active-devs.html
-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-portage-dev] [PATCH 0/4] rsync: add key refresh retry (bug 649276)

2018-04-01 Thread Fabian Groffen
On 01-04-2018 20:29:59 +0200, Michał Górny wrote:
> > > This essentially looks like ~700 lines of code to try to workaround
> > > broken networking. I would rather try to do that using 5 lines of code
> > > but that's just me, and my programs aren't enterprise quality. I just
> > > hope it actually solves as many problems as it's going to introduce.
> > 
> > The vast majority of this code is generic and reusable, and I do intend
> > to reuse it. For example, the executor support will be an essential
> > piece for the asyncio.AbstractEventLoop for bug 649588.
> 
> Sure it is and sure you will. But tell me: who is going to maintain it
> all? Because as far as I can see, we're still dealing with a bus factor
> of one and all you're doing is making it worse. More code, more
> complexity, more creeping featurism and hacks.

Well, well, well.  This could be said about a lot of code, including
your own.

> Last time you went away, you left us with a horribly unmaintainable
> package manager full of complexity, hacks and creeping featurism
> and a Portage team whose members had barely any knowledge of the code.
> Just when things started moving again, you came back and we're back to
> square one.

I don't see why this has to be made personal.  In the olde days people
just pushed whatever they needed, now it's reviewed and acked, so how
can it be back to square one?

> Today Portage once again is a one-developer project, full of more
> complexity, more hacks and more creeping featurism. And we once again
> have a bus factor of one -- one developer who apparently knows
> everything, does everything and tries to be nice to everyone, except he
> really ignores others, makes a lot of empty promises and consistently
> makes the health of the project go from bad to worse.

Errr, didn't you recently start your own fork with creeping featurism of
its own because mainline was/is too slow?

> So, please tell me: what happens when you leave again? How have you used
> your time in the project? What have you done to make sure that
> the project stays alive without you? Because as far as I can see, adding
> few thousand of lines of practically unreviewed code every month does
> not help with that.

Why is this Zac's problem specifically?  Isn't this a general problem of
Gentoo?  And isn't your attitude contributing in a large factor to
the bus-factor getting down from 1 to 0?

> I forked Portage because I didn't want to fight with you. When I forked
> it, I declared that I will merge mainline changes regularly for
> the benefit of my users. But after a week, I really start feeling like
> that's going to be a really bad idea. Like it's time to forget about
> mainline Portage as a completely dead end, and go our separate ways.

So you fork.  Like many forks, that is because people feel that it isn't
going "their way".  Yay.  That doesn't make you "right".

> I'm seriously worried about the future of Gentoo. I'd really appreciate
> if you started focusing on that as well. I get that all this stuff looks
> cool on paper but few months or years from now, someone will curse
> 'whoever wrote that code' while trying to fix some nasty bug. Or get
> things moving forward. Or implement EAPI 8.

Perhaps it's time to look into the mirror.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] rfc: empty directories in ${D}

2018-03-29 Thread Fabian Groffen
On 29-03-2018 16:47:51 +0200, Michał Górny wrote:
> W dniu czw, 29.03.2018 o godzinie 09∶39 -0500, użytkownik William Hubbs
> napisał:
> > All,
> > 
> > I just happened to notice the following warning from portage when
> > bumping dhcpcd.
> > 
> > > One or more empty directories installed to /var:
> > >  /var/lib/dhcpcd
> > > If those directories need to be preserved, please make sure to create
> > > or mark them for keeping using 'keepdir'. Future versions of Portage
> > > will strip empty directories from installation image.
> > 
> > If we are going to require emptty  directories to be marked with
> > keepdir, I think we should hard fail the emerge rather than quietly
> > strip the empty directories. If we just strip the directories, this
> > will, more than likely, lead to broken packages. In the case of dhcpcd,
> > the upstream build system installs the /var/lib/dhcpcd directory, then
> > dhcpcd writes to the directory.
> > 
> 
> Are you saying that dozens of packages should suddenly start failing
> for users so that developers would feel more obliged to fix them?
> Provided that the packages are still maintained, and it won't be
> 'hey, we just made it impossible to install this package, maybe someone
> will fix it one day'.

I agree, packages shouldn't suddenly start failing.  Not during install,
not during runtime either.  For changes like this EAPIs were invented.

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] New category: app-metrics

2018-03-22 Thread Fabian Groffen
Hi!

On 20-03-2018 06:21:00 +0100, Manuel Rüger wrote:
> Hi everyone,
> 
> I'm planning to add a new category app-metrics, which is mainly (at
> least for my own use case) going to be used for prometheus[0] and its
> exporters providing endpoints for prometheus. It can be used for other
> packages whose _main_ purpose is to provide metrics, transform or
> consume them.
> 
> * net-analyzer/prometheus
> * app-admin/bind_exporter
> * app-admin/elasticsearch_exporter
> * app-admin/mongodb_exporter
> * app-admin/nginx-vts-exporter
> * app-admin/prom2json
> * dev-util/buildbot-prometheus

Does the graphite stack fit in this too, you think?

app-admin/diamond
app-admin/collectd
app-misc/carbon-c-relay
dev-python/carbon
net-analyzer/graphite-web
www-apps/grafana-bin

We might have more packages, but this is from the top off my head.

Thanks,
Fabian

> 
> In addition, the following packages will drop their prometheus- prefix
> during the package move:
> 
> * net-analyzer/prometheus-blackbox_exporter
> * net-analyzer/prometheus-node_exporter
> * net-analyzer/prometheus-redis_exporter
> * net-analyzer/prometheus-snmp_exporter
> * net-analyzer/prometheus-uwsgi_exporter
> * net-analyzer/prometheus-pushgateway
> * net-analyzer/prometheus-alertmanager
> 
> With the growing adoption of prometheus I expect more exporters to be
> added (I have five more that I want to add in the near future).
> 
> 
> Thanks,
> 
> Manuel
> 
> [0] https://prometheus.io
> 
> 




-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-dev] About making mastersync redundant

2018-03-20 Thread Fabian Groffen
Hi,

I know infra is working on fixing this, so they better focus on that for
now.  Thank you to infra for doing all the work!

When this is resolved, perhaps we should have a discussion on how to
make this service redundant?  Currently the prefix rsync generation is
redundant (== 2 generators) so I'm interested to discuss and see if that
would be feasible for gx86 too.

Fabian

On 19-03-2018 21:13:37 -0400, Alec Warner wrote:
> This is just an FYI: [1]https://infra-status.gentoo.org/
> 
> Hoping to have this fixed in a couple of days. In the meantime you may see
> missing snapshots (for emerge webrsync) and no rsync propagation.
> 
> -A
> 
> 
> 
>  References:
>1. https://infra-status.gentoo.org/
> 
> read_char: errno==EILSEQ; invalid byte sequence for UTF-8: 
-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] about non-ebuild files in the tree (and verification thereof)

2018-02-28 Thread Fabian Groffen
On 28-02-2018 22:08:54 +, Robin H. Johnson wrote:
> On Wed, Feb 28, 2018 at 04:10:52PM +0100, Fabian Groffen wrote:
> > Hi,
> > 
> > I'm working on a verification implementation of
> > https://www.gentoo.org/glep/glep-0074.html and ran into the following
> > scenario that I don't know if it's right or wrong:
> ...
> > Does anybody know or have a pointer to what the policies on files in our
> > ebuild dirs actually is?
> PMS, 4.3 Package directories:
> A package directory may contain other files or directories, whose
> purpose is not covered by this specification.

Ah, forwards compatibility.

> GLEP74 itself makes no determination of files being permitted in a given
> directory.
> 
> > Now in a rsync checkout of the Prefix tree, where my own implementation
> > also runs the fat manifest creation, this entry is not present, because
> > I always believed only metadata.xml, ChangeLog* and *.ebuild files were
> > allowed.
> I'd say your separate implementation is wrong in this case, but that
> file also should not permit at this time.

I might change it not to bother about what should be in/out, but just
assume it's right as-is.  For now it is a nice headsup about something
being unexpected.

> > Now I'm confused as to whether this is the case or not, I can't find a
> > GLEP or anything, but repoman also is as happy as it can be on this odd
> > file (I thought it used to complain about stray/unadded files).
> I personally think repoman should complain about it because it's weird.

I'm sure this particular file was a mistake, that went unnoticed for a
very long time.  I do feel this should one way or the other not be
allowed.

Thanks for your insights,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-dev] about non-ebuild files in the tree (and verification thereof)

2018-02-28 Thread Fabian Groffen
Hi,

I'm working on a verification implementation of
https://www.gentoo.org/glep/glep-0074.html and ran into the following
scenario that I don't know if it's right or wrong:

Consider net-misc/srf-ip-conn-srv
% ls
files  Manifest  metadata.xml  srf-ip-conn-srv-.ebuild
srf-ip-conn-srv.pid
% cat Manifest
DIST jsmn-35086597a72d.tar.gz 11056 
DIST srf-ip-conn-140c9b8a8619.tar.gz 112882 

Notice that there is a .pid file in the ebuild dir, checked in git,
which even contains what appears to be a pid.  It isn't used from the
ebuild as far as I can tell.  Apart from being odd, this is actually
irrelevant.

In an rsync checkout of the gentoo-x86 tree, I see in the Manifest for
this package a DATA entry for the .pid-file.  Hence, verification with
both gemato as well as my own implementation succeed because the
.pid-file is acknowledged.

Now in a rsync checkout of the Prefix tree, where my own implementation
also runs the fat manifest creation, this entry is not present, because
I always believed only metadata.xml, ChangeLog* and *.ebuild files were
allowed.

Now I'm confused as to whether this is the case or not, I can't find a
GLEP or anything, but repoman also is as happy as it can be on this odd
file (I thought it used to complain about stray/unadded files).

Does anybody know or have a pointer to what the policies on files in our
ebuild dirs actually is?

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-dev] about commits in the future

2018-02-21 Thread Fabian Groffen
Please consider commit 2fb923d758749be609c1daab2a72ad4f1ec4c2a9
(use something like
   git log --pretty=fuller app-emulation/nemu/nemu-1.4.0.ebuild)

It was made roughly one day in the future.  I'm wondering:
1) is this bad at all?
2) if it is bad (got me confused for another reason) is it technically
   possible to reject such commits?
3) if 2) should we decide on some clock skew and reject anything which
   is beyond that?

How do others feel about this?

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] RFC: Bugzilla arch list reordering

2018-01-23 Thread Fabian Groffen
On 23-01-2018 12:49:00 +0100, Dirkjan Ochtman wrote:
> On Tue, Jan 23, 2018 at 12:46 PM, Michał Górny <[1]mgo...@gentoo.org> wrote:
> 
> > Since neither of the proposals has received any specific reply, I'm not
> > sure how to proceed from here. I suppose we can possibly have two lists
> > in different order so that people could use whichever they prefer.
> 
> Not sure having two lists in Bugzilla would be an improvement. It would be 
> nice
> if more people expressed a preference, though!

I like the "popularity" order, but on top of that, I'd like to group
32/64 bits versions of arches.  Basically djc converted into:

AMD64
X86
PPC64
PPC
ARM64
ARM
SPARC64 (?)
SPARC
ALPHA
IA64
HPPA
MIPS
...

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-dev] Re: repo/gentoo:master commit in: sys-devel/llvm/

2018-01-20 Thread Fabian Groffen
On 19-01-2018 22:21:55 +, Michał Górny wrote:
> URL:https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5bcc52f0
> 
> sys-devel/llvm: Fix implicit dependency on app-arch/libxar
> 
> Support conditionally using app-arch/libxar in LLVM 6+, and explicitly

libxar

> @@ -45,6 +45,7 @@ RDEPEND="
>   libedit? ( dev-libs/libedit:0=[${MULTILIB_USEDEP}] )
>   libffi? ( >=virtual/libffi-3.0.13-r1:0=[${MULTILIB_USEDEP}] )
>   ncurses? ( >=sys-libs/ncurses-5.9-r3:0=[${MULTILIB_USEDEP}] )
> + xar? ( app-arch/xar )

xar here

> --- a/sys-devel/llvm/metadata.xml
> +++ b/sys-devel/llvm/metadata.xml
> @@ -20,5 +20,7 @@
>   Support querying terminal properties using 
> ncurses' terminfo
>   Build compiler-rt's sanitizers
>   Install the Clang static analyzer 
> (requires USE=clang)
> + Support dumping LLVM bitcode sections in 
> Mach-O files
> + (uses app-arch/libxar)

I think you mean app-arch/xar everywhere?

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


[gentoo-dev] about stable, dev and exp profile status

2018-01-10 Thread Fabian Groffen
On 07-01-2018 21:25:28 +0100, Michał Górny wrote:
> I'd like to follow this with a more precise proposal. Namely, redefine
> the current profile statuses to apply the following:
> 
> a. stable -> fully tested, all depgraph breakages are errors,
> 
> b. exp -> fully tested, all depgraph breakages are warnings,
> 
> c. dev -> developer's playground, not tested.

I always was under the impression the following order (and explanation)
was the case:

stable -> development -> experimental

For this reason, e.g. Prefix profiles are (still) experimental, which
means they really shouldn't bother non-Prefix people.  Prefix users and
developers work on an environment where those profiles are promoted to
development ones, such that repoman kicks in for their work.  (At least
that was the idea.)

I see you (re-)define dev as "developer's playground", and I wonder if
in that case it wouldn't be better to introduce a new one instead?

Maybe I'm just one of a few who thinks the order is reversed now.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Upcoming posting restrictions on the gentoo-dev mailing list

2018-01-10 Thread Fabian Groffen
On 09-01-2018 22:20:56 +0100, Andreas K. Huettel wrote:
> If, as a non-developer, you want to participate in a discussion on 
> gentoo-dev, 
> - either reply directly to the author of a list mail and ask him/her to 
> forward your message,
> - or ask any Gentoo developer of your choice to get you whitelisted.
>
> If, as a developer, you want to have someone whitelisted, please comment on
> bug 644070 [3]. Similar to Bugzilla editbugs permission, if you are vouching
> for a contributor you are expected to keep an eye on their activity.

For the record, I do not like this decision.  Not just because it puts a
burden on developers to become comrel agents.  A technical solution like
this, doesn't solve the actual problem.  Unfortunately this solution
destroys much more than it intends to fix, which is a loss for Gentoo.

Since this turns -dev into -core-light, I suggest to anyone who doesn't
like this direction that we move discussions to -project instead.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: PGP signature


Re: [gentoo-dev] Re: repo/gentoo:master commit in: dev-libs/libunibreak/

2017-12-14 Thread Fabian Groffen
On 14-12-2017 14:41:17 +0100, Michał Górny wrote:
> W dniu czw, 14.12.2017 o godzinie 13∶56 +0100, użytkownik Fabian Groffen
> napisał:
> > On 14-12-2017 13:39:18 +0100, Michał Górny wrote:
> > > Dnia 14 grudnia 2017 13:21:47 CET, Fabian Groffen <grob...@gentoo.org> 
> > > napisał(a):
> > > > Can we make it a policy to list /what/ QA issues are the justification
> > > > for commits like these?  A description in the commit message would be
> > > > preferred, but a pointer to a location where said issues can be found
> > > > would do too.
> > > 
> > > Maintainer-needed is reason enough. If somebody couldn't be bothered to 
> > > maintain what he committed, why should we bother to list the issues?
> > > 
> > > Using repoman and looking at CI mails is also a good idea.
> > 
> > Obviously for me to learn something, I won't/can't use repoman here.  So
> > a pointer to said CI mails from the message of the QA commit would be
> > nice.
> 
> Last I checked, I wasn't personally responsible for teaching people
> ebuild writing 101 while on phone. But here you go (in malformed paste
> of ebuild below).

You simply replied, and therefore took ownership from QA point of view.
I can't help it you do that whilst on the phone.  In fact, this is
email, so being on the phone is not a good reason to be vague and avoid
answering questions in the first place.

[snip issues]

Thanks, much appreciated.  I'm completely convinced now.  I'm referring
back to my earlier suggestion to include such list or the type of issues
found when a drastic commit like the one we discuss is done under the QA
flag.  It's good to know that the QA issue complaint was valid, and
improvements can be made.

A final suggestion is to talk to the committer before taking such
drastic actions, in a situation where our users aren't endangered.  As
this one is, in my opinion.
There is more problematic stuff in the tree, but teach a man to fish ...
next time the problems may be avoided.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] Re: repo/gentoo:master commit in: dev-libs/libunibreak/

2017-12-14 Thread Fabian Groffen
On 14-12-2017 14:43:11 +0100, Michał Górny wrote:
> Bull-shit.
> 
> Breaking the dependency tree was a *honest* mistake on the person who
> reverted this commit.

Honestly, so making mistakes is evaluated based on honesty, then still,
did Andreas (not a QA member as far as I can see) contact Andrey upfront
to establish how honest his mistake was?

> Andrey pretty clearly stated that he did this *on purpose*.

Andreas also did his commit *on purpose*.  Honestly.  And he made things
worse, now actually *affecting* our users.  ...

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] Re: repo/gentoo:master commit in: dev-libs/libunibreak/

2017-12-14 Thread Fabian Groffen
On 14-12-2017 14:34:51 +0100, Michał Górny wrote:
> W dniu czw, 14.12.2017 o godzinie 20∶24 +0700, użytkownik
> gro...@gentoo.org napisał:
> > If the developers of liblinebreak had not decided to rename their library, 
> > I could safely bump it from 2.1 to 4.0, in spite of the fact that it is 
> > maintainer-needed, right?
> > Am I personally responsible for their decision to use the new name 
> > libunibreak?
> 
> No, it wouldn't be fine. We might not even have noticed it if you
> at least bothered to update the Manifest. Please stop looking for
> excuses and loopholes, and start doing something good for Gentoo.

So, breaking the tree, just because someone forgot to set the
maintainer field is doing something good for Gentoo?  (That's called QA?)
https://archives.gentoo.org/gentoo-commits/message/7f34111f0e45f451d5b3a5a58b057d51

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] Re: repo/gentoo:master commit in: dev-libs/libunibreak/

2017-12-14 Thread Fabian Groffen
On 14-12-2017 13:39:18 +0100, Michał Górny wrote:
> Dnia 14 grudnia 2017 13:21:47 CET, Fabian Groffen <grob...@gentoo.org> 
> napisał(a):
> >Can we make it a policy to list /what/ QA issues are the justification
> >for commits like these?  A description in the commit message would be
> >preferred, but a pointer to a location where said issues can be found
> >would do too.
> 
> Maintainer-needed is reason enough. If somebody couldn't be bothered to 
> maintain what he committed, why should we bother to list the issues?

It seems to me you are avoiding the question.  There are no issues with
the ebuild.  It seems like there is just a false claim there are QA
issues, and that is used as waiver to remove the package.

> Using repoman and looking at CI mails is also a good idea.

repoman full (stable) is happy on 8b4ea0f6d2bed140116f69855d1d3100ea0cf020.
qa-reports.gentoo.org has nothing to report
gentoo-qa@ ML has nothing to report

Please list the QA issues:

> >On 14-12-2017 12:10:59 +, Andreas Hüttel wrote:
> >> Also other QA issues.

Apart from that maintainer-needed has nothing to do with Quality of an
ebuild, you mentioned it as an QA issue, so I am interested in the
"other" QA issues, which seems to suggest 2+ problems in this *ebuild*.

For the record, I didn't commit this ebuild.  I'm just extremely unhappy
about the tiggerhippy response of QA which in my opinion is totally
uncalled for, and am extremely worried about the integrity of QA because
of seemingly false claims to justify actions.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] Re: repo/gentoo:master commit in: dev-libs/libunibreak/

2017-12-14 Thread Fabian Groffen
On 14-12-2017 13:39:18 +0100, Michał Górny wrote:
> Dnia 14 grudnia 2017 13:21:47 CET, Fabian Groffen <grob...@gentoo.org> 
> napisał(a):
> >Can we make it a policy to list /what/ QA issues are the justification
> >for commits like these?  A description in the commit message would be
> >preferred, but a pointer to a location where said issues can be found
> >would do too.
> 
> Maintainer-needed is reason enough. If somebody couldn't be bothered to 
> maintain what he committed, why should we bother to list the issues?
> 
> Using repoman and looking at CI mails is also a good idea.

Obviously for me to learn something, I won't/can't use repoman here.  So
a pointer to said CI mails from the message of the QA commit would be
nice.

Thanks,
Fabian

> >
> >Thanks,
> >Fabian
> >
> >On 14-12-2017 12:10:59 +, Andreas Hüttel wrote:
> >> URL:   
> >https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=34e2c43f
> >> 
> >> Revert "dev-libs/libunibreak: initial import"
> >> 
> >> Please write on the chalkboard 100 times: "I will not add ebuilds as
> >maintainer-needed".
> >> Also other QA issues.
> >> This reverts commit 8b4ea0f6d2bed140116f69855d1d3100ea0cf020.
> >> 
> >>  dev-libs/libunibreak/Manifest   |  1 -
> >>  dev-libs/libunibreak/libunibreak-4.0.ebuild | 39
> >-
> >>  dev-libs/libunibreak/metadata.xml   | 14 ---
> >>  3 files changed, 54 deletions(-)
> >> 
> >> diff --git a/dev-libs/libunibreak/Manifest
> >b/dev-libs/libunibreak/Manifest
> >> deleted file mode 100644
> >> index 487fd898f5d..000
> >> --- a/dev-libs/libunibreak/Manifest
> >> +++ /dev/null
> >> @@ -1 +0,0 @@
> >> -DIST libunibreak-4.0.tar.gz 629403 SHA256
> >f7329bef1eb169d3363f040cefcc323cfd0a0bc53290a35a395e1d1adc849539 SHA512
> >43da73f66fabd8fdef444c5a06ad1800464a0aeab590938522d6c19973950a242f2ccc0575a93d10d87bdcf82610452117ac081ddb73f47271a8c2a65897e11c
> >WHIRLPOOL
> >ad71bc5910ca3dff994651022a5a6c6093cd4023852270fa243848f7576287b7cec4ad02a6b27686149491cb52824a93fdb6a6dac4c878d67db2f4041282d300
> >> 
> >> diff --git a/dev-libs/libunibreak/libunibreak-4.0.ebuild
> >b/dev-libs/libunibreak/libunibreak-4.0.ebuild
> >> deleted file mode 100644
> >> index f531bb66e38..000
> >> --- a/dev-libs/libunibreak/libunibreak-4.0.ebuild
> >> +++ /dev/null
> >> @@ -1,39 +0,0 @@
> >> -# Copyright 1999-2017 Gentoo Foundation
> >> -# Distributed under the terms of the GNU General Public License v2
> >> -
> >> -EAPI=6
> >> -inherit versionator
> >> -
> >> -DESCRIPTION="Line and word breaking library"
> >> -HOMEPAGE="http://vimgadgets.sourceforge.net/libunibreak/;
> >>
> >-SRC_URI="https://github.com/adah1972/${PN}/releases/download/${PN}_$(replace_all_version_separators
> >'_')/${P}.tar.gz"
> >> -
> >> -LICENSE="ZLIB"
> >> -SLOT="0"
> >> -KEYWORDS="~amd64 ~arm ~ppc ~x86"
> >> -IUSE="doc +man static-libs"
> >> -
> >> -DEPEND="man? ( app-doc/doxygen )"
> >> -RDEPEND="!dev-libs/liblinebreak"
> >> -
> >> -src_prepare() {
> >> -  use man && echo -e 'GENERATE_MAN=YES\nGENERATE_HTML=NO' >> Doxyfile
> >> -  default
> >> -}
> >> -
> >> -src_configure() {
> >> -  econf \
> >> -  $(use_enable static-libs static)
> >> -  use doc && export HTML_DOCS=( doc/html/. )
> >> -}
> >> -
> >> -src_compile() {
> >> -  default
> >> -  use man && doxygen || die
> >> -}
> >> -
> >> -src_install() {
> >> -  default
> >> -  prune_libtool_files
> >> -  use man && find "${D}/usr/include" -type f -execdir doman
> >"${S}/doc/man/man3/{}.3" \;
> >> -}
> >> 
> >> diff --git a/dev-libs/libunibreak/metadata.xml
> >b/dev-libs/libunibreak/metadata.xml
> >> deleted file mode 100644
> >> index a8bbb441f29..000
> >> --- a/dev-libs/libunibreak/metadata.xml
> >> +++ /dev/null
> >> @@ -1,14 +0,0 @@
> >> -
> >> - >"http://www.gentoo.org/dtd/metadata.dtd;>
> >> -
> >> -  
> >> -  
> >> -  Libunibreak is an implementation of the line breaking and word
> >breaking algorithms
> >> -  as described in Unicode Standard Annex 14 and Unicode Standard
> >Annex 29. It is
> >> -  designed to be used in a generic text renderer.
> >> -  
> >> -  
> >> -  Generate man pages with doxygen.
> >> -  Install html API documentation.
> >> -  
> >> -
> >> 
> >> 
> 
> 
> -- 
> Best regards,
> Michał Górny (by phone)
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


[gentoo-dev] Re: repo/gentoo:master commit in: dev-libs/libunibreak/

2017-12-14 Thread Fabian Groffen
Can we make it a policy to list /what/ QA issues are the justification
for commits like these?  A description in the commit message would be
preferred, but a pointer to a location where said issues can be found
would do too.

Thanks,
Fabian

On 14-12-2017 12:10:59 +, Andreas Hüttel wrote:
> URL:https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=34e2c43f
> 
> Revert "dev-libs/libunibreak: initial import"
> 
> Please write on the chalkboard 100 times: "I will not add ebuilds as 
> maintainer-needed".
> Also other QA issues.
> This reverts commit 8b4ea0f6d2bed140116f69855d1d3100ea0cf020.
> 
>  dev-libs/libunibreak/Manifest   |  1 -
>  dev-libs/libunibreak/libunibreak-4.0.ebuild | 39 
> -
>  dev-libs/libunibreak/metadata.xml   | 14 ---
>  3 files changed, 54 deletions(-)
> 
> diff --git a/dev-libs/libunibreak/Manifest b/dev-libs/libunibreak/Manifest
> deleted file mode 100644
> index 487fd898f5d..000
> --- a/dev-libs/libunibreak/Manifest
> +++ /dev/null
> @@ -1 +0,0 @@
> -DIST libunibreak-4.0.tar.gz 629403 SHA256 
> f7329bef1eb169d3363f040cefcc323cfd0a0bc53290a35a395e1d1adc849539 SHA512 
> 43da73f66fabd8fdef444c5a06ad1800464a0aeab590938522d6c19973950a242f2ccc0575a93d10d87bdcf82610452117ac081ddb73f47271a8c2a65897e11c
>  WHIRLPOOL 
> ad71bc5910ca3dff994651022a5a6c6093cd4023852270fa243848f7576287b7cec4ad02a6b27686149491cb52824a93fdb6a6dac4c878d67db2f4041282d300
> 
> diff --git a/dev-libs/libunibreak/libunibreak-4.0.ebuild 
> b/dev-libs/libunibreak/libunibreak-4.0.ebuild
> deleted file mode 100644
> index f531bb66e38..000
> --- a/dev-libs/libunibreak/libunibreak-4.0.ebuild
> +++ /dev/null
> @@ -1,39 +0,0 @@
> -# Copyright 1999-2017 Gentoo Foundation
> -# Distributed under the terms of the GNU General Public License v2
> -
> -EAPI=6
> -inherit versionator
> -
> -DESCRIPTION="Line and word breaking library"
> -HOMEPAGE="http://vimgadgets.sourceforge.net/libunibreak/;
> -SRC_URI="https://github.com/adah1972/${PN}/releases/download/${PN}_$(replace_all_version_separators
>  '_')/${P}.tar.gz"
> -
> -LICENSE="ZLIB"
> -SLOT="0"
> -KEYWORDS="~amd64 ~arm ~ppc ~x86"
> -IUSE="doc +man static-libs"
> -
> -DEPEND="man? ( app-doc/doxygen )"
> -RDEPEND="!dev-libs/liblinebreak"
> -
> -src_prepare() {
> - use man && echo -e 'GENERATE_MAN=YES\nGENERATE_HTML=NO' >> Doxyfile
> - default
> -}
> -
> -src_configure() {
> - econf \
> - $(use_enable static-libs static)
> - use doc && export HTML_DOCS=( doc/html/. )
> -}
> -
> -src_compile() {
> - default
> - use man && doxygen || die
> -}
> -
> -src_install() {
> - default
> - prune_libtool_files
> - use man && find "${D}/usr/include" -type f -execdir doman 
> "${S}/doc/man/man3/{}.3" \;
> -}
> 
> diff --git a/dev-libs/libunibreak/metadata.xml 
> b/dev-libs/libunibreak/metadata.xml
> deleted file mode 100644
> index a8bbb441f29..000
> --- a/dev-libs/libunibreak/metadata.xml
> +++ /dev/null
> @@ -1,14 +0,0 @@
> -
> -http://www.gentoo.org/dtd/metadata.dtd;>
> -
> - 
> - 
> - Libunibreak is an implementation of the line breaking and word 
> breaking algorithms
> - as described in Unicode Standard Annex 14 and Unicode Standard 
> Annex 29. It is
> - designed to be used in a generic text renderer.
> - 
> - 
> - Generate man pages with doxygen.
> - Install html API documentation.
> - 
> -
> 
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] [RFC] GLEP 74 post-Council review update [v5]

2017-12-01 Thread Fabian Groffen
ag (rather than ``DATA``),
> 
> - all ``.ebuild`` files need to use the ``EBUILD`` tag,
> 
> - the ``metadata.xml`` and ``ChangeLog`` files need to use
>   the ``MISC`` tag,
> 
> - the Manifest can be signed to provide authenticity verification,
> 
> - an uncompressed Manifest must always exist, and a compressed Manifest
>   of identical content may be present.
> 
> Once the backwards compatibility is no longer a concern, the above
> no longer needs to hold and the deprecated tags can be removed.
> 
> 
> Reference Implementation
> 
> 
> The reference implementation for this GLEP is being developed
> as the gemato project [#GEMATO]_.
> 
> 
> Credits
> ===
> 
> Thanks to all the people whose contributions were invaluable
> to the creation of this GLEP. This includes but is not limited to:
> 
> - Robin Hugh Johnson,
> - Ulrich Müller.
> 
> Additionally, thanks to Robin Hugh Johnson for the original
> MetaManifest GLEP series which served both as inspiration and source
> of many concepts used in this GLEP. Recursively, also thanks to all
> the people who contributed to the original GLEPs.
> 
> 
> References
> ==
> 
> .. [#GLEP44] GLEP 44: Manifest2 format
>(https://www.gentoo.org/glep/glep-0044.html)
> 
> .. [#GLEP57] GLEP 57: Security of distribution of Gentoo software
>- Overview
>(https://www.gentoo.org/glep/glep-0057.html)
> 
> .. [#GLEP58] GLEP 58: Security of distribution of Gentoo software
>- Infrastructure to User distribution - MetaManifest
>(https://www.gentoo.org/glep/glep-0058.html)
> 
> .. [#GLEP59] GLEP 59: Manifest2 hash policies and security implications
>(https://www.gentoo.org/glep/glep-0059.html)
> 
> .. [#GLEP60] GLEP 60: Manifest2 filetypes
>(https://www.gentoo.org/glep/glep-0060.html)
> 
> .. [#GLEP61] GLEP 61: Manifest2 compression
>(https://www.gentoo.org/glep/glep-0061.html)
> 
> .. [#UNICODE] The Unicode standard
>(https://unicode.org/versions/latest/)
> 
> .. [#PMS-FETCH] Package Manager Specification: Dependency Specification
>Format - SRC_URI
>(https://projects.gentoo.org/pms/6/pms.html#x1-940008.2.10)
> 
> .. [#FILE-NAMING-RULES] Ebuild File Format -- Gentoo Development Guide
>
> (https://devmanual.gentoo.org/ebuild-writing/file-format/#file-naming-rules)
> 
> .. [#MD5] RFC1321: The MD5 Message-Digest Algorithm
>(https://www.ietf.org/rfc/rfc1321.txt)
> 
> .. [#RIPEMD160] The hash function RIPEMD-160
>(https://homes.esat.kuleuven.be/~bosselae/ripemd160.html)
> 
> .. [#SHS] FIPS PUB 180-4: Secure Hash Standard (SHS)
>(http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.180-4.pdf)
> 
> .. [#WHIRLPOOL] The WHIRLPOOL Hash Function
>(http://www.larc.usp.br/~pbarreto/WhirlpoolPage.html)
> 
> .. [#BLAKE2] BLAKE2 -- fast secure hashing
>(https://blake2.net/)
> 
> .. [#SHA3] FIPS PUB 202: SHA-3 Standard: Permutation-Based Hash
>and Extendable-Output Functions
>(http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf)
> 
> .. [#STREEBOG] GOST R 34.11-2012: Streebog Hash Function
>(https://www.streebog.net/)
> 
> .. [#C08] Cappos, J et al. (2008). "Attacks on Package Managers"
>
> (https://www2.cs.arizona.edu/stork/packagemanagersecurity/attacks-on-package-managers.html)
> 
> .. [#DIST] According to Robin H. Johnson, 8.4% of all DIST entries
>at the time of writing are duplicate, representing 2 MiB
>out of 25 MiB of DIST entries altogether.
> 
> .. [#GEMATO] gemato: Gentoo Manifest Tool
>(https://github.com/mgorny/gemato/)
> 
> 
> Copyright
> =
> This work is licensed under the Creative Commons Attribution-ShareAlike 3.0
> Unported License. To view a copy of this license, visit
> http://creativecommons.org/licenses/by-sa/3.0/.
> 
> -- 
> Best regards,
> Michał Górny
> 
> 

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


[gentoo-dev] Re: [RFC, PATCH] user.eclass: gracefully return when unprivileged

2017-11-22 Thread Fabian Groffen
On 21-11-2017 20:19:21 +0900, Benda Xu wrote:
> Francesco Riosa <viv...@gmail.com> writes:
> 
> > maybe ewarn() is more appropriate than einfo()?
> > Just in case it's executed outside the scope of prefix
> 
> I can't remember any use case when portage (or, paludis, etc.) is
> executed as a normal user but not a from Prefix.
>
> This message will only affect Prefix users, who won't be surprised that
> they cannot create new groups or users.  Therefore I think einfo is more
> appropriate.
> 
> 
> Furthermore, we do have Prefix that runs as a root, mostly usable on NAS
> or smartphones, when we do ultimately like portage to manage groups and
> users.

I think we could definitely live with this until someone requests
otherwise.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


[gentoo-dev] Re: Prefix bootstrap script maintainability (Was: No more stable keywords for Games)

2017-11-22 Thread Fabian Groffen
On 20-11-2017 11:53:32 -0600, R0b0t1 wrote:
> Hello friends!
> 
> On Monday, November 20, 2017, Sergei Trofimovich <[1]sly...@gentoo.org> wrote:
> > On Sun, 19 Nov 2017 22:47:35 -0600
> > R0b0t1 <[2]r03...@gmail.com> wrote:
> >
> >> Understanding an existing codebase should not be a technical
> >> challenge. I had to resort to reimplementing all of the steps myself,
> >> in part to understand if they were done properly in the first place.
> >
> > Looks like you are an expert in this area now and are perfectly capable
> > of submitting the patches. I can review them at least for crossdev.
> >
> 
> In my goal to understand bootstrap-rap I am still in the process of creating
> something crossdev-like that can be used to generate a toolchain.
> 
> A recurring problem I have had is that this set of related tasks - generating
> cross compilers and packages, generating an initramfs, or generating a 
> prefixed
> pseudoinstallation - all start by reimplementing some subset of portage.
> 
> For prefix/RAP it makes sense, for the others possibly not.

You may also want to understand that cross-compiling (or compilers) in
itself is a very difficult topic to get right.  Mixing that with Prefix
FAICT never got out of the lab-setting in which it was attempted.

> >> Unfortunately these are things that the original authors should
> >> produce. Experience has shown me that documentation written by
> >> ancillary contributors that do not have deep experience with the code
> >> base is poor.

Like Benda said, documentation can always be improved.

In the case for bootstrap-prefix, it used to be documented in terms of
steps and why one had to do them that way.  Somewhere at the start of
2006, when there was like 150 packages, and one arch (ppc-macos), said
script didn't exist.  As it stands today, the key decisions and
workarounds are actually documented, but as RAP actually shows, if you
only focus on a specific use-case, you can get rid of a lot of (what
appears to be) nonsense.  It's the context in which you look at the
projects you refer to.

> Yes, that is what I am doing with my own code as I have the time to
> write it. I mostly still have no idea what is going on in the already
> written code.

Perhaps open up the dicussion on the related project's mailing lists.
At least I haven't come across any request to explain certain
bits/decisions yet.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] "Not a git repository" error on ebuild

2017-11-07 Thread Fabian Groffen
On 07-11-2017 21:33:41 +0200, Alan McKinnon wrote:
> On 07/11/2017 21:10, Damo Brisbane wrote:
> > Hi,
> > 
> > I am getting an error on custom fabio-1.5.2.ebuild
> > <https://github.com/damobrisbane/damo-overlay/blob/master/net-proxy/fabio/fabio-1.5.2.ebuild>
> >  -
> > "Not a git repository (or any of the parent directories)..." I've been
> > looking on web, trying some things, but still message is coming up,
> > appreciate suggestions. Note package is installing fine, just not sure
> > how to clean up this message.
> > 
> >  Thanks in advance:
> 
> 
> 
> 
> Please post the ebuild. That's a git error (not a portage error) and it
> means you are trying to run "git " without running "git
> init" first

Chances are git is called from the Makefile.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-portage-dev] [PATCH v2] __dyn_install: improve reporting of build and image sizes

2017-08-28 Thread Fabian Groffen
On 27-08-2017 12:53:23 -0700, Zac Medico wrote:
> On 08/27/2017 08:06 AM, Fabian Groffen wrote:
> > Prior to this commit, the reported sizes would look like:
> > 
> >  * Final size of build directory: 34942 KiB
> >  * Final size of installed tree: 5627 KiB
> > 
> > Because the sizes aren't aligned, it is hard to (visually) compare them.
> > On top of this, because the numbers are sometimes bigger, print a human
> > friendly size after the KiB size if applicable, like so:
> > 
> >  * Final size of build directory: 1906 KiB (1.8 MiB)
> >  * Final size of installed tree: 7 KiB
> > 
> > It should be noted that in case both sizes have a human-readable
> > variant, they are also aligned.
> > 
> > The helper functions are defined and used in a subshell to avoid
> > pollution of the caller's environment.
> > ---
> >  bin/phase-functions.sh | 53 
> > ++++++----
> >  1 file changed, 49 insertions(+), 4 deletions(-)
> 
> Looks good. Please merge.

Pushed, thanks!

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


[gentoo-portage-dev] [PATCH v2] __dyn_install: improve reporting of build and image sizes

2017-08-27 Thread Fabian Groffen
Prior to this commit, the reported sizes would look like:

 * Final size of build directory: 34942 KiB
 * Final size of installed tree: 5627 KiB

Because the sizes aren't aligned, it is hard to (visually) compare them.
On top of this, because the numbers are sometimes bigger, print a human
friendly size after the KiB size if applicable, like so:

 * Final size of build directory: 1906 KiB (1.8 MiB)
 * Final size of installed tree: 7 KiB

It should be noted that in case both sizes have a human-readable
variant, they are also aligned.

The helper functions are defined and used in a subshell to avoid
pollution of the caller's environment.
---
 bin/phase-functions.sh | 53 ++
 1 file changed, 49 insertions(+), 4 deletions(-)

diff --git a/bin/phase-functions.sh b/bin/phase-functions.sh
index dfd8733c8..ce174ba91 100644
--- a/bin/phase-functions.sh
+++ b/bin/phase-functions.sh
@@ -598,10 +598,55 @@ __dyn_install() {
 
# record build & installed size in build log
if type -P du &>/dev/null; then
-   local sz=( $(du -ks "${WORKDIR}") )
-   einfo "Final size of build directory: ${sz[0]} KiB"
-   sz=( $(du -ks "${D}") )
-   einfo "Final size of installed tree: ${sz[0]} KiB"
+   local nsz=( $(du -ks "${WORKDIR}") )
+   local isz=( $(du -ks "${D}") )
+
+   # subshell to avoid polluting the caller env with the helper
+   # functions below
+   (
+   # align $1 to the right to the width of the widest of 
$1 and $2
+   padl() {
+   local s1=$1
+   local s2=$2
+   local width=${#s1}
+   [[ ${#s2} -gt ${width} ]] && width=${#s2}
+   printf "%*s" ${width} "${s1}"
+   }
+
+   # transform number in KiB into MiB, GiB or TiB based on 
size
+   human() {
+   local s1=$1
+   local units=( KiB MiB GiB TiB )
+
+   s1=$((s1 * 10))
+   while [[ ${s1} -gt 10240 && ${#units[@]} -gt 1 
]] ; do
+   s1=$((s1 / 1024 ))
+   units=( ${units[@]:1} )
+   done
+
+   local r=${s1: -1}
+   s1=$((s1 / 10))
+   printf "%s.%s %s" "${s1}" "${r}" "${units[0]}"
+   }
+
+   size() {
+   local s1=$1
+   local s2=$2
+   local out="$(padl "${s1}" "${s2}") KiB"
+
+   if [[ ${s1} -gt 1024 ]] ; then
+   s1=$(human ${s1})
+   if [[ ${s2} -gt 1024 ]] ; then
+   s2=$(human ${s2})
+   s1=$(padl ${s1} ${s2})
+   fi
+   out+=" (${s1})"
+   fi
+   echo "${out}"
+   }
+   einfo "Final size of build directory: $(size ${nsz[0]} 
${isz[0]})"
+   einfo "Final size of installed tree:  $(size ${isz[0]} 
${nsz[0]})"
+   )
__vecho
fi
 
-- 
2.14.1




Re: [gentoo-portage-dev] [PATCH] _collision_protect: report progress in work todo

2017-08-27 Thread Fabian Groffen
On 26-08-2017 14:46:30 -0700, Zac Medico wrote:
> On 08/24/2017 06:28 AM, Fabian Groffen wrote:
> > Currently Portage reports its progress in checking collisions forward
> > every 1000th file like so:
> > 
> >  * checking 4149 files for package collisions
> > 1000 files checked ...
> > 2000 files checked ...
> > 3000 files checked ...
> > 4000 files checked ...
> >>>> Merging sys-apps/portage-2.3.8 to /
> > 
> > Change it to countdown style so it is easier to anticipate what the
> > next action will be:
> > 
> >  * checking 4149 files for package collisions
> > 3149 files remaining ...
> > 2149 files remaining ...
> > 1149 files remaining ...
> > 149 files remaining ...
> >>>> Merging sys-apps/portage-2.3.8 to /
> > ---
> >  pym/portage/dbapi/vartree.py | 5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
> > index 7c8f150bb..04a40b732 100644
> > --- a/pym/portage/dbapi/vartree.py
> > +++ b/pym/portage/dbapi/vartree.py
> > @@ -3420,13 +3420,14 @@ class dblink(object):
> > dirs_ro = set()
> > symlink_collisions = []
> > destroot = self.settings['ROOT']
> > +   totfiles = len(file_list) + len(symlink_list)
> > showMessage(_(" %s checking %d files for package 
> > collisions\n") % \
> > -   (colorize("GOOD", "*"), len(file_list) + 
> > len(symlink_list)))
> > +   (colorize("GOOD", "*"), totfiles))
> > for i, (f, f_type) in enumerate(chain(
> > ((f, "reg") for f in file_list),
> > ((f, "sym") for f in symlink_list))):
> > if i % 1000 == 0 and i != 0:
> > -   showMessage(_("%d files checked ...\n") 
> > % i)
> > +   showMessage(_("%d files remaining 
> > ...\n") % (totfiles - i))
> >  
> > dest_path = normalize_path(
> > os.path.join(destroot, 
> > f.lstrip(os.path.sep)))
> > 
> 
> Looks good. Please merge.

Pushed, thanks

Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


[gentoo-portage-dev] [PATCH] _collision_protect: report progress in work todo

2017-08-24 Thread Fabian Groffen
Currently Portage reports its progress in checking collisions forward
every 1000th file like so:

 * checking 4149 files for package collisions
1000 files checked ...
2000 files checked ...
3000 files checked ...
4000 files checked ...
>>> Merging sys-apps/portage-2.3.8 to /

Change it to countdown style so it is easier to anticipate what the
next action will be:

 * checking 4149 files for package collisions
3149 files remaining ...
2149 files remaining ...
1149 files remaining ...
149 files remaining ...
>>> Merging sys-apps/portage-2.3.8 to /
---
 pym/portage/dbapi/vartree.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index 7c8f150bb..04a40b732 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -3420,13 +3420,14 @@ class dblink(object):
dirs_ro = set()
symlink_collisions = []
destroot = self.settings['ROOT']
+   totfiles = len(file_list) + len(symlink_list)
showMessage(_(" %s checking %d files for package 
collisions\n") % \
-   (colorize("GOOD", "*"), len(file_list) + 
len(symlink_list)))
+   (colorize("GOOD", "*"), totfiles))
for i, (f, f_type) in enumerate(chain(
((f, "reg") for f in file_list),
((f, "sym") for f in symlink_list))):
if i % 1000 == 0 and i != 0:
-   showMessage(_("%d files checked ...\n") 
% i)
+   showMessage(_("%d files remaining 
...\n") % (totfiles - i))
 
dest_path = normalize_path(
os.path.join(destroot, 
f.lstrip(os.path.sep)))
-- 
2.14.1




[gentoo-portage-dev] [PATCH] __dyn_install: improve reporting of build and image sizes

2017-08-24 Thread Fabian Groffen
Prior to this commit, the reported sizes would look like:

 * Final size of build directory: 34942 KiB
 * Final size of installed tree: 5627 KiB

Because the sizes aren't aligned, it is hard to (visually) compare them.
On top of this, because the numbers are sometimes bigger, print a human
friendly size after the KiB size if applicable, like so:

 * Final size of build directory: 1906 KiB (1.8 MiB)
 * Final size of installed tree: 7 KiB

It should be noted that in case both sizes have a human-readable
variant, they are also aligned.
---
 bin/phase-functions.sh | 49 +
 1 file changed, 45 insertions(+), 4 deletions(-)

diff --git a/bin/phase-functions.sh b/bin/phase-functions.sh
index dfd8733c8..af45a0d49 100644
--- a/bin/phase-functions.sh
+++ b/bin/phase-functions.sh
@@ -598,10 +598,51 @@ __dyn_install() {
 
# record build & installed size in build log
if type -P du &>/dev/null; then
-   local sz=( $(du -ks "${WORKDIR}") )
-   einfo "Final size of build directory: ${sz[0]} KiB"
-   sz=( $(du -ks "${D}") )
-   einfo "Final size of installed tree: ${sz[0]} KiB"
+   local nsz=( $(du -ks "${WORKDIR}") )
+   local isz=( $(du -ks "${D}") )
+
+   # align $1 to the right to the width of the widest of $1 and $2
+   padl() {
+   local s1=$1
+   local s2=$2
+   local width=${#s1}
+   [[ ${#s2} -gt ${width} ]] && width=${#s2}
+   printf "%*s" ${width} "${s1}"
+   }
+
+   # transform number in KiB into MiB, GiB or TiB based on size
+   human() {
+   local s1=$1
+   local units=( KiB MiB GiB TiB )
+
+   s1=$((s1 * 10))
+   while [[ ${s1} -gt 10240 && ${#units[@]} -gt 1 ]] ; do
+   s1=$((s1 / 1024 ))
+   units=( ${units[@]:1} )
+   done
+
+   local r=${s1: -1}
+   s1=$((s1 / 10))
+   printf "%s.%s %s" "${s1}" "${r}" "${units[0]}"
+   }
+
+   size() {
+   local s1=$1
+   local s2=$2
+   local out="$(padl "${s1}" "${s2}") KiB"
+
+   if [[ ${s1} -gt 1024 ]] ; then
+   s1=$(human ${s1})
+   if [[ ${s2} -gt 1024 ]] ; then
+   s2=$(human ${s2})
+   s1=$(padl ${s1} ${s2})
+   fi
+   out+=" (${s1})"
+   fi
+   echo "${out}"
+   }
+   einfo "Final size of build directory: $(size ${nsz[0]} 
${isz[0]})"
+   einfo "Final size of installed tree:  $(size ${isz[0]} 
${nsz[0]})"
__vecho
fi
 
-- 
2.14.1




Re: [gentoo-dev] New package neomutt

2017-08-10 Thread Fabian Groffen
On 10-08-2017 16:09:53 +0200, Michał Górny wrote:
> ...which probably makes sense if you treat is as a continuation of mail-
> client/mutt package. However, since we package it separately, using
> the same name is going to create more confusion than renaming it to
> match the package name.
> 
> If I install 'dev-foo/foobar', I usually expect to find the program name
> 'foobar', not just 'bar'.

I think we're clear now that the eselect idea wasn't the proper way
forward.  Nicholas and I are discussing off-list what the next approach
will be, given the suggestions you and others have done in this thread.

Thanks,
Fabian


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] New package neomutt

2017-08-10 Thread Fabian Groffen
On 10-08-2017 14:13:29 +0200, Marc Schiffbauer wrote:
> * Nicolas Bock schrieb am 10.08.17 um 11:35 Uhr:
> > It does of course. What's appropriate here depends on whether we 
> > think somebody might want to have both mutt and neomutt installed 
> > at the same time. If we don't allow this use case, we don't have 
> > to worry about eselect and the neomutt binary will be called 
> > 'mutt' (as it is called by upstream already). If we do allow this 
> > use case, being able to eselect makes sense because then the 
> > binary is still always called 'mutt'.
> 
> Why not just have mutt and/or neomutt for both packages? Whoever only 
> wants neomutt and run it with 'mutt' can "alias mutt=neomutt" and be 
> done.

Both packages install /usr/bin/mutt by upstream's default (because
neomutt is supposed to be a drop-in replacement of mutt).

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] New package neomutt

2017-08-10 Thread Fabian Groffen
On 10-08-2017 09:40:30 +0200, Michał Górny wrote:
> On czw, 2017-08-10 at 06:58 +0200, Nicolas Bock wrote:
> > On Mon, Jul 31, 2017 at 09:11:19AM +0200, Nicolas Bock wrote:
> > > Hi,
> > > 
> > > I would like to add neomutt to the tree. This new package is meant as 
> > > an alternative and not a replacement of the existing mutt package.
> > 
> > Thanks for all of the great suggestions and feedback!
> > 
> > This is round two. I have update the ebuild with all your 
> > suggestions. I have also added support for eselecting between mutt 
> > and neomutt. Before the eselect ebuild can land though, we need to 
> > rename the mutt binary so that the managed link can be called 
> > mutt.
> 
> What for? How many people are exactly in the dire need of having both
> installed simultaneously and switching between them? If you really can't
> learn to type the new command, add IUSE=symlink blocking original mutt
> and be done with it. Don't add more unowned files to /usr by another
> poorly written eselect module.

Be nice!  No need to be bitchy here (and in the rest of your review).
Nicolas is just trying.

Me, as maintainer of Mutt, thought it was a good idea, because it allows
people to easily have both installed at the same time, which in this
interesting time for both projects is not a weird thing to have.

If there is a policy/move to get rid of eselect, then sorry, I am not
aware of that.  I can live with a symlink USE-flag.  It doesn't seem
very elegant to me, but it would work for this scenario.


Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] New package neomutt

2017-07-31 Thread Fabian Groffen
On 31-07-2017 04:55:58 -0500, Matthew Thode wrote:
> On 17-07-31 09:11:19, Nicolas Bock wrote:
> > Hi,
> > 
> > I would like to add neomutt to the tree. This new package is meant 
> > as an alternative and not a replacement of the existing mutt 
> > package.
> > 
> > Thanks,
> > 
> > Nick
> > 
> > -- 
> > Nicolas Bock <nicolasb...@gentoo.org>
> 
> It was my understanding that neomutt was mainly mutt with a bunch of
> patches added on, from what I can see, those patches are already handled
> by use flags in the mutt package itself.
>
> https://www.neomutt.org/about.html describes itself as a large set of
> feature patches and not a fork as well.  Are there missing patches that
> need to be added to the mutt package?

These days NeoMutt really is a fork, with a complete code-re-indent,
function name changes, etc.[1]  They move fast, deviating from Mutt and
no longer submit patches to Mutt.  It remains to be seen where both
projects end up, IMO.  It is no longer feasible to add features from
NeoMutt to Mutt, and Mutt moves along its own path (with
features/improvements) as well.

For now it seems useful to me to have both mutt and neomutt around.  I
sent my detailed comments on the neomutt ebuild to Nicholas off-list
already.  The changes suggested should show even more how the two are
different.

Thanks,
Fabian

[1] 
http://mailman.neomutt.org/pipermail/neomutt-devel-neomutt.org/2017-April/000364.html

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


[gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: eclass/

2017-05-16 Thread Fabian Groffen
On 16-05-2017 21:20:01 +0200, Michał Górny wrote:
> This eclass is used by almost 6700 ebuilds. I am trying *hard* to commit
> changes in large patchsets to reduce cache updates.
[snip]
> Normally, I would revert it but I don't feel like causing third huge
> cache update today.

What is the actual problem with that?  Hardware or more deltas to
download by users?  Just wondering.

Thanks,
Fabian

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] Proposal: profiles/arches.desc - improve repoman flexibility (with other benefits)

2017-03-27 Thread Fabian Groffen
On 27-03-2017 09:56:50 +0200, Ulrich Mueller wrote:
> >>>>> On Mon, 27 Mar 2017, Fabian Groffen wrote:
> 
> >> > When you say "arch" you actually mean a keyword as per GLEP-53[1]
> >> > right?
> >> 
> >> Which doesn't agree with actual usage in the tree, though.
> 
> > That surprises me.  Do you have an example of that?
> 
> The GLEP says about the OS suffix:
> 
> "The right hand part indicates the operating system or distribution,
> such as linux, macos, solaris or fbsd. If the right hand part is
> omitted, it implies the operating system/distribution type is
> GNU/Linux."
> 
> So if I understand this correctly, x86-linux should be equivalent to
> x86. But in reality, the linux suffix denotes that it is a prefix
> arch. I'm not saying that this is bad, only it's not what the GLEP
> says.

I see.  The lack of explicit mentioning what the difference means allows
for different interpretations.  I always *assumed* it meant Gentoo (1
part) vs Gentoo/Alt (2 parts).

> Until recently there was also x64-freebsd vs amd64-fbsd, where both
> the arch and the OS part denoted the same, but used different tokens
> to distinguish between prefix and non-prefix. (And I don't understand
> why amd64 is called x64 on prefix. A different OS suffix should be
> sufficient.)

It kind of proves the point that two fields in a keyword isn't "enough
for everyone".

Back to the topic of the thread, is it possible to make the difference
between e.g. x86, x86-linux, x86-solaris and x86-macos in this proposal?

Thanks,
Fabian


> >> > [1] https://wiki.gentoo.org/wiki/GLEP:53


-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] Proposal: profiles/arches.desc - improve repoman flexibility (with other benefits)

2017-03-27 Thread Fabian Groffen
On 26-03-2017 22:02:38 +0200, Ulrich Mueller wrote:
> >>>>> On Sun, 26 Mar 2017, Fabian Groffen wrote:
> 
> > When you say "arch" you actually mean a keyword as per GLEP-53[1]
> > right?
> 
> Which doesn't agree with actual usage in the tree, though.

That surprises me.  Do you have an example of that?

Thanks,
Fabian

> 
> Ulrich
> 
> > [1] https://wiki.gentoo.org/wiki/GLEP:53



-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] Proposal: profiles/arches.desc - improve repoman flexibility (with other benefits)

2017-03-26 Thread Fabian Groffen
On 26-03-2017 20:54:51 +0200, Andreas K. Huettel wrote:
> Congratulations for getting this far. What's your opinion?

When you say "arch" you actually mean a keyword as per GLEP-53[1] right?

Thanks,
Fabian


[1] https://wiki.gentoo.org/wiki/GLEP:53

-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


Re: [gentoo-dev] News item for Exim 4.88

2017-03-01 Thread Fabian Groffen
On 01-03-2017 14:04:07 -0600, Matthias Maier wrote:
> 
> On Wed, Mar  1, 2017, at 13:43 CST, Fabian Groffen <grob...@gentoo.org> wrote:
> 
> > Hi,
> >
> > I'd like to push out attached news item ASAP.  Please review.
> 
> Looks good. Has a clear and precise structure.

Thanks, I'll adjust the RC capitalisation and push it out.

Fabian


> > Title: =mail-mta/exim-4.88 problem with chunking
> > Author: Fabian Groffen <grob...@gentoo.org>
> > Content-Type: text/plain
> > Posted: 2017-03-01
> > Revision: 1
> > News-Item-Format: 1.0
> > Display-If-Installed: =mail-mta/exim-4.88
> > 
> > Exim maintainers discovered that version 4.88 has some serious problems
> > with its CHUNKING extension.  To quote:
> > 
> >   There are various known problems which can result in messages stuck in
> >   queues and remote servers dropping connections on larger mails.
> > 
> > In Gentoo, Exim 4.88 is the only stable version available, hence all
> > Exim users are advised to either upgrade to an unstable 4.89 Release
> > Candidate, or patch the configuration as follows:
> 
> Maybe not capitalizing "release candidate".
> 
> > 
> > 1) in the main configuration section, add:
> > 
> >   chunking_advertise_hosts =
> > 
> > 2) for each SMTP transport, add:
> > 
> >   hosts_try_chunking =



-- 
Fabian Groffen
Gentoo on a different level


signature.asc
Description: Digital signature


[gentoo-dev] News item for Exim 4.88

2017-03-01 Thread Fabian Groffen
Hi,

I'd like to push out attached news item ASAP.  Please review.

Thanks in advance,
Fabian

-- 
Fabian Groffen
Gentoo on a different level
Title: =mail-mta/exim-4.88 problem with chunking
Author: Fabian Groffen <grob...@gentoo.org>
Content-Type: text/plain
Posted: 2017-03-01
Revision: 1
News-Item-Format: 1.0
Display-If-Installed: =mail-mta/exim-4.88

Exim maintainers discovered that version 4.88 has some serious problems
with its CHUNKING extension.  To quote:

  There are various known problems which can result in messages stuck in
  queues and remote servers dropping connections on larger mails.

In Gentoo, Exim 4.88 is the only stable version available, hence all
Exim users are advised to either upgrade to an unstable 4.89 Release
Candidate, or patch the configuration as follows:

1) in the main configuration section, add:

  chunking_advertise_hosts =

2) for each SMTP transport, add:

  hosts_try_chunking =

Please see also the announcement sent to exim-announce:
https://lists.exim.org/lurker/message/20170301.031117.ff024aa8.en.html


signature.asc
Description: Digital signature


  1   2   3   4   5   6   >