Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-16 Thread Zac Medico
On 03/16/2018 02:13 PM, Michał Górny wrote:
> W dniu pią, 16.03.2018 o godzinie 10∶07 -0700, użytkownik Zac Medico
> napisał:
>> On 03/16/2018 03:08 AM, Michał Górny wrote:
>>> W dniu czw, 15.03.2018 o godzinie 22∶10 -0700, użytkownik Zac Medico
>>> napisał:
 On 03/15/2018 12:22 PM, Michał Górny wrote:
> Hi,
>
> Here are three of four INSTALL_MASK updates I've sent long time ago
> which were not really reviewed. The fourth patch added support
> for repo-defined install-mask.conf and I'll do that separately.
>
> Those patches focus on smaller changes. What they change, in order:
>
> 1. Removes explicit file removal code for FEATURES=no*. Instead, those
>values are converted into additional INSTALL_MASK entries
>and handled directly via INSTALL_MASK processing.
>
> 2. Rework INSTALL_MASK to filter files while installing instead of
>pre-stripping them. In other words, before: INSTALL_MASK removes
>files from ${D} before merge. After: ${D} contains all the files,
>Portage just skip INSTALL_MASK-ed stuff, verbosely indicating that.
>
> 3. Adds support for exclusions in INSTALL_MASK. In other words, you
>can do stuff like:
>
>  INSTALL_MASK="/usr/share/locale -/usr/share/locale/en_US"
>
> I have been using this via user patches since the last submission.
> Guessing by 'git log', this means almost 2 years now.
>
> --
> Best regards,
> Michał Górny
>
> Michał Górny (3):
>   portage.package.ebuild.config: Move FEATURES=no* handling there
>   portage.dbapi.vartree: Move INSTALL_MASK handling into merging
>   portage.dbapi.vartree: Support exclusions in INSTALL_MASK
>
>  bin/misc-functions.sh|  30 --
>  pym/portage/dbapi/vartree.py | 104 
> ++-
>  pym/portage/package/ebuild/config.py |  11 
>  3 files changed, 77 insertions(+), 68 deletions(-)

 I like this patch set but here are some important things that I want it
 to do differently:

 1) For the unmerge code, it needs to read the appropriate
 /var/db/pkg/*/*/{PKG,}INSTALL_MASK file in order to account for the
 {PKG,}INSTALL_MASK settings that existed when the package was built
 (PKG_INSTALL_MASK) and merged (INSTALL_MASK). A binary package should
 use the value of INSTALL_MASK that existed at build time.
 2) In order to support bashrc {PKG,}INSTALL_MASK settings, we need to
 write the values from the environment to
 ${PORTAGE_BUILDDIR}/build-info/{PKG,}INSTALL_MASK and read them from
 there (we do this for many other variables including QA_PREBUILT).
>>>
>>> I presume bin/phase-functions.sh __dyn_install is where I'm supposed to
>>> write them. Could you suggest where is the best place to read them back?
>>
>> We can read them back just when they are needed.
>>
>> PKG_INSTALL_MASK should be handled in the EbuildPhase class when
>> self.phase is "package". In order to preserve behavior, EbuildPhase will
>> have to create a temporary copy of ${D} and apply PKG_INSTALL_MASK to
>> it, for __dyn_package to use.
> 
> But do I need to change anything for PKG_INSTALL_MASK? My original patch
> did not touch that, so it can just continue happening as it is now.
> 
>> INSTALL_MASK should be handled in the dblink treewalk method like it is now.
> 
> But we also need to read it for unmerge, correct?

Oh right. We should load it in the dblink _match_contents or getcontents
method, since we need it for operation of the _match_contents method
which is called by isowner.

>>> Should the merge code do that explicitly while handling INSTALL_MASK, or
>>> should some of the config classes do that?
>>
>> The config class only needs to be involved if we want to expose some API
>> related to {PKG,}INSTALL_MASK there, but the config class is bloated
>> enough as it is so it's better to expose a helper class like the
>> ConfigProtect class.
> 


-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-16 Thread Michał Górny
W dniu pią, 16.03.2018 o godzinie 10∶07 -0700, użytkownik Zac Medico
napisał:
> On 03/16/2018 03:08 AM, Michał Górny wrote:
> > W dniu czw, 15.03.2018 o godzinie 22∶10 -0700, użytkownik Zac Medico
> > napisał:
> > > On 03/15/2018 12:22 PM, Michał Górny wrote:
> > > > Hi,
> > > > 
> > > > Here are three of four INSTALL_MASK updates I've sent long time ago
> > > > which were not really reviewed. The fourth patch added support
> > > > for repo-defined install-mask.conf and I'll do that separately.
> > > > 
> > > > Those patches focus on smaller changes. What they change, in order:
> > > > 
> > > > 1. Removes explicit file removal code for FEATURES=no*. Instead, those
> > > >values are converted into additional INSTALL_MASK entries
> > > >and handled directly via INSTALL_MASK processing.
> > > > 
> > > > 2. Rework INSTALL_MASK to filter files while installing instead of
> > > >pre-stripping them. In other words, before: INSTALL_MASK removes
> > > >files from ${D} before merge. After: ${D} contains all the files,
> > > >Portage just skip INSTALL_MASK-ed stuff, verbosely indicating that.
> > > > 
> > > > 3. Adds support for exclusions in INSTALL_MASK. In other words, you
> > > >can do stuff like:
> > > > 
> > > >  INSTALL_MASK="/usr/share/locale -/usr/share/locale/en_US"
> > > > 
> > > > I have been using this via user patches since the last submission.
> > > > Guessing by 'git log', this means almost 2 years now.
> > > > 
> > > > --
> > > > Best regards,
> > > > Michał Górny
> > > > 
> > > > Michał Górny (3):
> > > >   portage.package.ebuild.config: Move FEATURES=no* handling there
> > > >   portage.dbapi.vartree: Move INSTALL_MASK handling into merging
> > > >   portage.dbapi.vartree: Support exclusions in INSTALL_MASK
> > > > 
> > > >  bin/misc-functions.sh|  30 --
> > > >  pym/portage/dbapi/vartree.py | 104 
> > > > ++-
> > > >  pym/portage/package/ebuild/config.py |  11 
> > > >  3 files changed, 77 insertions(+), 68 deletions(-)
> > > 
> > > I like this patch set but here are some important things that I want it
> > > to do differently:
> > > 
> > > 1) For the unmerge code, it needs to read the appropriate
> > > /var/db/pkg/*/*/{PKG,}INSTALL_MASK file in order to account for the
> > > {PKG,}INSTALL_MASK settings that existed when the package was built
> > > (PKG_INSTALL_MASK) and merged (INSTALL_MASK). A binary package should
> > > use the value of INSTALL_MASK that existed at build time.
> > > 2) In order to support bashrc {PKG,}INSTALL_MASK settings, we need to
> > > write the values from the environment to
> > > ${PORTAGE_BUILDDIR}/build-info/{PKG,}INSTALL_MASK and read them from
> > > there (we do this for many other variables including QA_PREBUILT).
> > 
> > I presume bin/phase-functions.sh __dyn_install is where I'm supposed to
> > write them. Could you suggest where is the best place to read them back?
> 
> We can read them back just when they are needed.
> 
> PKG_INSTALL_MASK should be handled in the EbuildPhase class when
> self.phase is "package". In order to preserve behavior, EbuildPhase will
> have to create a temporary copy of ${D} and apply PKG_INSTALL_MASK to
> it, for __dyn_package to use.

But do I need to change anything for PKG_INSTALL_MASK? My original patch
did not touch that, so it can just continue happening as it is now.

> INSTALL_MASK should be handled in the dblink treewalk method like it is now.

But we also need to read it for unmerge, correct?

> > Should the merge code do that explicitly while handling INSTALL_MASK, or
> > should some of the config classes do that?
> 
> The config class only needs to be involved if we want to expose some API
> related to {PKG,}INSTALL_MASK there, but the config class is bloated
> enough as it is so it's better to expose a helper class like the
> ConfigProtect class.

-- 
Best regards,
Michał Górny




Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-16 Thread Zac Medico
On 03/16/2018 03:08 AM, Michał Górny wrote:
> W dniu czw, 15.03.2018 o godzinie 22∶10 -0700, użytkownik Zac Medico
> napisał:
>> On 03/15/2018 12:22 PM, Michał Górny wrote:
>>> Hi,
>>>
>>> Here are three of four INSTALL_MASK updates I've sent long time ago
>>> which were not really reviewed. The fourth patch added support
>>> for repo-defined install-mask.conf and I'll do that separately.
>>>
>>> Those patches focus on smaller changes. What they change, in order:
>>>
>>> 1. Removes explicit file removal code for FEATURES=no*. Instead, those
>>>values are converted into additional INSTALL_MASK entries
>>>and handled directly via INSTALL_MASK processing.
>>>
>>> 2. Rework INSTALL_MASK to filter files while installing instead of
>>>pre-stripping them. In other words, before: INSTALL_MASK removes
>>>files from ${D} before merge. After: ${D} contains all the files,
>>>Portage just skip INSTALL_MASK-ed stuff, verbosely indicating that.
>>>
>>> 3. Adds support for exclusions in INSTALL_MASK. In other words, you
>>>can do stuff like:
>>>
>>>  INSTALL_MASK="/usr/share/locale -/usr/share/locale/en_US"
>>>
>>> I have been using this via user patches since the last submission.
>>> Guessing by 'git log', this means almost 2 years now.
>>>
>>> --
>>> Best regards,
>>> Michał Górny
>>>
>>> Michał Górny (3):
>>>   portage.package.ebuild.config: Move FEATURES=no* handling there
>>>   portage.dbapi.vartree: Move INSTALL_MASK handling into merging
>>>   portage.dbapi.vartree: Support exclusions in INSTALL_MASK
>>>
>>>  bin/misc-functions.sh|  30 --
>>>  pym/portage/dbapi/vartree.py | 104 
>>> ++-
>>>  pym/portage/package/ebuild/config.py |  11 
>>>  3 files changed, 77 insertions(+), 68 deletions(-)
>>
>> I like this patch set but here are some important things that I want it
>> to do differently:
>>
>> 1) For the unmerge code, it needs to read the appropriate
>> /var/db/pkg/*/*/{PKG,}INSTALL_MASK file in order to account for the
>> {PKG,}INSTALL_MASK settings that existed when the package was built
>> (PKG_INSTALL_MASK) and merged (INSTALL_MASK). A binary package should
>> use the value of INSTALL_MASK that existed at build time.
> 
>> 2) In order to support bashrc {PKG,}INSTALL_MASK settings, we need to
>> write the values from the environment to
>> ${PORTAGE_BUILDDIR}/build-info/{PKG,}INSTALL_MASK and read them from
>> there (we do this for many other variables including QA_PREBUILT).
> 
> I presume bin/phase-functions.sh __dyn_install is where I'm supposed to
> write them. Could you suggest where is the best place to read them back?

We can read them back just when they are needed.

PKG_INSTALL_MASK should be handled in the EbuildPhase class when
self.phase is "package". In order to preserve behavior, EbuildPhase will
have to create a temporary copy of ${D} and apply PKG_INSTALL_MASK to
it, for __dyn_package to use.

INSTALL_MASK should be handled in the dblink treewalk method like it is now.


> Should the merge code do that explicitly while handling INSTALL_MASK, or
> should some of the config classes do that?

The config class only needs to be involved if we want to expose some API
related to {PKG,}INSTALL_MASK there, but the config class is bloated
enough as it is so it's better to expose a helper class like the
ConfigProtect class.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH] portage.dbapi.vartree: Remove one more unfounded virtual case

2018-03-16 Thread Zac Medico
On 03/16/2018 03:05 AM, Michał Górny wrote:
> ---
>  pym/portage/dbapi/vartree.py | 7 ---
>  1 file changed, 7 deletions(-)
> 
> diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
> index 8b1b77f7d..6406682d6 100644
> --- a/pym/portage/dbapi/vartree.py
> +++ b/pym/portage/dbapi/vartree.py
> @@ -3747,13 +3747,6 @@ class dblink(object):
>   is_binpkg = self.settings.get("EMERGE_FROM") == "binary"
>   slot = ''
>   for var_name in ('CHOST', 'SLOT'):
> - if var_name == 'CHOST' and self.cat == 'virtual':
> - try:
> - os.unlink(os.path.join(inforoot, 
> var_name))
> - except OSError:
> - pass
> - continue
> -
>   try:
>   with io.open(_unicode_encode(
>   os.path.join(inforoot, var_name),
> 

Looks good, please merge.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature


Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-16 Thread Michał Górny
W dniu czw, 15.03.2018 o godzinie 22∶10 -0700, użytkownik Zac Medico
napisał:
> On 03/15/2018 12:22 PM, Michał Górny wrote:
> > Hi,
> > 
> > Here are three of four INSTALL_MASK updates I've sent long time ago
> > which were not really reviewed. The fourth patch added support
> > for repo-defined install-mask.conf and I'll do that separately.
> > 
> > Those patches focus on smaller changes. What they change, in order:
> > 
> > 1. Removes explicit file removal code for FEATURES=no*. Instead, those
> >values are converted into additional INSTALL_MASK entries
> >and handled directly via INSTALL_MASK processing.
> > 
> > 2. Rework INSTALL_MASK to filter files while installing instead of
> >pre-stripping them. In other words, before: INSTALL_MASK removes
> >files from ${D} before merge. After: ${D} contains all the files,
> >Portage just skip INSTALL_MASK-ed stuff, verbosely indicating that.
> > 
> > 3. Adds support for exclusions in INSTALL_MASK. In other words, you
> >can do stuff like:
> > 
> >  INSTALL_MASK="/usr/share/locale -/usr/share/locale/en_US"
> > 
> > I have been using this via user patches since the last submission.
> > Guessing by 'git log', this means almost 2 years now.
> > 
> > --
> > Best regards,
> > Michał Górny
> > 
> > Michał Górny (3):
> >   portage.package.ebuild.config: Move FEATURES=no* handling there
> >   portage.dbapi.vartree: Move INSTALL_MASK handling into merging
> >   portage.dbapi.vartree: Support exclusions in INSTALL_MASK
> > 
> >  bin/misc-functions.sh|  30 --
> >  pym/portage/dbapi/vartree.py | 104 
> > ++-
> >  pym/portage/package/ebuild/config.py |  11 
> >  3 files changed, 77 insertions(+), 68 deletions(-)
> 
> I like this patch set but here are some important things that I want it
> to do differently:
> 
> 1) For the unmerge code, it needs to read the appropriate
> /var/db/pkg/*/*/{PKG,}INSTALL_MASK file in order to account for the
> {PKG,}INSTALL_MASK settings that existed when the package was built
> (PKG_INSTALL_MASK) and merged (INSTALL_MASK). A binary package should
> use the value of INSTALL_MASK that existed at build time.

> 2) In order to support bashrc {PKG,}INSTALL_MASK settings, we need to
> write the values from the environment to
> ${PORTAGE_BUILDDIR}/build-info/{PKG,}INSTALL_MASK and read them from
> there (we do this for many other variables including QA_PREBUILT).

I presume bin/phase-functions.sh __dyn_install is where I'm supposed to
write them. Could you suggest where is the best place to read them back?
Should the merge code do that explicitly while handling INSTALL_MASK, or
should some of the config classes do that?

-- 
Best regards,
Michał Górny




[gentoo-portage-dev] [PATCH] portage.dbapi.vartree: Remove one more unfounded virtual case

2018-03-16 Thread Michał Górny
---
 pym/portage/dbapi/vartree.py | 7 ---
 1 file changed, 7 deletions(-)

diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index 8b1b77f7d..6406682d6 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -3747,13 +3747,6 @@ class dblink(object):
is_binpkg = self.settings.get("EMERGE_FROM") == "binary"
slot = ''
for var_name in ('CHOST', 'SLOT'):
-   if var_name == 'CHOST' and self.cat == 'virtual':
-   try:
-   os.unlink(os.path.join(inforoot, 
var_name))
-   except OSError:
-   pass
-   continue
-
try:
with io.open(_unicode_encode(
os.path.join(inforoot, var_name),
-- 
2.16.2




[gentoo-portage-dev] [PATCH] EventLoop: implement time method for asyncio compat (bug 591760)

2018-03-16 Thread Zac Medico
Use time.monotonic() which is available in Python 3.3 and later,
and otherwise emulate it by using an offset to counteract any
backward movements.

Bug: https://bugs.gentoo.org/591760
---
 pym/portage/util/_eventloop/EventLoop.py | 19 ++-
 pym/portage/util/monotonic.py| 32 
 2 files changed, 46 insertions(+), 5 deletions(-)
 create mode 100644 pym/portage/util/monotonic.py

diff --git a/pym/portage/util/_eventloop/EventLoop.py 
b/pym/portage/util/_eventloop/EventLoop.py
index 89ac2a3b3..f472a3dae 100644
--- a/pym/portage/util/_eventloop/EventLoop.py
+++ b/pym/portage/util/_eventloop/EventLoop.py
@@ -1,4 +1,4 @@
-# Copyright 1999-2016 Gentoo Foundation
+# Copyright 1999-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 from __future__ import division
@@ -9,7 +9,6 @@ import os
 import select
 import signal
 import sys
-import time
 
 try:
import fcntl
@@ -29,6 +28,7 @@ portage.proxy.lazyimport.lazyimport(globals(),
 
 from portage import OrderedDict
 from portage.util import writemsg_level
+from portage.util.monotonic import monotonic
 from ..SlotObject import SlotObject
 from .PollConstants import PollConstants
 from .PollSelectAdapter import PollSelectAdapter
@@ -515,7 +515,7 @@ class EventLoop(object):
self._timeout_handlers[source_id] = \
self._timeout_handler_class(
interval=interval, function=function, 
args=args,
-   source_id=source_id, 
timestamp=time.time())
+   source_id=source_id, 
timestamp=self.time())
if self._timeout_interval is None or \
self._timeout_interval > interval:
self._timeout_interval = interval
@@ -538,7 +538,7 @@ class EventLoop(object):
return bool(calls)
 
ready_timeouts = []
-   current_time = time.time()
+   current_time = self.time()
for x in self._timeout_handlers.values():
elapsed_seconds = current_time - x.timestamp
# elapsed_seconds < 0 means the system clock 
has been adjusted
@@ -558,7 +558,7 @@ class EventLoop(object):
calls += 1
x.calling = True
try:
-   x.timestamp = time.time()
+   x.timestamp = self.time()
if not x.function(*x.args):
self.source_remove(x.source_id)
finally:
@@ -684,6 +684,15 @@ class EventLoop(object):
# The call_soon method inherits thread safety from the idle_add method.
call_soon_threadsafe = call_soon
 
+   def time(self):
+   """Return the time according to the event loop's clock.
+
+   This is a float expressed in seconds since an epoch, but the
+   epoch, precision, accuracy and drift are unspecified and may
+   differ per event loop.
+   """
+   return monotonic()
+
def call_later(self, delay, callback, *args):
"""
Arrange for the callback to be called after the given delay 
seconds
diff --git a/pym/portage/util/monotonic.py b/pym/portage/util/monotonic.py
new file mode 100644
index 0..da1cce1c5
--- /dev/null
+++ b/pym/portage/util/monotonic.py
@@ -0,0 +1,32 @@
+# Copyright 2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+__all__ = ['monotonic']
+
+import time
+try:
+   import threading
+except ImportError:
+   import dummy_threading as threading
+
+monotonic = getattr(time, 'monotonic', None)
+
+if monotonic is None:
+   def monotonic():
+   """
+   Emulate time.monotonic() which is available in Python 3.3 and 
later.
+
+   @return: A float expressed in seconds since an epoch.
+   """
+   with monotonic._lock:
+   current = time.time()
+   delta = current - monotonic._previous
+   if delta < 0:
+   monotonic._offset -= delta
+   monotonic._previous = current + monotonic._offset
+   return monotonic._previous
+
+   # offset is used to counteract any backward movements
+   monotonic._offset = 0
+   monotonic._previous = time.time()
+   monotonic._lock = threading.Lock()
-- 
2.13.6




Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-16 Thread Joakim Tjernlund
On Thu, 2018-03-15 at 22:10 -0700, Zac Medico wrote:
> On 03/15/2018 12:22 PM, Michał Górny wrote:
> > Hi,
> > 
> > Here are three of four INSTALL_MASK updates I've sent long time ago
> > which were not really reviewed. The fourth patch added support
> > for repo-defined install-mask.conf and I'll do that separately.
> > 
> > Those patches focus on smaller changes. What they change, in order:
> > 
> > 1. Removes explicit file removal code for FEATURES=no*. Instead, those
> >values are converted into additional INSTALL_MASK entries
> >and handled directly via INSTALL_MASK processing.
> > 
> > 2. Rework INSTALL_MASK to filter files while installing instead of
> >pre-stripping them. In other words, before: INSTALL_MASK removes
> >files from ${D} before merge. After: ${D} contains all the files,
> >Portage just skip INSTALL_MASK-ed stuff, verbosely indicating that.
> > 
> > 3. Adds support for exclusions in INSTALL_MASK. In other words, you
> >can do stuff like:
> > 
> >  INSTALL_MASK="/usr/share/locale -/usr/share/locale/en_US"
> > 
> > I have been using this via user patches since the last submission.
> > Guessing by 'git log', this means almost 2 years now.
> > 
> > --
> > Best regards,
> > Michał Górny
> > 
> > Michał Górny (3):
> >   portage.package.ebuild.config: Move FEATURES=no* handling there
> >   portage.dbapi.vartree: Move INSTALL_MASK handling into merging
> >   portage.dbapi.vartree: Support exclusions in INSTALL_MASK
> > 
> >  bin/misc-functions.sh|  30 --
> >  pym/portage/dbapi/vartree.py | 104 
> > ++-
> >  pym/portage/package/ebuild/config.py |  11 
> >  3 files changed, 77 insertions(+), 68 deletions(-)
> 
> I like this patch set but here are some important things that I want it
> to do differently:
> 
> 1) For the unmerge code, it needs to read the appropriate
> /var/db/pkg/*/*/{PKG,}INSTALL_MASK file in order to account for the
> {PKG,}INSTALL_MASK settings that existed when the package was built
> (PKG_INSTALL_MASK) and merged (INSTALL_MASK). A binary package should
> use the value of INSTALL_MASK that existed at build time.

Why does unmerge code need to know PKG_INSTALL_MASK? The files are not
installed so nothing to filter ?
similarly for merge of binary pkgs I guess, PKG_INSTALL_MASKed files are not
in the binary packed so no need to something special in this case ?

> 
> 2) In order to support bashrc {PKG,}INSTALL_MASK settings, we need to
> write the values from the environment to
> ${PORTAGE_BUILDDIR}/build-info/{PKG,}INSTALL_MASK and read them from
> there (we do this for many other variables including QA_PREBUILT).


Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-16 Thread Michał Górny
W dniu pią, 16.03.2018 o godzinie 08∶11 +, użytkownik Joakim
Tjernlund napisał:
> On Thu, 2018-03-15 at 20:22 +0100, Michał Górny wrote:
> > CAUTION: This email originated from outside of the organization. Do not 
> > click links or open attachments unless you recognize the sender and know 
> > the content is safe.
> > 
> > 
> > Hi,
> > 
> > Here are three of four INSTALL_MASK updates I've sent long time ago
> > which were not really reviewed. The fourth patch added support
> > for repo-defined install-mask.conf and I'll do that separately.
> > 
> > Those patches focus on smaller changes. What they change, in order:
> > 
> > 1. Removes explicit file removal code for FEATURES=no*. Instead, those
> >values are converted into additional INSTALL_MASK entries
> >and handled directly via INSTALL_MASK processing.
> > 
> > 2. Rework INSTALL_MASK to filter files while installing instead of
> >pre-stripping them. In other words, before: INSTALL_MASK removes
> >files from ${D} before merge. After: ${D} contains all the files,
> >Portage just skip INSTALL_MASK-ed stuff, verbosely indicating that.
> 
> Will this also remove corresponding split debug files?
> There would be little/no point in keeping debug syms if the binary has been
> MASKed 
> 

Nope. Add both paths to INSTALL_MASK. Expecting it to do implicit magic
is a very bad idea.

-- 
Best regards,
Michał Górny




[gentoo-portage-dev] [PATCH] Stop crippling metadata for virtual/*

2018-03-16 Thread Michał Górny
There is no technical requirement that virtual/* packages will reliably
contain no executable code. Therefore, stop crippling the metadata
stored in vdb for them.
---
 bin/phase-functions.sh | 24 ++--
 pym/portage/package/ebuild/doebuild.py | 11 ---
 2 files changed, 14 insertions(+), 21 deletions(-)

diff --git a/bin/phase-functions.sh b/bin/phase-functions.sh
index 0ceddb451..6ae61b2f0 100644
--- a/bin/phase-functions.sh
+++ b/bin/phase-functions.sh
@@ -662,23 +662,19 @@ __dyn_install() {
local f x
IFS=$' \t\n\r'
for f in CATEGORY DEFINED_PHASES FEATURES INHERITED IUSE \
-   PF PKGUSE SLOT KEYWORDS HOMEPAGE DESCRIPTION ; do
+   PF PKGUSE SLOT KEYWORDS HOMEPAGE DESCRIPTION \
+   ASFLAGS CBUILD CC CFLAGS CHOST CTARGET CXX \
+   CXXFLAGS EXTRA_ECONF EXTRA_EINSTALL EXTRA_MAKE \
+   LDFLAGS LIBCFLAGS LIBCXXFLAGS QA_CONFIGURE_OPTIONS \
+   QA_DESKTOP_FILE QA_PREBUILT PROVIDES_EXCLUDE REQUIRES_EXCLUDE ; 
do
+
x=$(echo -n ${!f})
[[ -n $x ]] && echo "$x" > $f
done
-   if [[ $CATEGORY != virtual ]] ; then
-   for f in ASFLAGS CBUILD CC CFLAGS CHOST CTARGET CXX \
-   CXXFLAGS EXTRA_ECONF EXTRA_EINSTALL EXTRA_MAKE \
-   LDFLAGS LIBCFLAGS LIBCXXFLAGS QA_CONFIGURE_OPTIONS \
-   QA_DESKTOP_FILE QA_PREBUILT PROVIDES_EXCLUDE 
REQUIRES_EXCLUDE ; do
-   x=$(echo -n ${!f})
-   [[ -n $x ]] && echo "$x" > $f
-   done
-   # whitespace preserved
-   for f in QA_AM_MAINTAINER_MODE ; do
-   [[ -n ${!f} ]] && echo "${!f}" > $f
-   done
-   fi
+   # whitespace preserved
+   for f in QA_AM_MAINTAINER_MODE ; do
+   [[ -n ${!f} ]] && echo "${!f}" > $f
+   done
echo "${USE}"   > USE
echo "${EAPI:-0}"   > EAPI
 
diff --git a/pym/portage/package/ebuild/doebuild.py 
b/pym/portage/package/ebuild/doebuild.py
index 8cec8e4da..8436c0b10 100644
--- a/pym/portage/package/ebuild/doebuild.py
+++ b/pym/portage/package/ebuild/doebuild.py
@@ -1930,13 +1930,10 @@ def _post_src_install_write_metadata(settings):
if v is not None:
write_atomic(os.path.join(build_info_dir, k), v + '\n')
 
-   # The following variables are irrelevant for virtual packages.
-   if settings.get('CATEGORY') != 'virtual':
-
-   for k in ('CHOST',):
-   v = settings.get(k)
-   if v is not None:
-   write_atomic(os.path.join(build_info_dir, k), v 
+ '\n')
+   for k in ('CHOST',):
+   v = settings.get(k)
+   if v is not None:
+   write_atomic(os.path.join(build_info_dir, k), v + '\n')
 
with io.open(_unicode_encode(os.path.join(build_info_dir,
'BUILD_TIME'), encoding=_encodings['fs'], errors='strict'),
-- 
2.16.2




Re: [gentoo-portage-dev] [PATCH 0/3] INSTALL_MASK refurbishing resubmit

2018-03-16 Thread Joakim Tjernlund
On Thu, 2018-03-15 at 20:22 +0100, Michał Górny wrote:
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open attachments unless you recognize the sender and know the 
> content is safe.
> 
> 
> Hi,
> 
> Here are three of four INSTALL_MASK updates I've sent long time ago
> which were not really reviewed. The fourth patch added support
> for repo-defined install-mask.conf and I'll do that separately.
> 
> Those patches focus on smaller changes. What they change, in order:
> 
> 1. Removes explicit file removal code for FEATURES=no*. Instead, those
>values are converted into additional INSTALL_MASK entries
>and handled directly via INSTALL_MASK processing.
> 
> 2. Rework INSTALL_MASK to filter files while installing instead of
>pre-stripping them. In other words, before: INSTALL_MASK removes
>files from ${D} before merge. After: ${D} contains all the files,
>Portage just skip INSTALL_MASK-ed stuff, verbosely indicating that.

Will this also remove corresponding split debug files?
There would be little/no point in keeping debug syms if the binary has been
MASKed 

 Jocke

Re: [gentoo-portage-dev] [PATCH 3/3] portage.dbapi.vartree: Support exclusions in INSTALL_MASK

2018-03-16 Thread Joakim Tjernlund
On Fri, 2018-03-16 at 08:50 +0100, Michał Górny wrote:
> CAUTION: This email originated from outside of the organization. Do not click 
> links or open attachments unless you recognize the sender and know the 
> content is safe.
> 
> 
> W dniu czw, 15.03.2018 o godzinie 21∶44 +, użytkownik Joakim
> Tjernlund napisał:
> > On Thu, 2018-03-15 at 17:02 -0400, Alec Warner wrote:
> > > 
> > > 
> > > On Thu, Mar 15, 2018 at 3:22 PM, Michał Górny  wrote:
> > > > Allow INSTALL_MASK patterns to start with '-' to indicate that
> > > > a specific match is to be excluded from being masked. In this case,
> > > > the last matching pattern determines whether the file is actually
> > > > filtered out or kept.
> > > > ---
> > 
> > Yes, please ! I just needed this feature 2 hours ago.
> > I need the same for PKG_INSTALL_MASK
> > 
> > I hope this allows me to do:
> > INSTALL_MASK="/usr/share/i18n/locales/* -/usr/share/i18n/locales/en_GB 
> > -/usr/share/i18n/locales/sv_SE"
> > to all in /usr/share/i18n/locales/* but /usr/share/i18n/locales/en_GB and 
> > /usr/share/i18n/locales/sv_SE
> > removed in usr/share/i18n/locales/ ?
> > 
> 
> Yes, that is the intended use case.

Thanks, while on the subject I had a similar idea for stripping:

Today one have to build all files in glibc with debug syms just to get debug 
syms for
perf/valgrind in ld.so so it would be great if one could specify that just some 
files
should have debug syms.

 Jocke

Re: [gentoo-portage-dev] [PATCH 3/3] portage.dbapi.vartree: Support exclusions in INSTALL_MASK

2018-03-16 Thread Michał Górny
W dniu czw, 15.03.2018 o godzinie 21∶44 +, użytkownik Joakim
Tjernlund napisał:
> On Thu, 2018-03-15 at 17:02 -0400, Alec Warner wrote:
> > 
> > 
> > On Thu, Mar 15, 2018 at 3:22 PM, Michał Górny  wrote:
> > > Allow INSTALL_MASK patterns to start with '-' to indicate that
> > > a specific match is to be excluded from being masked. In this case,
> > > the last matching pattern determines whether the file is actually
> > > filtered out or kept.
> > > ---
> 
> Yes, please ! I just needed this feature 2 hours ago.
> I need the same for PKG_INSTALL_MASK 
> 
> I hope this allows me to do:
> INSTALL_MASK="/usr/share/i18n/locales/* -/usr/share/i18n/locales/en_GB 
> -/usr/share/i18n/locales/sv_SE"
> to all in /usr/share/i18n/locales/* but /usr/share/i18n/locales/en_GB and 
> /usr/share/i18n/locales/sv_SE
> removed in usr/share/i18n/locales/ ?
> 

Yes, that is the intended use case.

-- 
Best regards,
Michał Górny