Re: [gentoo-portage-dev] [PATCH 2/2] Add iter_completed convenience function (bug 648790)

[Zac Medico]

On 02/25/2018 07:17 PM, Alec Warner wrote:
> 
> 
> On Sun, Feb 25, 2018 at 8:50 PM, Zac Medico  > wrote:
> 
> The iter_completed function is similar to asyncio.as_completed, but
> takes an iterator of futures as input, and includes support for
> max_jobs and max_load parameters. The default values for max_jobs
> and max_load correspond to multiprocessing.cpu_count().
> 
> Example usage for async_aux_get:
> 
>   import portage
>   from portage.util.futures.iter_completed import iter_completed
> 
>   portdb = portage.portdb
>   future_cpv = {}
> 
> 
> I'm not sure I grasp the purpose of this dict, can't we just modify the
> async aux get to return the cpv from the future?

If we do that then we should probably return all of the other aux_get
inputs too, including mylist, mytree, and myrepo. Pretty soon it feels
like there's a lot of clutter here. If we leave the burden to the
caller, then the API is simpler, and it's not much of a burden to the
caller anyway.

>   def future_generator():
>     for cpv in portdb.cp_list('sys-apps/portage'):
>       future = portdb.async_aux_get(cpv, portage.auxdbkeys)
>       future_cpv[id(future)] = cpv
>       yield future
> 
> 
> for cpv in portdb.cp_list('...'):
>    yield portdb.async_aux_get(cpv, portage.auxdbkeys)
>  
> 
>   for future in iter_completed(future_generator()):
>     cpv = future_cpv.pop(id(future))
>     try:
>       result = future.result()
>     except KeyError as e:
>       # aux_get failed
>       print('error:', cpv, e)
>     else:
>       print(cpv, result)
> 
> 
> for future in iter_completed(future_generator()):
>   try:
>     cpv, result = future.result() 
>   except KeyError as e:
>     print('error', cpv, e)
>  
> 
> Or do we expect callers to need other things to key off of in this API?

Yeah it's complicated because of the number of input arguments to
aux_get. You can have the same cpv existing in multiple repos. It's so
much simpler to let the caller manage the mapping from input arguments
to future instance.
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH 2/2] Add iter_completed convenience function (bug 648790)

[Alec Warner]

On Sun, Feb 25, 2018 at 8:50 PM, Zac Medico  wrote:

> The iter_completed function is similar to asyncio.as_completed, but
> takes an iterator of futures as input, and includes support for
> max_jobs and max_load parameters. The default values for max_jobs
> and max_load correspond to multiprocessing.cpu_count().
>
> Example usage for async_aux_get:
>
>   import portage
>   from portage.util.futures.iter_completed import iter_completed
>
>   portdb = portage.portdb
>   future_cpv = {}
>

I'm not sure I grasp the purpose of this dict, can't we just modify the
async aux get to return the cpv from the future?


>
>   def future_generator():
> for cpv in portdb.cp_list('sys-apps/portage'):
>   future = portdb.async_aux_get(cpv, portage.auxdbkeys)
>   future_cpv[id(future)] = cpv
>   yield future
>
>
for cpv in portdb.cp_list('...'):
   yield portdb.async_aux_get(cpv, portage.auxdbkeys)


>   for future in iter_completed(future_generator()):
> cpv = future_cpv.pop(id(future))
> try:
>   result = future.result()
> except KeyError as e:
>   # aux_get failed
>   print('error:', cpv, e)
> else:
>   print(cpv, result)
>

for future in iter_completed(future_generator()):
  try:
cpv, result = future.result()
  except KeyError as e:
print('error', cpv, e)


Or do we expect callers to need other things to key off of in this API?

-A


> See: https://docs.python.org/3/library/asyncio-task.html#
> asyncio.as_completed
> Bug: https://bugs.gentoo.org/648790
> ---
>  .../tests/util/futures/test_iter_completed.py  | 50 
>  pym/portage/util/_async/FuturePollTask.py  | 27 ++
>  pym/portage/util/futures/iter_completed.py | 63 ++
>  pym/portage/util/futures/wait.py   | 95
> ++
>  4 files changed, 235 insertions(+)
>  create mode 100644 pym/portage/tests/util/futures/test_iter_completed.py
>  create mode 100644 pym/portage/util/_async/FuturePollTask.py
>  create mode 100644 pym/portage/util/futures/iter_completed.py
>  create mode 100644 pym/portage/util/futures/wait.py
>
> diff --git a/pym/portage/tests/util/futures/test_iter_completed.py
> b/pym/portage/tests/util/futures/test_iter_completed.py
> new file mode 100644
> index 0..6607d871c
> --- /dev/null
> +++ b/pym/portage/tests/util/futures/test_iter_completed.py
> @@ -0,0 +1,50 @@
> +# Copyright 2018 Gentoo Foundation
> +# Distributed under the terms of the GNU General Public License v2
> +
> +import time
> +from portage.tests import TestCase
> +from portage.util._async.ForkProcess import ForkProcess
> +from portage.util._eventloop.global_event_loop import global_event_loop
> +from portage.util.futures.iter_completed import iter_completed
> +
> +
> +class SleepProcess(ForkProcess):
> +   __slots__ = ('future', 'seconds')
> +   def _start(self):
> +   self.addExitListener(self._future_done)
> +   ForkProcess._start(self)
> +
> +   def _future_done(self, task):
> +   self.future.set_result(self.seconds)
> +
> +   def _run(self):
> +   time.sleep(self.seconds)
> +
> +
> +class IterCompletedTestCase(TestCase):
> +
> +   def testIterCompleted(self):
> +
> +   # Mark this as todo, since we don't want to fail if heavy
> system
> +   # load causes the tasks to finish in an unexpected order.
> +   self.todo = True
> +
> +   loop = global_event_loop()
> +   tasks = [
> +   SleepProcess(seconds=0.200),
> +   SleepProcess(seconds=0.100),
> +   SleepProcess(seconds=0.001),
> +   ]
> +
> +   expected_order = sorted(task.seconds for task in tasks)
> +
> +   def future_generator():
> +   for task in tasks:
> +   task.future = loop.create_future()
> +   task.scheduler = loop
> +   task.start()
> +   yield task.future
> +
> +   for seconds, future in zip(expected_order,
> iter_completed(future_generator(),
> +   max_jobs=None, max_load=None, loop=loop)):
> +   self.assertEqual(seconds, future.result())
> diff --git a/pym/portage/util/_async/FuturePollTask.py
> b/pym/portage/util/_async/FuturePollTask.py
> new file mode 100644
> index 0..6b7cdf7d5
> --- /dev/null
> +++ b/pym/portage/util/_async/FuturePollTask.py
> @@ -0,0 +1,27 @@
> +# Copyright 2018 Gentoo Foundation
> +# Distributed under the terms of the GNU General Public License v2
> +
> +import os
> +import signal
> +
> +from _emerge.AbstractPollTask import AbstractPollTask
> +
> +
> +class FuturePollTask(AbstractPollTask):
> +   """
> +   Wraps a Future in an AsynchronousTask, which is useful for
> +   scheduling with TaskScheduler.
> +   """
> +

[gentoo-portage-dev] [PATCH 1/2] portdbapi: add async_aux_get method (bug 648790)

[Zac Medico]

Add async_aux_get method that returns a Future and otherwise
behaves identically to aux_get. Use async_aux_get to implement
the synchronous aux_get method.

Bug: https://bugs.gentoo.org/648790
---
 pym/portage/dbapi/porttree.py | 91 +--
 1 file changed, 70 insertions(+), 21 deletions(-)

diff --git a/pym/portage/dbapi/porttree.py b/pym/portage/dbapi/porttree.py
index f5979d2d0..abcd47238 100644
--- a/pym/portage/dbapi/porttree.py
+++ b/pym/portage/dbapi/porttree.py
@@ -45,6 +45,7 @@ import traceback
 import warnings
 import errno
 import collections
+import functools
 
 try:
from urllib.parse import urlparse
@@ -577,11 +578,46 @@ class portdbapi(dbapi):
"stub code for returning auxilliary db information, such as 
SLOT, DEPEND, etc."
'input: "sys-apps/foo-1.0",["SLOT","DEPEND","HOMEPAGE"]'
'return: ["0",">=sys-libs/bar-1.0","http://www.foo.com;] or 
raise PortageKeyError if error'
+   # For external API consumers, self._event_loop returns a new 
event
+   # loop on each access, so a local reference is needed in order
+   # to avoid instantiating more than one.
+   loop = self._event_loop
+   return loop.run_until_complete(
+   self.async_aux_get(mycpv, mylist, mytree=mytree,
+   myrepo=myrepo, loop=loop))
+
+   def async_aux_get(self, mycpv, mylist, mytree=None, myrepo=None, 
loop=None):
+   """
+   Asynchronous form form of aux_get.
+
+   @param mycpv: cpv for an ebuild
+   @type mycpv: str
+   @param mylist: list of metadata keys
+   @type mylist: list
+   @param mytree: The canonical path of the tree in which the 
ebuild
+   is located, or None for automatic lookup
+   @type mytree: str
+   @param myrepo: name of the repo in which the ebuild is located,
+   or None for automatic lookup
+   @type myrepo: str
+   @param loop: event loop (defaults to global event loop)
+   @type loop: EventLoop
+   @return: list of metadata values
+   @rtype: asyncio.Future (or compatible)
+   """
+   # Don't default to self._event_loop here, since that creates a
+   # local event loop for thread safety, and that could easily lead
+   # to simultaneous instantiation of multiple event loops here.
+   # Callers of this method certainly want the same event loop to
+   # be used for all calls.
+   loop = loop or global_event_loop()
+   future = loop.create_future()
cache_me = False
if myrepo is not None:
mytree = self.treemap.get(myrepo)
if mytree is None:
-   raise PortageKeyError(myrepo)
+   future.set_exception(PortageKeyError(myrepo))
+   return future
 
if mytree is not None and len(self.porttrees) == 1 \
and mytree == self.porttrees[0]:
@@ -596,43 +632,56 @@ class portdbapi(dbapi):
mylist).difference(self._aux_cache_keys):
aux_cache = self._aux_cache.get(mycpv)
if aux_cache is not None:
-   return [aux_cache.get(x, "") for x in mylist]
+   future.set_result([aux_cache.get(x, "") for x 
in mylist])
+   return future
cache_me = True
 
try:
cat, pkg = mycpv.split("/", 1)
except ValueError:
# Missing slash. Can't find ebuild so raise 
PortageKeyError.
-   raise PortageKeyError(mycpv)
+   future.set_exception(PortageKeyError(mycpv))
+   return future
 
myebuild, mylocation = self.findname2(mycpv, mytree)
 
if not myebuild:
writemsg("!!! aux_get(): %s\n" % \
_("ebuild not found for '%s'") % mycpv, 
noiselevel=1)
-   raise PortageKeyError(mycpv)
+   future.set_exception(PortageKeyError(mycpv))
+   return future
 
mydata, ebuild_hash = self._pull_valid_cache(mycpv, myebuild, 
mylocation)
-   doregen = mydata is None
-
-   if doregen:
-   if myebuild in self._broken_ebuilds:
-   raise PortageKeyError(mycpv)
-
-   proc = EbuildMetadataPhase(cpv=mycpv,
-   ebuild_hash=ebuild_hash, portdb=self,
-   

[gentoo-portage-dev] [PATCH 2/2] Add iter_completed convenience function (bug 648790)

[Zac Medico]

The iter_completed function is similar to asyncio.as_completed, but
takes an iterator of futures as input, and includes support for
max_jobs and max_load parameters. The default values for max_jobs
and max_load correspond to multiprocessing.cpu_count().

Example usage for async_aux_get:

  import portage
  from portage.util.futures.iter_completed import iter_completed

  portdb = portage.portdb
  future_cpv = {}

  def future_generator():
for cpv in portdb.cp_list('sys-apps/portage'):
  future = portdb.async_aux_get(cpv, portage.auxdbkeys)
  future_cpv[id(future)] = cpv
  yield future

  for future in iter_completed(future_generator()):
cpv = future_cpv.pop(id(future))
try:
  result = future.result()
except KeyError as e:
  # aux_get failed
  print('error:', cpv, e)
else:
  print(cpv, result)

See: https://docs.python.org/3/library/asyncio-task.html#asyncio.as_completed
Bug: https://bugs.gentoo.org/648790
---
 .../tests/util/futures/test_iter_completed.py  | 50 
 pym/portage/util/_async/FuturePollTask.py  | 27 ++
 pym/portage/util/futures/iter_completed.py | 63 ++
 pym/portage/util/futures/wait.py   | 95 ++
 4 files changed, 235 insertions(+)
 create mode 100644 pym/portage/tests/util/futures/test_iter_completed.py
 create mode 100644 pym/portage/util/_async/FuturePollTask.py
 create mode 100644 pym/portage/util/futures/iter_completed.py
 create mode 100644 pym/portage/util/futures/wait.py

diff --git a/pym/portage/tests/util/futures/test_iter_completed.py 
b/pym/portage/tests/util/futures/test_iter_completed.py
new file mode 100644
index 0..6607d871c
--- /dev/null
+++ b/pym/portage/tests/util/futures/test_iter_completed.py
@@ -0,0 +1,50 @@
+# Copyright 2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+import time
+from portage.tests import TestCase
+from portage.util._async.ForkProcess import ForkProcess
+from portage.util._eventloop.global_event_loop import global_event_loop
+from portage.util.futures.iter_completed import iter_completed
+
+
+class SleepProcess(ForkProcess):
+   __slots__ = ('future', 'seconds')
+   def _start(self):
+   self.addExitListener(self._future_done)
+   ForkProcess._start(self)
+
+   def _future_done(self, task):
+   self.future.set_result(self.seconds)
+
+   def _run(self):
+   time.sleep(self.seconds)
+
+
+class IterCompletedTestCase(TestCase):
+
+   def testIterCompleted(self):
+
+   # Mark this as todo, since we don't want to fail if heavy system
+   # load causes the tasks to finish in an unexpected order.
+   self.todo = True
+
+   loop = global_event_loop()
+   tasks = [
+   SleepProcess(seconds=0.200),
+   SleepProcess(seconds=0.100),
+   SleepProcess(seconds=0.001),
+   ]
+
+   expected_order = sorted(task.seconds for task in tasks)
+
+   def future_generator():
+   for task in tasks:
+   task.future = loop.create_future()
+   task.scheduler = loop
+   task.start()
+   yield task.future
+
+   for seconds, future in zip(expected_order, 
iter_completed(future_generator(),
+   max_jobs=None, max_load=None, loop=loop)):
+   self.assertEqual(seconds, future.result())
diff --git a/pym/portage/util/_async/FuturePollTask.py 
b/pym/portage/util/_async/FuturePollTask.py
new file mode 100644
index 0..6b7cdf7d5
--- /dev/null
+++ b/pym/portage/util/_async/FuturePollTask.py
@@ -0,0 +1,27 @@
+# Copyright 2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+import os
+import signal
+
+from _emerge.AbstractPollTask import AbstractPollTask
+
+
+class FuturePollTask(AbstractPollTask):
+   """
+   Wraps a Future in an AsynchronousTask, which is useful for
+   scheduling with TaskScheduler.
+   """
+   __slots__ = ('future',)
+   def _start(self):
+   self.future.add_done_callback(self._done_callback)
+
+   def _done_callback(self, future):
+   if future.cancelled():
+   self.cancelled = True
+   self.returncode = -signal.SIGINT
+   elif future.exception() is None:
+   self.returncode = os.EX_OK
+   else:
+   self.returncode = 1
+   self.wait()
diff --git a/pym/portage/util/futures/iter_completed.py 
b/pym/portage/util/futures/iter_completed.py
new file mode 100644
index 0..0540cc986
--- /dev/null
+++ b/pym/portage/util/futures/iter_completed.py
@@ -0,0 +1,63 @@
+# Copyright 2018 Gentoo 

[gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2018-02-25 23:59 UTC

[Robin H. Johnson]

The attached list notes all of the packages that were added or removed
from the tree, for the week ending 2018-02-25 23:59 UTC.

Removals:
app-emulation/qtemu  20180221-21:05 asturm 
e0c7dd5eac7
dev-libs/libindicate-qt  20180221-23:55 asturm 
353f49a4745
dev-ruby/camping 20180224-08:44 graaff 
a5e25f8c15c
dev-ruby/mongo   20180224-08:44 graaff 
dd7b38ec20b
dev-ruby/net-ssh-multi   20180224-08:44 graaff 
b0d86ca5a13
dev-ruby/rbtree  20180224-08:45 graaff 
fb8c8d6110f
dev-ruby/right_http_connection   20180224-08:45 graaff 
6b58d19f1a4
dev-ruby/snmplib 20180224-08:45 graaff 
399e1e9d79f
virtual/ruby-minitest20180224-08:46 graaff 
112676f4d1f
www-apps/jekyll-gist 20180224-08:46 graaff 
b2a96600255

Additions:
app-admin/elasticsearch_exporter 20180220-20:52 mrueg  
734bdefbd48
app-admin/kubectx20180220-17:31 mrueg  
94bceff80e9
app-shells/fzy   20180211-21:30 monsieurp  
b0bc8f3423f
dev-python/os-service-types  20180224-04:42 prometheanfire 
9c7e0b29cdc
dev-python/pyFFTW20180219-16:45 zerochaos  
520a1341018
dev-python/python-octaviaclient  20180224-07:17 prometheanfire 
ef2249b7fe2
dev-python/simplesoapy   20180219-16:46 zerochaos  
612ee967cec
dev-python/simplespectral20180219-16:47 zerochaos  
b383e139723
dev-util/uftrace 20180219-15:52 mrueg  
c32161a1655
media-video/dvd_info 20180223-00:21 beandog
d6b8d94ac7f
net-analyzer/sngrep  20180220-07:27 jer
cab50b4f6fe
net-misc/ntpsec  20180223-22:53 nerdboy
99c380a6547
net-wireless/soapybladerf20180219-16:50 zerochaos  
dce16c98789
net-wireless/soapyhackrf 20180219-16:49 zerochaos  
c7517743c6a
net-wireless/soapy_power 20180219-16:51 zerochaos  
9a5e1b8399e
net-wireless/soapyrtlsdr 20180219-16:49 zerochaos  
8e2b5cbf7d6
net-wireless/soapysdr20180219-16:50 zerochaos  
8d81387a5e6
net-wireless/soapyuhd20180219-16:48 zerochaos  
979c6f51ad2
sci-physics/vgm  20180121-16:35 monsieurp  
44e267976ac
sys-apps/thunderbolt-software-user-space 20180220-17:29 monsieurp  
68d4b4f7e4b
www-misc/buku20171210-17:15 monsieurp  
31543699451
x11-terms/cool-retro-term20180219-22:37 slyfox 
26568447189

--
Robin Hugh Johnson
Gentoo Linux Developer
E-Mail : robb...@gentoo.org
GnuPG FP   : 11AC BA4F 4778 E3F6 E4ED  F38E B27B 944E 3488 4E85
Removed Packages:
www-apps/jekyll-gist,removed,graaff,20180224-08:46,b2a96600255
virtual/ruby-minitest,removed,graaff,20180224-08:46,112676f4d1f
dev-ruby/snmplib,removed,graaff,20180224-08:45,399e1e9d79f
dev-ruby/right_http_connection,removed,graaff,20180224-08:45,6b58d19f1a4
dev-ruby/rbtree,removed,graaff,20180224-08:45,fb8c8d6110f
dev-ruby/net-ssh-multi,removed,graaff,20180224-08:44,b0d86ca5a13
dev-ruby/mongo,removed,graaff,20180224-08:44,dd7b38ec20b
dev-ruby/camping,removed,graaff,20180224-08:44,a5e25f8c15c
dev-libs/libindicate-qt,removed,asturm,20180221-23:55,353f49a4745
app-emulation/qtemu,removed,asturm,20180221-21:05,e0c7dd5eac7
Added Packages:
dev-python/python-octaviaclient,added,prometheanfire,20180224-07:17,ef2249b7fe2
dev-python/os-service-types,added,prometheanfire,20180224-04:42,9c7e0b29cdc
net-misc/ntpsec,added,nerdboy,20180223-22:53,99c380a6547
media-video/dvd_info,added,beandog,20180223-00:21,d6b8d94ac7f
www-misc/buku,added,monsieurp,20171210-17:15,31543699451
sys-apps/thunderbolt-software-user-space,added,monsieurp,20180220-17:29,68d4b4f7e4b
app-shells/fzy,added,monsieurp,20180211-21:30,b0bc8f3423f
app-admin/elasticsearch_exporter,added,mrueg,20180220-20:52,734bdefbd48
sci-physics/vgm,added,monsieurp,20180121-16:35,44e267976ac
app-admin/kubectx,added,mrueg,20180220-17:31,94bceff80e9
net-analyzer/sngrep,added,jer,20180220-07:27,cab50b4f6fe
x11-terms/cool-retro-term,added,slyfox,20180219-22:37,26568447189
net-wireless/soapy_power,added,zerochaos,20180219-16:51,9a5e1b8399e
net-wireless/soapysdr,added,zerochaos,20180219-16:50,8d81387a5e6
net-wireless/soapybladerf,added,zerochaos,20180219-16:50,dce16c98789
net-wireless/soapyhackrf,added,zerochaos,20180219-16:49,c7517743c6a
net-wireless/soapyrtlsdr,added,zerochaos,20180219-16:49,8e2b5cbf7d6
net-wireless/soapyuhd,added,zerochaos,20180219-16:48,979c6f51ad2
dev-python/simplespectral,added,zerochaos,20180219-16:47,b383e139723
dev-python/simplesoapy,added,zerochaos,20180219-16:46,612ee967cec

Re: [gentoo-portage-dev] [PATCH] Deprecate EAPI 6_pre1

[Zac Medico]

On 02/25/2018 11:59 AM, Michał Górny wrote:
> Deprecated the testing variant of EAPI 6.
> ---
>  pym/portage/__init__.py | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/pym/portage/__init__.py b/pym/portage/__init__.py
> index 99f3f98ac..4773738b2 100644
> --- a/pym/portage/__init__.py
> +++ b/pym/portage/__init__.py
> @@ -1,4 +1,4 @@
> -# Copyright 1998-2014 Gentoo Foundation
> +# Copyright 1998-2018 Gentoo Foundation
>  # Distributed under the terms of the GNU General Public License v2
>  
>  from __future__ import unicode_literals
> @@ -462,8 +462,8 @@ def abssymlink(symlink, target=None):
>  
>  _doebuild_manifest_exempt_depend = 0
>  
> -_testing_eapis = frozenset(["4-python", "4-slot-abi", "5-progress", 
> "5-hdepend", "6_pre1"])
> -_deprecated_eapis = frozenset(["4_pre1", "3_pre2", "3_pre1", "5_pre1", 
> "5_pre2"])
> +_testing_eapis = frozenset(["4-python", "4-slot-abi", "5-progress", 
> "5-hdepend"])
> +_deprecated_eapis = frozenset(["4_pre1", "3_pre2", "3_pre1", "5_pre1", 
> "5_pre2", "6_pre1"])
>  _supported_eapis = frozenset([str(x) for x in range(portage.const.EAPI + 1)] 
> + list(_testing_eapis) + list(_deprecated_eapis))
>  
>  def _eapi_is_deprecated(eapi):
> 

Looks good, please merge.
-- 
Thanks,
Zac

[gentoo-portage-dev] [PATCH] Deprecate EAPI 6_pre1

[Michał Górny]

Deprecated the testing variant of EAPI 6.
---
 pym/portage/__init__.py | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/pym/portage/__init__.py b/pym/portage/__init__.py
index 99f3f98ac..4773738b2 100644
--- a/pym/portage/__init__.py
+++ b/pym/portage/__init__.py
@@ -1,4 +1,4 @@
-# Copyright 1998-2014 Gentoo Foundation
+# Copyright 1998-2018 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
 from __future__ import unicode_literals
@@ -462,8 +462,8 @@ def abssymlink(symlink, target=None):
 
 _doebuild_manifest_exempt_depend = 0
 
-_testing_eapis = frozenset(["4-python", "4-slot-abi", "5-progress", 
"5-hdepend", "6_pre1"])
-_deprecated_eapis = frozenset(["4_pre1", "3_pre2", "3_pre1", "5_pre1", 
"5_pre2"])
+_testing_eapis = frozenset(["4-python", "4-slot-abi", "5-progress", 
"5-hdepend"])
+_deprecated_eapis = frozenset(["4_pre1", "3_pre2", "3_pre1", "5_pre1", 
"5_pre2", "6_pre1"])
 _supported_eapis = frozenset([str(x) for x in range(portage.const.EAPI + 1)] + 
list(_testing_eapis) + list(_deprecated_eapis))
 
 def _eapi_is_deprecated(eapi):
-- 
2.16.2

Re: [gentoo-dev] [PATCH] cmake-utils.eclass: Extend ASM rules to ASM-ATT

[Francesco Riosa]

2018-02-25 10:06 GMT+01:00 Michał Górny :

> Some CMake projects use ASM-ATT rather than ASM, so extend our rule
> overrides to that.
>

for the curious:
https://cmake.org/Wiki/CMake/Assembler#ASM-ATT

ASM-ATT

This can be used for assembler files in AT assembler syntax. This
includes the GNU assembler gas.

   - Supported assembler names: as, gas, may have toolchain specific prefix
   - Supported source files extensions: .s, .asm
   - .S files, i.e. assembler files which require preprocessing, are not
   supported
   - Involved files: CMakeASM-ATTInformation.cmake,
   CMakeDetermineASM-ATTCompiler.cmake, CMakeTestASM-ATTCompiler.cmake




>
> Bug: https://bugs.gentoo.org/625844
> ---
>  eclass/cmake-utils.eclass | 3 +++
>  1 file changed, 3 insertions(+)
>
> diff --git a/eclass/cmake-utils.eclass b/eclass/cmake-utils.eclass
> index b9f69a824b14..ef3f3c2607f8 100644
> --- a/eclass/cmake-utils.eclass
> +++ b/eclass/cmake-utils.eclass
> @@ -516,6 +516,7 @@ cmake-utils_src_configure() {
> fi
> cat > "${build_rules}" <<- _EOF_ || die
> SET (CMAKE_ASM_COMPILE_OBJECT "
>  ${includes} ${CPPFLAGS}  -o  -c " CACHE
> STRING "ASM compile command" FORCE)
> +   SET (CMAKE_ASM-ATT_COMPILE_OBJECT "
>  ${includes} ${CPPFLAGS}  -o  -c " CACHE
> STRING "ASM compile command" FORCE)
> SET (CMAKE_C_COMPILE_OBJECT " 
> ${includes} ${CPPFLAGS}  -o  -c " CACHE STRING "C
> compile command" FORCE)
> SET (CMAKE_CXX_COMPILE_OBJECT "
>  ${includes} ${CPPFLAGS}  -o  -c " CACHE
> STRING "C++ compile command" FORCE)
> SET (CMAKE_Fortran_COMPILE_OBJECT
> "  ${includes} ${FCFLAGS}  -o
>  -c " CACHE STRING "Fortran compile command" FORCE)
> @@ -531,6 +532,7 @@ cmake-utils_src_configure() {
> local toolchain_file=${BUILD_DIR}/gentoo_toolchain.cmake
> cat > ${toolchain_file} <<- _EOF_ || die
> SET (CMAKE_ASM_COMPILER "${myCC/ /;}")
> +   SET (CMAKE_ASM-ATT_COMPILER "${myCC/ /;}")
> SET (CMAKE_C_COMPILER "${myCC/ /;}")
> SET (CMAKE_CXX_COMPILER "${myCXX/ /;}")
> SET (CMAKE_Fortran_COMPILER "${myFC/ /;}")
> @@ -609,6 +611,7 @@ cmake-utils_src_configure() {
> if [[ ${CMAKE_BUILD_TYPE} != Gentoo && ${EAPI} != 5 ]]; then
> cat >> ${common_config} <<- _EOF_ || die
> SET (CMAKE_ASM_FLAGS_${CMAKE_BUILD_TYPE^^} ""
> CACHE STRING "")
> +   SET (CMAKE_ASM-ATT_FLAGS_${CMAKE_BUILD_TYPE^^} ""
> CACHE STRING "")
> SET (CMAKE_C_FLAGS_${CMAKE_BUILD_TYPE^^} "" CACHE
> STRING "")
> SET (CMAKE_CXX_FLAGS_${CMAKE_BUILD_TYPE^^} ""
> CACHE STRING "")
> SET (CMAKE_Fortran_FLAGS_${CMAKE_BUILD_TYPE^^} ""
> CACHE STRING "")
> --
> 2.16.2
>
>
>

Re: [gentoo-dev] glibc 2.16/19 for Gentoo Prefix on antique kernels

[Benda Xu]

Hi Michał,

Michał Górny  writes:

> I don't think this is the first old version Prefix team needs keeping.
> Another example are old versions of LLVM.

I am sure you are aware that Prefix has two variants: one is
prefix-rpath targeting MacOS, Solaris, AIX, Cygwin, Interix and a subset
of GNU/Linux; the other is prefix-standalone, targeting GNU/Linux and
Android/Linux.[1]

For LLVM example, it is prefix-rpath, which hosts its own overlay at
repo/proj/prefix.git.  Besides LLVM there are other hacks at present in
the overlay.  But we still keep the ultimate goal of merging prefix.git
into gentoo.git.

What we are discussing in this thread, however, is prefix-standalone, it
uses the official gentoo repository without any overlay.  It works
perfectly for kernel-2.6.26+ and linux-3.2+.  So, creating an overlay of
2 ebuilds for prefix-standalone is an overkill.

Yours,
Benda

1. https://wiki.gentoo.org/wiki/Project:Prefix

Re: [gentoo-dev] glibc 2.16/19 for Gentoo Prefix on antique kernels

[Michał Górny]

W dniu nie, 25.02.2018 o godzinie 18∶25 +0900, użytkownik Benda Xu
napisał:
> Hi Michał,
> 
> Michał Górny  writes:
> 
> > > So I would like to hear what you guys think if I:
> > > 
> > >   - keep glibc-2.19 and glibc-2.16 in tree and unmasking them in the
> > > selected Prefix profiles;
> > >  
> > >   - maintain those selected outdated glibc versions on the
> > > infrastructure of the Toolchain Project[5];
> > > 
> > >   - (optional) add an exception to the toolchain support policy[6].
> > 
> > How about moving them to an overlay?
> 
> I have reflected on this option and concluded it is an overkill to
> create an overlay for only 1 package.
> 

I don't think this is the first old version Prefix team needs keeping.
Another example are old versions of LLVM.

-- 
Best regards,
Michał Górny

Re: [gentoo-dev] glibc 2.16/19 for Gentoo Prefix on antique kernels

[Benda Xu]

Hi Michał,

Michał Górny  writes:

>> So I would like to hear what you guys think if I:
>> 
>>   - keep glibc-2.19 and glibc-2.16 in tree and unmasking them in the
>> selected Prefix profiles;
>>  
>>   - maintain those selected outdated glibc versions on the
>> infrastructure of the Toolchain Project[5];
>> 
>>   - (optional) add an exception to the toolchain support policy[6].
>
> How about moving them to an overlay?

I have reflected on this option and concluded it is an overkill to
create an overlay for only 1 package.

Yours,
Benda

Re: [gentoo-dev] glibc 2.16/19 for Gentoo Prefix on antique kernels

[Michał Górny]

W dniu nie, 25.02.2018 o godzinie 15∶17 +0900, użytkownik Benda Xu
napisał:
> Hi all,
> 
> Yes, it's 2018.  But there are still RHEL 4 and 5 systems running
> antique kernels such as 2.6.8 and 2.6.18.  In my experience, many of
> them are data acquisition hubs or computing clusters.  No administrator
> cares about security as long as they "work".
> 
> Under the form "Prefix", Gentoo is set out to rescue users trapped in
> these abandoned wastelands of antiques.  After years of work, we have
> achieved that goal, except one minor thing: glibc periodically drop
> support for old linux kernels, the lastest glibc supporting linux 2.6.8
> is 2.16 and for linux-2.6.18 it is glibc-2.19.
> 
> With the recent reunion of the Toolchain Project, old glibc versions are
> masked and removed, accelerating the adoption of new versions.  This
> opens a new oppotunity for the Prefix: people stops caring about
> unsupported glibc versions, the Prefix Project can take them over
> without worrying about breaking other peoples' machines.
> 
> Now, profiles/default/linux/amd64/17.0/no-multilib/prefix/kernel-2.6.16+
> unmasks  /lib/gentoo/functions.sh transition.  prefix/kernel-2.6+ with glibc-2.16
> is also planned.  In addition, glibc have to be patched to get python3
> built[1-3], which is urgent once portage drops python2[4].
> 
> 
> So I would like to hear what you guys think if I:
> 
>   - keep glibc-2.19 and glibc-2.16 in tree and unmasking them in the
> selected Prefix profiles;
>  
>   - maintain those selected outdated glibc versions on the
> infrastructure of the Toolchain Project[5];
> 
>   - (optional) add an exception to the toolchain support policy[6].


How about moving them to an overlay?

> 
> Thanks and cheers!
> Benda
> 
> 1. https://bugs.python.org/issue28092
> 2. https://bugs.python.org/issue31255
> 3. https://bugs.python.org/issue29157
> 4. 
> https://archives.gentoo.org/gentoo-project/message/7eb61502d827476a9326b0f180dbd2fa
> 5. https://wiki.gentoo.org/wiki/Project:Toolchain/Patchsets_with_Git
> 6. https://wiki.gentoo.org/wiki/Project:Toolchain/Support_policies
> 

-- 
Best regards,
Michał Górny

[gentoo-dev] [PATCH] cmake-utils.eclass: Extend ASM rules to ASM-ATT

[Michał Górny]

Some CMake projects use ASM-ATT rather than ASM, so extend our rule
overrides to that.

Bug: https://bugs.gentoo.org/625844
---
 eclass/cmake-utils.eclass | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/eclass/cmake-utils.eclass b/eclass/cmake-utils.eclass
index b9f69a824b14..ef3f3c2607f8 100644
--- a/eclass/cmake-utils.eclass
+++ b/eclass/cmake-utils.eclass
@@ -516,6 +516,7 @@ cmake-utils_src_configure() {
fi
cat > "${build_rules}" <<- _EOF_ || die
SET (CMAKE_ASM_COMPILE_OBJECT "  
${includes} ${CPPFLAGS}  -o  -c " CACHE STRING "ASM 
compile command" FORCE)
+   SET (CMAKE_ASM-ATT_COMPILE_OBJECT " 
 ${includes} ${CPPFLAGS}  -o  -c " CACHE STRING 
"ASM compile command" FORCE)
SET (CMAKE_C_COMPILE_OBJECT "  
${includes} ${CPPFLAGS}  -o  -c " CACHE STRING "C 
compile command" FORCE)
SET (CMAKE_CXX_COMPILE_OBJECT "  
${includes} ${CPPFLAGS}  -o  -c " CACHE STRING "C++ 
compile command" FORCE)
SET (CMAKE_Fortran_COMPILE_OBJECT " 
 ${includes} ${FCFLAGS}  -o  -c " CACHE STRING 
"Fortran compile command" FORCE)
@@ -531,6 +532,7 @@ cmake-utils_src_configure() {
local toolchain_file=${BUILD_DIR}/gentoo_toolchain.cmake
cat > ${toolchain_file} <<- _EOF_ || die
SET (CMAKE_ASM_COMPILER "${myCC/ /;}")
+   SET (CMAKE_ASM-ATT_COMPILER "${myCC/ /;}")
SET (CMAKE_C_COMPILER "${myCC/ /;}")
SET (CMAKE_CXX_COMPILER "${myCXX/ /;}")
SET (CMAKE_Fortran_COMPILER "${myFC/ /;}")
@@ -609,6 +611,7 @@ cmake-utils_src_configure() {
if [[ ${CMAKE_BUILD_TYPE} != Gentoo && ${EAPI} != 5 ]]; then
cat >> ${common_config} <<- _EOF_ || die
SET (CMAKE_ASM_FLAGS_${CMAKE_BUILD_TYPE^^} "" CACHE 
STRING "")
+   SET (CMAKE_ASM-ATT_FLAGS_${CMAKE_BUILD_TYPE^^} "" CACHE 
STRING "")
SET (CMAKE_C_FLAGS_${CMAKE_BUILD_TYPE^^} "" CACHE 
STRING "")
SET (CMAKE_CXX_FLAGS_${CMAKE_BUILD_TYPE^^} "" CACHE 
STRING "")
SET (CMAKE_Fortran_FLAGS_${CMAKE_BUILD_TYPE^^} "" CACHE 
STRING "")
-- 
2.16.2