Re: [gentoo-dev] Review: Apache AddHandler news item
Published a slightly improved version now: https://gitweb.gentoo.org/proj/gentoo-news.git/tree/2015/2015-04-06-apache-addhandler-addtype If there's anything wrong with it, please mail me directly (or put me in CC) so there is zero chance of slipping through. Thanks! Best, Sebastian
[gentoo-dev] [warning] the bug queue has 81 bugs
Our bug queue has 81 bugs! If you have some spare time, please help assign/sort a few bugs. To view the bug queue, click here: http://bit.ly/m8PQS5 Thanks!
[gentoo-dev] Re: New website: Where the .... is the ....ing documentation link?
Daniel Campbell posted on Sat, 04 Apr 2015 22:58:40 -0700 as excerpted: Aside from the Devmanual, most of Gentoo's docs are on the wiki, so you could make a bookmark and setup a search shortcut for it. Just bookmark the following address in Firefox: https://wiki.gentoo.org/index.php?search=%s Give it a keyword like 'gdocs' or something, then you can do gdocs Alsa and it'll search the wiki for ALSA. I'm not sure how it's done in Opera or Chrom[e|ium]. Thanks very much! =:^) I use kde's krunner for such searches, but setting that up is very similar, once you have a working URL to substitute the search into. You provided just that, which I hadn't thought of, so =:^) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
Re: [gentoo-portage-dev] [PATCH] fetch(): fix support for digest size=None
Dnia 2015-04-05, o godz. 10:57:57 Zac Medico zmed...@gentoo.org napisał(a): On 04/05/2015 12:59 AM, Michał Górny wrote: It seems that the code initially supported fetching without size 'digest' but it got broken over time. Add proper conditionals to avoid ugly failures in this case. If we're handling this case, then we should probably not assume that the dict contains a size key. So, we could check that dict.get(size) is not None. I agree. Could you take it from this point? I don't really have time to do this, so I limit myself to solving the bugs I need to get moving :/. -- Best regards, Michał Górny pgppLsAlc0znd.pgp Description: OpenPGP digital signature
[gentoo-portage-dev] [PATCH] ro_checker: skip parents of EPREFIX dir (bug 544624)
The ro_checker code added in commit
47ef9a0969474f963dc8e52bfbbb8bc075e8d73c incorrectly asserts that the
parent directories of EPREFIX be writable. Fix it to skip the parents,
since they are irrelevant. This does not affect the case where EPREFIX
is empty, since all directories are checked in that case.
Fixes 47ef9a096947: (Test for read-only filesystems, fixes bug 378869)
X-Gentoo-Bug: 544624
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=544624
---
pym/portage/dbapi/vartree.py | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/pym/portage/dbapi/vartree.py b/pym/portage/dbapi/vartree.py
index 277c2f1..1c0deab 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -3751,6 +3751,7 @@ class dblink(object):
line_ending_re = re.compile('[\n\r]')
srcroot_len = len(srcroot)
ed_len = len(self.settings[ED])
+ eprefix_len = len(self.settings[EPREFIX])
while True:
@@ -3792,7 +3793,11 @@ class dblink(object):
break
relative_path = parent[srcroot_len:]
- dirlist.append(os.path.join(destroot,
relative_path))
+ if len(relative_path) = eprefix_len:
+ # Files are never installed outside of
the prefix,
+ # therefore we skip the readonly
filesystem check for
+ # parent directories of the prefix (see
bug 544624).
+ dirlist.append(os.path.join(destroot,
relative_path))
for fname in files:
try:
--
2.3.1
Re: [gentoo-portage-dev] [PATCH] ro_checker: skip parents of EPREFIX dir (bug 544624)
On Sun, 5 Apr 2015 21:28:07 -0700
Zac Medico zmed...@gentoo.org wrote:
The ro_checker code added in commit
47ef9a0969474f963dc8e52bfbbb8bc075e8d73c incorrectly asserts that the
parent directories of EPREFIX be writable. Fix it to skip the parents,
since they are irrelevant. This does not affect the case where EPREFIX
is empty, since all directories are checked in that case.
Fixes 47ef9a096947: (Test for read-only filesystems, fixes bug
378869) X-Gentoo-Bug: 544624
X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=544624
---
pym/portage/dbapi/vartree.py | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/pym/portage/dbapi/vartree.py
b/pym/portage/dbapi/vartree.py index 277c2f1..1c0deab 100644
--- a/pym/portage/dbapi/vartree.py
+++ b/pym/portage/dbapi/vartree.py
@@ -3751,6 +3751,7 @@ class dblink(object):
line_ending_re = re.compile('[\n\r]')
srcroot_len = len(srcroot)
ed_len = len(self.settings[ED])
+ eprefix_len = len(self.settings[EPREFIX])
while True:
@@ -3792,7 +3793,11 @@ class dblink(object):
break
relative_path = parent[srcroot_len:]
-
dirlist.append(os.path.join(destroot, relative_path))
+ if len(relative_path) = eprefix_len:
+ # Files are never installed
outside of the prefix,
+ # therefore we skip the
readonly filesystem check for
+ # parent directories of the
prefix (see bug 544624).
+
dirlist.append(os.path.join(destroot, relative_path))
for fname in files:
try:
LGTM
--
Brian Dolbec dolsen
Re: [gentoo-portage-dev] [PATCH v2] fetch(): fix support for digest size=None
On Sun, 5 Apr 2015 21:47:12 -0700 Zac Medico zmed...@gentoo.org wrote: From: Michał Górny mgo...@gentoo.org It seems that the code initially supported fetching without size 'digest' but it got broken over time. Add proper conditionals to avoid ugly failures in this case. Signed-off-by: Zac Medico zmed...@gentoo.org --- [PATCH v2] uses dict.get() so that the size key is not required https://github.com/zmedico/portage/tree/mgorny-fetch-size-missing pym/portage/checksum.py | 2 +- pym/portage/package/ebuild/fetch.py | 7 --- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index f24a90f..642602e 100644 --- a/pym/portage/checksum.py +++ b/pym/portage/checksum.py @@ -303,7 +303,7 @@ def verify_all(filename, mydict, calc_prelink=0, strict=0): reason = Reason unknown try: mysize = os.stat(filename)[stat.ST_SIZE] - if mydict[size] != mysize: + if mydict.get(size) is not None and mydict[size] != mysize: return False,(_(Filesize does not match recorded size), mysize, mydict[size]) except OSError as e: if e.errno == errno.ENOENT: diff --git a/pym/portage/package/ebuild/fetch.py b/pym/portage/package/ebuild/fetch.py index 7b856a2..7e4e6fe 100644 --- a/pym/portage/package/ebuild/fetch.py +++ b/pym/portage/package/ebuild/fetch.py @@ -700,7 +700,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0, os.unlink(myfile_path) except OSError: pass - elif distdir_writable: + elif distdir_writable and size is not None: if mystat.st_size fetch_resume_size and \ mystat.st_size size: # If the file already exists and the size does not @@ -806,8 +806,9 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0, # assume that it is fully downloaded. continue else: - if mystat.st_size mydigests[myfile][size] and \ - not restrict_fetch: + if (mydigests[myfile].get(size) is not None + and mystat.st_size mydigests[myfile][size] + and not restrict_fetch): fetched = 1 # Try to resume this download. elif parallel_fetchonly and \ mystat.st_size == mydigests[myfile][size]: OK :) merge please -- Brian Dolbec dolsen
[gentoo-dev] Automated Package Removal and Addition Tracker, for the week ending 2015-04-05 23:59 UTC
The attached list notes all of the packages that were added or removed from the tree, for the week ending 2015-04-05 23:59 UTC. Removals: media-plugins/vdr-pcd 2015-03-30 13:33:19 hd_brummy app-admin/eselect-audicle 2015-03-31 20:35:45 ulm app-admin/eselect-awk 2015-03-31 20:35:45 ulm app-admin/eselect-bashcomp 2015-03-31 20:35:45 ulm app-admin/eselect-blas 2015-03-31 20:35:45 ulm app-admin/eselect-cblas 2015-03-31 20:35:45 ulm app-admin/eselect-cdparanoia2015-03-31 20:35:45 ulm app-admin/eselect-chuck 2015-03-31 20:35:46 ulm app-admin/eselect-ctags 2015-03-31 20:35:46 ulm app-admin/eselect-ecj 2015-03-31 20:35:46 ulm app-admin/eselect-emacs 2015-03-31 20:35:46 ulm app-admin/eselect-esd 2015-03-31 20:35:46 ulm app-admin/eselect-fontconfig2015-03-31 20:35:46 ulm app-admin/eselect-gnat 2015-03-31 20:35:46 ulm app-admin/eselect-gnome-shell-extensions2015-03-31 20:35:46 ulm app-admin/eselect-infinality2015-03-31 20:35:46 ulm app-admin/eselect-java 2015-03-31 20:35:46 ulm app-admin/eselect-lapack2015-03-31 20:35:46 ulm app-admin/eselect-lcdfilter 2015-03-31 20:35:46 ulm app-admin/eselect-lib-bin-symlink 2015-03-31 20:35:46 ulm app-admin/eselect-lua 2015-03-31 20:35:47 ulm app-admin/eselect-maven 2015-03-31 20:35:47 ulm app-admin/eselect-mesa 2015-03-31 20:35:47 ulm app-admin/eselect-metasploit2015-03-31 20:35:47 ulm app-admin/eselect-miniaudicle 2015-03-31 20:35:47 ulm app-admin/eselect-mpg1232015-03-31 20:35:47 ulm app-admin/eselect-mpost 2015-03-31 20:35:47 ulm app-admin/eselect-notify-send 2015-03-31 20:35:47 ulm app-admin/eselect-oodict2015-03-31 20:35:47 ulm app-admin/eselect-opencascade 2015-03-31 20:35:47 ulm app-admin/eselect-opencl2015-03-31 20:35:48 ulm app-admin/eselect-opengl2015-03-31 20:35:48 ulm app-admin/eselect-package-manager 2015-03-31 20:35:48 ulm app-admin/eselect-pdftex2015-03-31 20:35:48 ulm app-admin/eselect-php 2015-03-31 20:35:48 ulm app-admin/eselect-pinentry 2015-03-31 20:35:48 ulm app-admin/eselect-postgresql2015-03-31 20:35:48 ulm app-admin/eselect-python2015-03-31 20:35:48 ulm app-admin/eselect-qtgraphicssystem 2015-03-31 20:35:48 ulm app-admin/eselect-rails 2015-03-31 20:35:48 ulm app-admin/eselect-renpy 2015-03-31 20:35:48 ulm app-admin/eselect-ruby 2015-03-31 20:35:48 ulm app-admin/eselect-rust 2015-03-31 20:35:49 ulm app-admin/eselect-sh2015-03-31 20:35:49 ulm app-admin/eselect-sndpeek 2015-03-31 20:35:49 ulm app-admin/eselect-timezone 2015-03-31 20:35:49 ulm app-admin/eselect-timidity 2015-03-31 20:35:49 ulm app-admin/eselect-unison2015-03-31 20:35:49 ulm app-admin/eselect-vdr 2015-03-31 20:35:49 ulm app-admin/eselect-vi2015-03-31 20:35:49 ulm app-admin/eselect-wxwidgets 2015-03-31 20:35:49 ulm app-admin/eselect-xvmc 2015-03-31 20:35:49 ulm dev-perl/config-general 2015-04-01 22:27:39 dilfridge dev-util/pkgcore-checks 2015-04-02 03:15:23 radhermit dev-python/rax-backup-schedule-python-novaclient-ext2015-04-03 01:36:03 prometheanfire dev-ruby/gemoji 2015-04-03 07:15:17 graaff sci-libs/openmm 2015-04-03
[gentoo-dev] Re: Review: Apache AddHandler news item
Sebastian Pipping posted on Mon, 06 Apr 2015 01:29:19 +0200 as excerpted: Published a slightly improved version now: https://gitweb.gentoo.org/proj/gentoo-news.git/tree/2015/2015-04-06- apache-addhandler-addtype If there's anything wrong with it, please mail me directly (or put me in CC) so there is zero chance of slipping through. Thanks! [also mailing sp...@gentoo.org as requested] $ echo Apache AddHandler/AddType vulnerability protection | wc -c 51 GLEP 42 says max title length 44 chars. 51-44=7 chars too long. Off the top of my head, maybe just s/vulnerability/vuln/ ? That'd cut 9 chars for 42, leaving two to spare. Anyone with a better idea? That's the big one. Here's a couple more minor English usage change suggestions as well. (Changes denoted in caps here, obviously lowercase them): Line 25, add also: may be helpful. Unfortunately, it can ALSO be a security threat. Line 74 s/at/in/: You may be using AddHandler or AddType IN other places, -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
[gentoo-dev] Re: New website: Where the .... is the ....ing documentation link?
Alec Warner posted on Sun, 05 Apr 2015 09:33:24 -0700 as excerpted: No need to shout. Apology to you and others. And thanks. While I am of course familiar with CAPS=shout, I always considered it entire phrases or whole sentences in caps, not single words, which was simply emphasis. But it seems others consider even single caps-words shouting, so from now on I'll try to use either *bold* or /italics/ emphasis instead. Also, as I was reminded, a bug would have been more appropriate. If events don't make it unnecessary, I'll try to file one later this week. Thanks again. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
[gentoo-dev] Last rites: sys-fs/evms
# Maciej Mrozowski reave...@gentoo.org (06 Apr 2015) # Does not link against latest ncurses and no interest to fix it. # Superseded by LVM2 and all clusters should be migrated by now. # Removal in 30 days. sys-fs/evms I kept it in tree for a while longer for volumes migration purpose, but all good things must come to an end eventually. regards MM
Re: [gentoo-portage-dev] [PATCH] repoman: change metadata.dtd protocol to https
On Sat, 4 Apr 2015 23:31:25 -0700 Zac Medico zmed...@gentoo.org wrote: The old http uri redirects to the new https uri, so use the new https uri directly. X-Gentoo-Bug: 469888 X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=469888 --- bin/repoman | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/bin/repoman b/bin/repoman index 7101a00..6074c91 100755 --- a/bin/repoman +++ b/bin/repoman @@ -503,7 +503,7 @@ metadata_xml_encoding = 'UTF-8' metadata_xml_declaration = '?xml version=1.0 encoding=%s?' % \ (metadata_xml_encoding,) metadata_doctype_name = 'pkgmetadata' -metadata_dtd_uri = 'http://www.gentoo.org/dtd/metadata.dtd' +metadata_dtd_uri = 'https://www.gentoo.org/dtd/metadata.dtd' # force refetch if the local copy creation time is older than this metadata_dtd_ctime_interval = 60 * 60 * 24 * 7 # 7 days Hmm, I thought that was being moved to api.gentoo.org. I just checked and it isn't there... I want to verify that it isn't destined there in the very near future. -- Brian Dolbec dolsen
Re: [gentoo-dev] Re: RFC: News item for net-firewall/shorewall all-in-one package migration
On Sun, 5 Apr 2015 04:12:17 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: Thomas D. posted on Sat, 04 Apr 2015 22:09:36 +0200 as excerpted: Title: New net-firewall/shorewall all-in-one package But I'm not wedded to either idea; they're just what I came up with off the top of my head. If someone has a better idea... (FWIW I had a /terrible/ time finding that glep on the new website to double-check, and it's good I did as I thought it was 42, but that's a gripe for a new thread...) It seems these keen folk ought be given support for a reasonable idea with user support / desire and presented with due attention to glep requirements. -- kind regards Ian Delaney
Re: [gentoo-dev] libressl status
On 04/05/2015 01:17 PM, Rich Freeman wrote: If you're going to fork a library, and don't intend to keep the packages API-compatible, then change the filenames. What is so hard about this? LIbressl was even an outside fork, so it didn't come with any of the baggage of we're the real libssl team or whatever. Libressl is (widely) API compatible with openssl. As said numerous times before: they broke the API when they thought it is security relevant. This is about the fact that they _added_ features which are not present in openssl. However, openntpd still compiles with openssl. Sure, we can do the USE=libressl route like we did with libav, but since this is still new would it make more sense to just rename the libressl files so that they can still go in /usr/lib but without being called libssl? Then any package that wants to use them will need to have their build logic changed accordingly. They aren't drop-in replacements for each other anyway, as much as people would wish they were, so we should resist the urge to pretend that they are. By that you are effectively forking libressl and causing a huge mess downstream for both developers and users. It may even cause cross-distro breakage. I can name several examples of this happening due to debian going it's own way. Last of which affected openclonk (upstream merged a patch from a debian dev who expected any distro does the same hacks they do). So please, have a little sense for QA. Those ideas are just making it worse. This is something that has to be resolved upstream. If they don't cooperate long-term, then their fork will just die out for sure (and for good). However, I currently don't see strong signs for that.
Re: [gentoo-dev] libressl status
On 5 April 2015 at 05:44, Paul B. Henson hen...@acm.org wrote: I guess I'll just let this simmer for now and see how things develop. My preference (I think, at least at the moment) would be for both implementations to be able to coexist, like openssl and gnutls. It looks like that's the way it's heading in pkgsrc (the other place I'm maintaining openntpd), which should make things relatively simple there. If that's not going to be an option with Gentoo hopefully the best alternative will become clearer at some point ;). The problem with that is that now you have to make sure that transitive dependencies are still functional. Since as you point out the two packages are vastly API compatible, it makes them ABI incompatible and conflicting. The functions can have the same name, and vastly the same parameters, but they may be using different size for data, for instance. I pointed this out last year[1][2] already. Symbol collision is a nasty problem because it's almost invisible as long as the API/ABI is close enough, but for libraries like OpenSSL/LibreSSL, this is a huge security risk, too. [1] https://blog.flameeyes.eu/2014/07/libressl-drop-in-and-abi-leakage [2] https://blog.flameeyes.eu/2014/07/libressl-and-the-bundled-libs-hurdle Diego Elio Pettenò — Flameeyes https://blog.flameeyes.eu/
Re: [gentoo-dev] libressl status
On Sun, Apr 5, 2015 at 8:23 AM, Diego Elio Pettenò flamee...@flameeyes.eu wrote: Since as you point out the two packages are vastly API compatible, it makes them ABI incompatible and conflicting. ++ If they really want to improve the security of function calls that they consider inherently secure, they should just introduce new functions and deprecate the old ones, or make old ones wrappers around new ones if that is appropriate. If they go with the deprecation route then they need to avoid using the same SONAMEs, and if they go the wrapper route they need to make sure that they're compatible across SONAMEs. Of course, any re-implementation could have bugs, but the key is that upstream has to recognize these incompatibilities as being valid bugs that they intend to fix. Right now if something designed to link against openssl breaks against libressl there is a good chance that libressl will just say it is intentional, which then leaves us in the position of having to deal with the mess. I agree that renaming things isn't a great solution either, and that this really needs to be fixed upstream. However, I don't really like it going into the tree the way it is, because sooner or later we're going to end up with blockers or bugs. Either you can't install foo with barssl, or you can, and it just might break but nobody really keeps track of that. It really doesn't matter which is the better implementation - this is just a really messy way to do a transition. -- Rich
Re: [gentoo-dev] Re: RFC: extending pkgmove (slotmove) actions to apply transitionally to ebuilds
On Sat, 4 Apr 2015, Michał Górny wrote: Dnia 2015-04-04, o godz. 21:36:37 Ulrich Mueller u...@gentoo.org napisał(a): But in the worst case, your hack can cause a broken dependency graph. On the one hand, above mentioned =app-misc/foo-1:0 matches all versions affected by the slotmove, so it should be converted. On the other hand, it is a perfectly valid dependency specification which could have been added after the slotmove, in which case it shouldn't be converted. You cannot know here what the intentions of the developer are. But it *will* be converted in vdb the next time updates are applied. IIUC, dependencies in the VDB will only be converted if the target of the slotmove is installed on the user's system? Ulrich pgp3baZdiFvQ1.pgp Description: PGP signature
[gentoo-dev] Last rites: app-emulation/emul-linux-x86-java, dev-java/sun-j2me-bin
# James Le Cuirot ch...@gentoo.org (05 Apr 2015) # A 32-on-64 Java VM is no longer considered necessary and a multilib # icedtea-bin would be preferred anyway. sun-j2me-bin depends on this # but it has been superseded by Oracle's JME and I doubt we'll ever # need that either. Removal in 30 days. app-emulation/emul-linux-x86-java dev-java/sun-j2me-bin -- James Le Cuirot (chewi) Gentoo Linux Developer pgpXyvIvi7jLy.pgp Description: OpenPGP digital signature
[gentoo-portage-dev] [PATCH] fetch(): fix support for digest size=None
It seems that the code initially supported fetching without size 'digest' but it got broken over time. Add proper conditionals to avoid ugly failures in this case. --- pym/portage/checksum.py | 2 +- pym/portage/package/ebuild/fetch.py | 7 --- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/pym/portage/checksum.py b/pym/portage/checksum.py index f24a90f..1efe74e 100644 --- a/pym/portage/checksum.py +++ b/pym/portage/checksum.py @@ -303,7 +303,7 @@ def verify_all(filename, mydict, calc_prelink=0, strict=0): reason = Reason unknown try: mysize = os.stat(filename)[stat.ST_SIZE] - if mydict[size] != mysize: + if mydict[size] is not None and mydict[size] != mysize: return False,(_(Filesize does not match recorded size), mysize, mydict[size]) except OSError as e: if e.errno == errno.ENOENT: diff --git a/pym/portage/package/ebuild/fetch.py b/pym/portage/package/ebuild/fetch.py index 7b856a2..f60db9e 100644 --- a/pym/portage/package/ebuild/fetch.py +++ b/pym/portage/package/ebuild/fetch.py @@ -700,7 +700,7 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0, os.unlink(myfile_path) except OSError: pass - elif distdir_writable: + elif distdir_writable and size is not None: if mystat.st_size fetch_resume_size and \ mystat.st_size size: # If the file already exists and the size does not @@ -806,8 +806,9 @@ def fetch(myuris, mysettings, listonly=0, fetchonly=0, # assume that it is fully downloaded. continue else: - if mystat.st_size mydigests[myfile][size] and \ - not restrict_fetch: + if (mydigests[myfile][size] is not None + and mystat.st_size mydigests[myfile][size] + and not restrict_fetch): fetched = 1 # Try to resume this download. elif parallel_fetchonly and \ mystat.st_size == mydigests[myfile][size]: -- 2.3.5
[gentoo-portage-dev] [PATCH] Contribute squashdelta syncing module
The squashdelta module provides syncing via SquashFS snapshots. For the
initial sync, a complete snapshot is fetched and placed in
/var/cache/portage/squashfs. On subsequent sync operations, deltas are
fetched from the mirror and used to reconstruct the newest snapshot.
The distfile fetching logic is reused to fetch the remote files
and verify their checksums. Additionally, the sha512sum.txt file should
be OpenPGP-verified after fetching but this is currently unimplemented.
After fetching, Portage tries to (re-)mount the SquashFS in repository
location.
---
cnf/repos.conf | 4 +
pym/portage/sync/modules/squashdelta/README| 124 +
pym/portage/sync/modules/squashdelta/__init__.py | 37
.../sync/modules/squashdelta/squashdelta.py| 192 +
4 files changed, 357 insertions(+)
create mode 100644 pym/portage/sync/modules/squashdelta/README
create mode 100644 pym/portage/sync/modules/squashdelta/__init__.py
create mode 100644 pym/portage/sync/modules/squashdelta/squashdelta.py
diff --git a/cnf/repos.conf b/cnf/repos.conf
index 1ca98ca..062fc0d 100644
--- a/cnf/repos.conf
+++ b/cnf/repos.conf
@@ -6,3 +6,7 @@ location = /usr/portage
sync-type = rsync
sync-uri = rsync://rsync.gentoo.org/gentoo-portage
auto-sync = yes
+
+# for daily squashfs snapshots
+#sync-type = squashdelta
+#sync-uri = mirror://gentoo/../snapshots/squashfs
diff --git a/pym/portage/sync/modules/squashdelta/README
b/pym/portage/sync/modules/squashdelta/README
new file mode 100644
index 000..994ae6d
--- /dev/null
+++ b/pym/portage/sync/modules/squashdelta/README
@@ -0,0 +1,124 @@
+==
+ squashdelta-sync
+==
+
+Introduction
+
+
+Squashdelta-sync provides the squashfs syncing module for Portage.
+When used as sync-type for the repository, it fetches the complete
+repository snapshot on initial sync, and then uses squashdeltas to
+efficiently update it.
+
+While initially intended for the daily snapshot of the Gentoo
+repository, the module is designed with flexibility in mind. It can be
+used to sync any repository, without enforcing any specific snapshotting
+interval or versioning rules. However, each snapshot version identifier
+must be unique in the scope of repository.
+
+
+Technical hosting details
+=
+
+The snapshot hosting needs to provide the following files:
+
+1. the current (newest) full SquashFS snapshot of the repository,
+ and optionally M past snapshots,
+
+2. the deltas from N past snapshots to the current snapshot,
+
+3. a ``sha512sum.txt`` file containing SHA-512 checksums of all hosted
+ files, optionally OpenPGP-signed.
+
+The following naming schemes are used for the snapshots and deltas,
+respectively::
+
+${repo_name}-${version}.sqfs
+${repo_name}-${old_version}-${new_version}.sqdelta
+
+where:
+
+* ``${repo_name}`` is the repository name (as specified
+ in ``repos.conf``),
+* ``${version}`` specifies the snapshot version,
+* ``${old_version}`` specifies the snapshot version which the delta
+ updates from,
+* ``${new_version}`` specifies the snapshot version which the delta
+ updates to.
+
+Version can be an arbitrary string. It does not need to be incremental,
+however each version must be unique in the repository scope.
+For example, the version can be a date, a revision number or a commit
+hash.
+
+The ``sha512sum.txt`` uses the format used by the GNU coreutils
+``sha512sum`` program. That is, it contains one or more lines consisting
+of hexadecimal SHA-512 checksum followed by whitespace, followed by
+a filename. Lines not matching that format should be ignored.
+
+Optionally, the ``sha512sum.txt`` may be OpenPGP-signed. In that case,
+the file conforms to the ASCII-armored OpenPGP message format, with
+the checksums being stored in the message body.
+
+Additionally, the ``sha512sum.txt`` needs to contain an additional line
+containing the following string::
+
+Current: ${repo_name}-${version}
+
+Stating the current (newest) snapshot version. If snapshots for multiple
+repositories are provided in the same directory (using the same
+``sha512sum.txt`` file), this line can occur multiple times or list
+multiple snapshots, whitespace-separated. In order not to introduce
+stray lines in the file, it is recommended to embed this information
+in the OpenPGP comment field.
+
+An example script generating daily deltas for a repository can be found
+in squashdelta-daily-gen_ repository.
+
+.. _squashdelta-daily-gen: https://bitbucket.org/mgorny/squashdelta-daily-gen
+
+
+Technical syncing details
+=
+
+When performing a sync, the script first fetches the ``sha512sum.txt``
+and processes it in order to determine the list of files available
+on the mirror. It should be noted that the script will never use
+a snapshot or delta that is not listed there. If the file is
+OpenPGP-signed, the signature is verified.
+
+The
Re: [gentoo-dev] libressl status
On 04/05/2015 06:44 AM, Paul B. Henson wrote: On Fri, Apr 03, 2015 at 01:31:53PM +0200, hasufell wrote: Not anymore. We will go for libressl USE flag for the same reason there is a libav USE flag now (working subslots etc). Um, ok. That still only allows one or the other to be installed though, right? So if you want a package that only works with openssl and one that only works with libressl, you are left wanting :)? Yep. Your only solution is to try something really hackish like NixOS. They would allow any such combination. But it comes with a price. Or you do it manually outside of the PM.
Re: [gentoo-dev] libressl status
On Sun, Apr 5, 2015 at 12:34 AM, Paul B. Henson hen...@acm.org wrote: They're pretty much decided on allowing both openssl and libressl to be installed concurrently and for a given application to use one or the other. The specific method for that packaging system is what they call a prefix; basically instead of /usr/pkg/lib/libssl it would be /usr/pkg/libressl/lib/libssl, and packages that needed it would get the right magic flags for the headers and libraries to be found. WTF. If you're going to fork a library, and don't intend to keep the packages API-compatible, then change the filenames. What is so hard about this? LIbressl was even an outside fork, so it didn't come with any of the baggage of we're the real libssl team or whatever. Sure, we can do the USE=libressl route like we did with libav, but since this is still new would it make more sense to just rename the libressl files so that they can still go in /usr/lib but without being called libssl? Then any package that wants to use them will need to have their build logic changed accordingly. They aren't drop-in replacements for each other anyway, as much as people would wish they were, so we should resist the urge to pretend that they are. -- Rich
Re: [gentoo-dev] shouldn't eselect be in app-eselect ?
On Sat, Apr 04, 2015 at 08:49:51PM +0200, Michał Górny wrote: Dnia 2015-04-04, o godz. 13:47:47 Alex Brandt alund...@gentoo.org napisał(a): On Saturday, April 04, 2015 14:41:37 Philip Webb wrote: I read the recent thread re the new app-eselect. Doing my weekly system update, it strikes me that 'eselect' itself sb there too. Time to paint the bikesheds again ... (smile) I don't disagree but will simply point out that if this becomes true, we should also move dev-lang/python to dev-python, dev-lang/ruby to dev-ruby, and dev-lang/perl to dev-perl (not an exhaustive list). Portage to app-portage/, web browsers with plugins to www-plugins/... also, dev-java/oracle-jdk-bin, dev-java/icedtea ... -- Best regards, Michał Górny
Re: [gentoo-dev] Importance of SLOTs on Java dependencies
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 My preferred solution would be create a revbump that solely amends (R)DEPEND, leaving the KEYWORDS exactly as they are. Sounds good to me (as long as repoman agrees :). - -- Andreas K. Huettel Gentoo Linux developer dilfri...@gentoo.org http://www.akhuettel.de/ -BEGIN PGP SIGNATURE- Version: GnuPG v2.0 iQJ8BAEBCgBmBQJVIWGTXxSAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ0RkJDMzI0NjNBOTIwMDY5MTQ2NkMzNDBF MTM4NkZEN0VGNEI1Nzc5AAoJEOE4b9fvS1d5Ht4P/RBt9rbsNkOkR03dzddjwSqK 1VJCF+YhWUXDatY1dwVtPJQ8bfv1H3qW1WB7XfhtOZqZzCqiDq7irS9aog6YgYcz NTWWZhYTG9PU9XV+CoFb+4jkVgHa8IuCrGfg0idNQd7QVktozJO7/LnJbu4PDzw3 exEHMuqVVj3iub0CZlBX0OJiii5G3k5CjuVw1GAKJ0j2Od55FtQnIMYJqJfJdZGj 7i0Aerr9glCwfhojxzcCy6hlJaMdSvlPPvZw/NCujETTZJbpM72TPzMs4xr7rXIJ T0zoNzKe05Mkmitm+Hw7OkY4Acfh94tVK3FzEUv1ZqFW7prEBjOmh3vSqsjHsuWf PQ0EApnIOI7bOIjJDwIbZaCtehr/727mDV4JLH/guFGD9+pvLHszGHDr5Ga+t/Mk TVmIWkevGVfsUcaVfLF7d2LCoVA0Nup5Mdudsj8yZ7fwAIL0DfceBM+Wdf3V5kE1 Vg0JDGlU3JYyfLf9lcuhVZcueYWOGY6n4UMbajtrUWyjsterktCRtpKM/1zmc+i/ cgnzsdzJde5JsSjrEU4RrPvezZKhJ0+LtcyI4d310OjGlDkK6pwFEe7NE9z4HqxX dBkhCsZDgVYAy6sT/LDD60Num2sC3BdBKNyk8gjIf5GoSFOLiINL/fmQQa87/pBM IHa+KkSrQHOB+GhB0VKF =tTas -END PGP SIGNATURE-
Re: [gentoo-dev] Re: RFC: making repoman complain about USE dependencies that lock packages at old version
Dnia 2015-04-05, o godz. 19:32:01 Ulrich Mueller u...@gentoo.org napisał(a): On Sun, 5 Apr 2015, Michał Górny wrote: This is quite a Portage patch topic but since devs are known to be unhappy about any change, I would like to start a bikeshed first. The idea is to make repoman/pcheck complain if the newest ebuild matched by version+slot restriction of dependency atom can't satisfy the USE dependency, or in other words, whenever the USE restriction forces lower version of package being installed. For example, let's assume the following: a. foo-1 has USE=bar, b. bar-1 depends on foo[bar], c. foo-2 no longer has USE=bar. In the usual scenario this means that bar-1 will silently lock foo at version 1. Since this is quite implicit, users don't notice the problem and left unnoticed for a long time, it starts to hurt bad at some point. Is foo[bar] even a legal dependency, if there are versions of foo that don't have the bar flag in their IUSE? (PMS doesn't seem to be entirely clear about this: ... it is an error for a use dependency to be applied to an ebuild which does not have the flag in question in IUSE_REFERENCEABLE.) So yes, make repoman shout about it. Loudly. :) Well, I would interpret the PMS here as: '[foo?]' restricts to foo-1 even with USE=-foo. Though Portage disagrees here, I guess :). With the added check, after step (c) above repoman/pcheck would warn on bar-1 that 'foo[bar]' dependency forces lower version than plain 'foo'. Since version and slot restrictions will be taken into consideration, it will possible to silence the warning by using: foo-2[bar] explicitly, if the dependency can't be fixed for foo-2 properly. While not improving the final result a lot, it will at least give an explicit '' or '=' atom that can be easily found with grep when considering old version removal. What are your thoughts? How are you going to treat foo[bar(-)] and foo[bar(+)]? Well, bar(-) restricts the possible version as well, so it desires a warning. As for bar(+), it's often the actual solution so no warning for that. -- Best regards, Michał Górny pgpxpg_buF2T3.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] RFC: making repoman complain about USE dependencies that lock packages at old version
On Sun, Apr 5, 2015 at 6:50 PM, Michał Górny mgo...@gentoo.org wrote: What are your thoughts? Sounds useful to me! Cheers, Dirkjan
Re: [gentoo-dev] libressl status
On Sun, Apr 5, 2015 at 2:35 PM, hasufell hasuf...@gentoo.org wrote: You are ranting at the wrong place. If you want to make a difference, take this to the openbsd mailing lists. It seems unlikely that this would make much of a difference. I think that allowing this package to create another ffmpeg vs libav mess is a mistake. -- Rich
Re: [gentoo-dev] libressl status
On 04/05/2015 03:25 PM, Rich Freeman wrote: On Sun, Apr 5, 2015 at 8:23 AM, Diego Elio Pettenò flamee...@flameeyes.eu wrote: Since as you point out the two packages are vastly API compatible, it makes them ABI incompatible and conflicting. ++ If they really want to improve the security of function calls that they consider inherently secure, they should just introduce new functions and deprecate the old ones, or make old ones wrappers around new ones if that is appropriate. If they go with the deprecation route then they need to avoid using the same SONAMEs, and if they go the wrapper route they need to make sure that they're compatible across SONAMEs. Of course, any re-implementation could have bugs, but the key is that upstream has to recognize these incompatibilities as being valid bugs that they intend to fix. Right now if something designed to link against openssl breaks against libressl there is a good chance that libressl will just say it is intentional, which then leaves us in the position of having to deal with the mess. I agree that renaming things isn't a great solution either, and that this really needs to be fixed upstream. However, I don't really like it going into the tree the way it is, because sooner or later we're going to end up with blockers or bugs. Either you can't install foo with barssl, or you can, and it just might break but nobody really keeps track of that. It really doesn't matter which is the better implementation - this is just a really messy way to do a transition. You are ranting at the wrong place. If you want to make a difference, take this to the openbsd mailing lists.
Re: [gentoo-portage-dev] Re: Dynamic USE dependencies
On Sun, Apr 5, 2015 at 11:47 AM, Martin Vaeth mar...@mvath.de wrote: One suggestion around this problem would be to use different directories for these two types of use-flags, say package.use and package.use.needed. I still think we need a better long-term solution, but the workaround is simpler than that. I try to keep files in directories for package.keywords and package.use. One starts with a z and is a dumping ground for auto-whatever. The other files are where I try to remember to store personal preferences. So, in theory at any time I can wipe the z-file and run emerge and watch it wrestle with re-creating it. However, I think a cleaner solution would be better... -- Rich
[gentoo-dev] Importance of SLOTs on Java dependencies
Hello all,
Firstly, I would like to take this opportunity to remind all devs
touching ebuilds with Java .jar dependencies about the importance of
restricting these dependencies to specific SLOTs.
There is no cross-platform or even platform-specific location for
Java .jar files so each distro tends to do its own thing. Gentoo's Java
eclasses install metadata about any .jar files in a file called
package.env either at /usr/share/${PN}-${SLOT} or just /usr/share/${PN}
when the SLOT is 0.
java-config is executed both at build time and at run time to read this
metadata so that it can build a classpath. If one of the dependencies
unexpectedly changes SLOT due to package updates then the chain breaks.
You must therefore *always* restrict the SLOT, even if that dependency
currently has a SLOT of 0.
Why do we SLOT Java stuff so much? Java applications tend to have many
third party dependencies so it is inevitable that we would sometimes
need to have more than one version of a library installed at one time.
The write once, run anywhere nature of Java also means that upstream
doesn't expect you to run against library versions other than the ones
they shipped their application with. We do have a tool to check for
compatibility between versions though to avoid SLOT proliferation
getting out of hand. Classpath conflicts involving multiple versions
have rarely been an issue up till now but we have thought of measures
to combat this in future if needs be.
I felt the need to write the above because I have seen many instances
where devs not familiar with Java packaging have made this mistake. Now
I need to ask what to do in the case of ebuilds that have already been
marked stable.
To bring up a real example, I would like to bump dev-java/jna with a
new SLOT for the new version. There are several reverse dependencies, 3
of which do not specify a SLOT, and 2 of these have already been marked
stable. Upon giving jna a new SLOT, all these packages would instantly
fail to build if jna:0 is not already installed and they would also
fail to run if jna:0 gets depcleaned. Simply leaving the stable ebuilds
as they are is therefore not an option. My preferred solution would be
create a revbump that solely amends (R)DEPEND, leaving the KEYWORDS
exactly as they are. This is controversial but what other choice is
there? I could delay the jna bump but this would push back this thread
of work by a month when I already have a huge backlog. Please do not let
bureaucracy get in the way here.
Of course I would certainly give any maintainers a heads up before
doing this. Unfortunately, in this particular case, I contacted miknix
about dev-embedded/arduino over 2 weeks ago and haven't heard anything
back. He hasn't been seen on IRC for over 5 months. :(
--
James Le Cuirot (chewi)
Gentoo Linux Developer
pgp0Pq1rUf7R8.pgp
Description: OpenPGP digital signature
Re: [gentoo-dev] New website: Where the .... is the ....ing documentation link?
On Sat, Apr 4, 2015 at 9:42 PM, Duncan 1i5t5.dun...@cox.net wrote: TL;DR: (1) Please consider either making Documentation a top-level-menu link (preferable), or at least change Support to Support and Documentation. (2) On the documentation lander page, add a one big list link to a browser-keyword-searchable list of docs, like the old docs project page had. File a bug, the bug tracker is at https://bugs.gentoo.org. I believe there is even a specific component for the website. Looking for the news glep to double-check the title header max char limit (for the shorewall news-item thread was the first time I've had to use the new website. I *HATE* it! It's a NIGHTMARE to navigate, compared to the old one! There's all this nice fuzzy buzzword bingo, but actually FINDING anything like DOCUMENTATION is an exercise in frustration! OK, no direct docs link. Well, getting started should have a nice documentation link, since you don't get far on gentoo without docs... No, not there... Well then, learn more CERTAINLY must lead to documentation allowing you to do just that!... BLECH!!! NO!!! Nothing there but more buzzword bingo (and a contact link and links to share more of my interests with privacy-insensitive corporations that already know too much about me at the bottom). OK, OK, Get involved must SURELY have that doc link, since people will need some documentation in ordered to do so!... Ugh, another rabbit trail... Oh, DUH! Must be under SUPPORT! OK, there it is! But even if I KNOW to look under support, it's STILL two clicks to documentation, where it WAS just one! And once on the documentation page, again, a whole bunch of fuzzy categories! Where's the one big list link, where a simple browser keyword search will immediately find the document for configuring gentoo for whatever fancy technology you're wanting to get up and running, be it something fancy like mdraid or lvm, or more basic like one's init system of choice or sound or OpenGL? But at least put a nice direct one-click link to docs direct on the front page. Please? No need to shout. (Tho unless it's directly in the main copy, I guess it'd still be two, since you have to click that weird icon with no actual words saying it's a menu, to open that, before you actually click what you want... *IF* you can actually find or guess what it's called... on the waaayyy too vague menu.) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
[gentoo-dev] RFC: making repoman complain about USE dependencies that lock packages at old version
Hi, This is quite a Portage patch topic but since devs are known to be unhappy about any change, I would like to start a bikeshed first. The idea is to make repoman/pcheck complain if the newest ebuild matched by version+slot restriction of dependency atom can't satisfy the USE dependency, or in other words, whenever the USE restriction forces lower version of package being installed. For example, let's assume the following: a. foo-1 has USE=bar, b. bar-1 depends on foo[bar], c. foo-2 no longer has USE=bar. In the usual scenario this means that bar-1 will silently lock foo at version 1. Since this is quite implicit, users don't notice the problem and left unnoticed for a long time, it starts to hurt bad at some point. With the added check, after step (c) above repoman/pcheck would warn on bar-1 that 'foo[bar]' dependency forces lower version than plain 'foo'. Since version and slot restrictions will be taken into consideration, it will possible to silence the warning by using: foo-2[bar] explicitly, if the dependency can't be fixed for foo-2 properly. While not improving the final result a lot, it will at least give an explicit '' or '=' atom that can be easily found with grep when considering old version removal. What are your thoughts? -- Best regards, Michał Górny pgpllRIKrmM77.pgp Description: OpenPGP digital signature
[gentoo-dev] Re: RFC: making repoman complain about USE dependencies that lock packages at old version
On Sun, 5 Apr 2015, Michał Górny wrote: This is quite a Portage patch topic but since devs are known to be unhappy about any change, I would like to start a bikeshed first. The idea is to make repoman/pcheck complain if the newest ebuild matched by version+slot restriction of dependency atom can't satisfy the USE dependency, or in other words, whenever the USE restriction forces lower version of package being installed. For example, let's assume the following: a. foo-1 has USE=bar, b. bar-1 depends on foo[bar], c. foo-2 no longer has USE=bar. In the usual scenario this means that bar-1 will silently lock foo at version 1. Since this is quite implicit, users don't notice the problem and left unnoticed for a long time, it starts to hurt bad at some point. Is foo[bar] even a legal dependency, if there are versions of foo that don't have the bar flag in their IUSE? (PMS doesn't seem to be entirely clear about this: ... it is an error for a use dependency to be applied to an ebuild which does not have the flag in question in IUSE_REFERENCEABLE.) So yes, make repoman shout about it. Loudly. :) With the added check, after step (c) above repoman/pcheck would warn on bar-1 that 'foo[bar]' dependency forces lower version than plain 'foo'. Since version and slot restrictions will be taken into consideration, it will possible to silence the warning by using: foo-2[bar] explicitly, if the dependency can't be fixed for foo-2 properly. While not improving the final result a lot, it will at least give an explicit '' or '=' atom that can be easily found with grep when considering old version removal. What are your thoughts? How are you going to treat foo[bar(-)] and foo[bar(+)]? Ulrich pgp0p2G4iqYOl.pgp Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] fetch(): fix support for digest size=None
On 04/05/2015 12:59 AM, Michał Górny wrote: It seems that the code initially supported fetching without size 'digest' but it got broken over time. Add proper conditionals to avoid ugly failures in this case. If we're handling this case, then we should probably not assume that the dict contains a size key. So, we could check that dict.get(size) is not None. -- Thanks, Zac
[gentoo-portage-dev] Re: Dynamic USE dependencies
Zac Medico zmed...@gentoo.org wrote: On 04/02/2015 09:32 AM, Rich Freeman wrote: Out of curiosity, what is keeping us from having USE flag dependencies handled dynamically, in the same way that package dependencies are? It's already able to adjust USE automatically via autounmask support. The problem is that once it is in your package.use (no matter whether added manually or with autounmask), it will stay there forever if the user does not analyze the dependencies manually eventually. And this is different from the dependency handling. The most obvious example (which presumably also Rich had in mind) is the case of abi_x86_32: Assume that you have package foo in your world file which (in version foo-1) depends on package bar. In version foo-2, which appears some years later, it no longer depends on package bar (but e.g. on package bazooka, instead). In this case, emerge --depclean will eventually unmerge bar. This works nicely. However, if foo-1 depends on bar[abi_x86_32(-)] and something else depends on bar[abi_x86_64] then, after upgrading to foo-2, most users wil keep compiling bar with ABI_X86=32 64 forever, because emerge --depclean (or any other tool) does not tell them that now they might remove abi_x86_32 from package bar. In fact, such a tool *cannot* exist, currently: There is no way to distinguish from package.use what is there because the user *wanted* this feature and what is there only because of a (possibly temporary) USE-dependency for another package. In other words: The difference between wanted and needed (reflected usually in the difference between the packages in world and those in /var/db/pkg) is not visible in package.use, in principle. So far, this was never a problem, because the number of flags which were there because they were needed (and not, because they were wanted) was rather low. But with the arrival of ABI_X86=32 the situation has changed; 100-200 needed flags are now normal on many systems. One suggestion around this problem would be to use different directories for these two types of use-flags, say package.use and package.use.needed. Perhaps it would make sense that portage --autounmask writes only to package.use.needed, und that there is a cleanup command which removes all data from package.use.needed except for that strictly necessary to resolve the dependencies of the running system (analogously like emerge --depclean currently removes unneeded packages).
Re: [gentoo-dev] RFC: making repoman complain about USE dependencies that lock packages at old version
On Sun, 5 Apr 2015 18:50:10 +0200 Michał Górny mgo...@gentoo.org wrote: What are your thoughts? if the version+slot restriction on foo in bar-1.ebuild contains foo packages that don't have bar useflag then it's an error in bar-1.ebuild (I think pms is pretty clear about this) Alexis.
Re: [gentoo-dev] libressl status
On 5 April 2015 at 19:59, Rich Freeman ri...@gentoo.org wrote: It seems unlikely that this would make much of a difference. I think that allowing this package to create another ffmpeg vs libav mess is a mistake. To be honest, the upstream developers should be fairly in the known regarding my opinion expressed in the blog posts, as they replied and read them before. We could also find other ways to deal with this by installing the packages as different library names and subverting pkg-config to choose between the two, but that would only work for packages using pkg-config to begin with and does not solve the transitive dependencies problem. Diego Elio Pettenò — Flameeyes https://blog.flameeyes.eu/
Re: [gentoo-dev] libressl status
On 04/05/2015 08:59 PM, Rich Freeman wrote: On Sun, Apr 5, 2015 at 2:35 PM, hasufell hasuf...@gentoo.org wrote: You are ranting at the wrong place. If you want to make a difference, take this to the openbsd mailing lists. It seems unlikely that this would make much of a difference. It doesn't make one here. I think that allowing this package to create another ffmpeg vs libav mess is a mistake. If you want to do some crazy downstream hackery, you'll have to do that yourself and count me out. It's not even clear what you want to fix. It was known from the start that we do not have ABI-compatibility. The only problem right now are libressl-exclusive features. These are currently optional codepaths afais, so packages still compile with openssl. Everything else is future prediction. That's why libressl is still in an overlay.
