Re: X509 login patches
On Mon, Dec 14, 2009 at 8:03 PM, Christos Triantafyllidis ct...@grid.auth.gr wrote: Hi all and welcome me to the list :), i'm using koji since a few week and i needed X509 authentication. Unfortunately current support for x509 was limited to: a) Use of the CN part only from the subject DN as the username Although traditionally CN can be the username of the user there are cases (like in our PKI) where CN is just Christos Triantafyllidis and of course many users can have the same name but different DNs. To avoid this but also keep the backwards compatibility i have introduced a new variable to be exported by both apache config (for git-web) and hub.conf (for the rest of the tools) called EnvVarForUserName which defines which variable to use as Username. For my case i have EnvVarForUserName = SSL_CLIENT_S_DN which uses the whole DN as username. What did you do about the email address? It normally uses c...@configured.org I should look at the patch of course. Steve b) Keep asking the user to provide their pass-phrase many times for the the same operation This leads (IMHO) many users to use password-less certificates. Unfortunately this is not acceptable according to our PKI policy so i added a callback to cache the passphrase within each koji execution. I have created some patches to both this limitations and i have uploaded the to my git repository[1]. Feel free to use/clone them. Best regards, Christos Triantafyllidis [1] http://git.afroditi.hellasgrid.gr/git/grid.auth.gr/koji.git -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Signing RPMs
On Thu, Nov 12, 2009 at 8:10 AM, Jitesh Shah jite...@marvell.com wrote:
..snip..
I to have wanted to get this to work.
I expect I have my key definition wrong, traceback below.
I have,
self.gpg_keys = {
'89D891FB': { 'name': 'oatrelease',
'description': 'EGEE SA1 (Operations
Automation Team) egee3-operations-automation-disc...@cern.ch',
} }
with
snip
Steve, you are using the subkey. You probably want to use the master
signing key i.e. the one listed under pub (47EBAC2B in your case)
Hi Jitesh,
Switching to the master key similar error as below.
./sign_unsigned.py --just-show dist-fc10
what is the level option?
'47EBAC2B': { 'name': 'oatrelease',
'description': 'EGEE SA1 (Operations
Automation Team) egee3-operations-automation-di
sc...@cern.ch',
}
Traceback (most recent call last):
File ./sign_unsigned.py, line 734, in module
x.run_command()
File ./sign_unsigned.py, line 284, in run_command
cmd()
File ./sign_unsigned.py, line 728, in cmd_default
self.sign_to_cache(uncached, self.options.level)
File ./sign_unsigned.py, line 638, in sign_to_cache
self.do_signing(pkglist, level)
File ./sign_unsigned.py, line 601, in do_signing
cmd = self.get_signing_command(level, mypaths[:nlen],
server=self.options.server)
File ./sign_unsigned.py, line 586, in get_signing_command
if self.gpg_keys[keyid]['size'] == 4096:
KeyError: None
The full edited script is here
http://cern.ch/steve.traylen/tmp/oat-sign_unsigned.py
is there something else I need to change?
Jitesh
Traceback (most recent call last):
File ./sign_unsigned.py, line 734, in module
x.run_command()
File ./sign_unsigned.py, line 285, in run_command
cmd()
File ./sign_unsigned.py, line 728, in cmd_default
self.sign_to_cache(uncached, self.options.level)
File ./sign_unsigned.py, line 638, in sign_to_cache
self.do_signing(pkglist, level)
File ./sign_unsigned.py, line 601, in do_signing
cmd = self.get_signing_command(level, mypaths[:nlen],
server=self.options.server)
File ./sign_unsigned.py, line 587, in get_signing_command
if self.gpg_keys[keyid]['size'] == 4096:
KeyError: None
Dennis
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
--
Steve Traylen
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
latest python-cheetah breaks koji.
Hi, I would just submit a bug but unsure to which package to submit in the first instance. On FC11 with Hi With, koji-hub-1.3.1-2.fc11.noarch koji-1.3.1-2.fc11.noarch koji-builder-1.3.1-2.fc11.noarch koji-web-1.3.1-2.fc11.noarch koji-utils-1.3.1-2.fc11.noarch python-cheetah-2.0.1-5.fc11.x86_64 then everything is good. Updating only python-cheetah to python-cheetah-2.2.2-1.fc11.x86_64 then python-web dumps to the browser as below, full traceback attached. Downgrading back to python-cheetah-2.0.1-5.fc11.x86_64 then all is good. File /usr/lib64/python2.6/site-packages/Cheetah/Compiler.py, line 1588, in __init__ source = unicode(source) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 374: ordinal not in range(128) full back trace attached. Steve -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: ActionNotAllowed: admin permission required
On Thu, May 14, 2009 at 7:42 PM, Lester Pimentel lester...@yahoo.com wrote: Hi all - I'm fighting with getting koji configured and running on a single machine to test it out on. Stepping through this wiki: http://fedoraproject.org/wiki/Koji/ServerHowTo Configuring it for ssl auth, I've gotten down to installing kojihub and when I try to do something like adding a user through koji, I get: [kojiad...@testmachine ~]$ koji add-user kojira ActionNotAllowed: admin permission required Here's what my users look like in the koji db: id | name | password | status | usertype | krb_principal +- ---+--++--+--- 10 | koji | | 0 | 0 | 12 | kojiadmin | | 0 | 1 | Any ideas on what I could be missing? Is this an ssl authentication error or database related? What about the insert into user_perms (user_id, perm_id) values (id of user inserted above, 1); in the howto? Steve Thanks, Lester -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: One build for multiple platforms?
On Thu, May 14, 2009 at 8:08 PM, Mike Bonnet mi...@redhat.com wrote:
Steve Traylen wrote:
Hi,
I thinking that the answer is that it is not currently possible but is
there any arrangement of configuration
to allow a build on say centos4 and centos5 concurrently.
A build tartget that has a fork to two buildroots and destination
tags. Both would need to work
for the overall task to work.
If you mean one koji build results in two builds being created in Koji,
then no that is not currently possible.
This sounds like something that could easily be handled with a Makefile
target though.
Create your separate dist-centos4 and dist-centos5 build/dest tags and
targets. Have a make build generate the appropriate SCM URL and call:
koji build --nowait dist-centos4 $SCMURL
koji build --nowait dist-centos5 $SCMURL
This works of course. I was hoping to force the dists to stay
in step though. builds are only tagged for el5 and el4 or not at all.
Rather like currently both i386 and x86_64 must both work to get either.
I can force the submitters to build on fc10 as well el4 and 5 to future proof
ourself.
Assuming you're using %{?dist} in your specfiles and have the
buildsys-macros defined correctly (and uniquely) in the -build tags, this
will create 2 separate builds in different tags from the same sources.
An item relevant to this. When building say a -el4.src.rpm on el5
koji build dist-centos5 foobar-1.5.2.el4.src.rpm
will always fail because the resulting foobar-1.5.2.el5.src.rpm does
not name match foobar-1.5.2.el4.src.rpm
Is this check useful? It requires one to create to src.rpms before they can
be submitted to dist-centos4 and 5. I can't for instance just grab a
src.rpm package
from fc10 and submit it.
(a cheeky p.s, did you get the slides I sent. no comment is just fine)
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
--
Steve Traylen
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: One build for multiple platforms?
On Thu, May 14, 2009 at 10:30 PM, Mike McLean mi...@redhat.com wrote: Steve Traylen wrote: Is this check useful? It requires one to create to src.rpms before they can be submitted to dist-centos4 and 5. I can't for instance just grab a src.rpm package from fc10 and submit it. Koji likes to know the NVR it is building before it starts building it. This allows Koji to mark the build as in progress. It also allows Koji to detect nvr conflicts early (rather than at the end of the build). If the resulting build does not match this NVR then that is pretty big consistency problem I too have run into this quirk. It is not that difficult to remake the srpm with the right dist tag before submitting. Indeed if there is a reason which there clearly is then its fine. As you say a quick rpm -Uvh rpmbuild -bs --define 'dist .el5' to the rescue. Steve -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
ActionNotAllowed: policy violation - but why?
Hi, I was reliably building on a tag before but now receive ActionNotAllowed: policy violation and can't seem shake it or understand why for a particular package Is is possible to get an explanation? http://skoji.cern.ch/koji/taskinfo?taskID=2918 This is following a build as CN=straylen koji build --nowait dist-centos4 ../SRPMS/mpich-1.2.7p1-2.el4.src.rpm My permissions. id | name | password | status | usertype | krb_principal +--+--++--+--- 1 | straylen | | 0 |0 | and user_id=1 does not appear in user_perms . i.e I am a boring user. The package has been added. koji list-pkgs --tag=dist-centos4 --package=mpich Package Tag Extra Arches Owner --- --- --- mpich dist-centos4 straylen Thanks again for the help. Steve -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: ActionNotAllowed: policy violation - but why?
On Fri, May 8, 2009 at 11:52 AM, Steve Traylen st...@traylen.net wrote: Hi, I was reliably building on a tag before but now receive ActionNotAllowed: policy violation A little bit more. ActionNotAllowed: policy violation -- all :: deny Steve and can't seem shake it or understand why for a particular package Is is possible to get an explanation? http://skoji.cern.ch/koji/taskinfo?taskID=2918 This is following a build as CN=straylen koji build --nowait dist-centos4 ../SRPMS/mpich-1.2.7p1-2.el4.src.rpm My permissions. id | name | password | status | usertype | krb_principal +--+--++--+--- 1 | straylen | | 0 | 0 | and user_id=1 does not appear in user_perms . i.e I am a boring user. The package has been added. koji list-pkgs --tag=dist-centos4 --package=mpich Package Tag Extra Arches Owner --- --- --- mpich dist-centos4 straylen Thanks again for the help. Steve -- Steve Traylen -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: ActionNotAllowed: policy violation - but why?
On Fri, May 8, 2009 at 4:04 PM, Mike Bonnet mi...@redhat.com wrote: Steve Traylen wrote: On Fri, May 8, 2009 at 11:52 AM, Steve Traylen st...@traylen.net wrote: Hi, I was reliably building on a tag before but now receive ActionNotAllowed: policy violation A little bit more. ActionNotAllowed: policy violation -- all :: deny What version of Koji are you running? I believe there was a bug in earlier versions that caused a missing policy entry in /etc/koji-hub/hub.conf to result in denials of everything. That should be fixed in 1.3.1. Alternately you could add a policy entry to allow building from srpm into the dist-centos4 tag: [policy] build_from_srpm = tag dist-centos4 :: allow has_perm admin :: allow all :: deny Hi Mike, I'm running stock FC10. koji-hub-1.3.1-1.fc10.noarch koji-1.3.1-1.fc10.noarch koji-web-1.3.1-1.fc10.noarch koji-utils-1.3.1-1.fc10.noarch koji-builder-1.3.1-1.fc10.noarch I've not had had any [policy] in the hub.conf file up to now and things have been okay, i.e I could build from cvs and svn for instance which I did yesterday. It's just building from srpm that has been blocked but I have not tried that in a while so that may have been the before the recent upgrade. Certainly making a very open policy [policy] build_from_srpm = tag dist-centos4 :: allow has_perm admin :: allow all :: all and then things proceed, I'll tune that now. More generally now about policies. Do you have some description on these and what can be set? If nothing exists if you can give me something brief then I'll try and write something up for the wiki. As it happens I'm giving a presentation in a weeks time to my colleagues on mock, koji and mash and certainly any content (e.g diagrams) I produce I'll write up in a generic way for inclusion in documentation. Thanks again Steve Steve and can't seem shake it or understand why for a particular package Is is possible to get an explanation? http://skoji.cern.ch/koji/taskinfo?taskID=2918 This is following a build as CN=straylen koji build --nowait dist-centos4 ../SRPMS/mpich-1.2.7p1-2.el4.src.rpm My permissions. id | name | password | status | usertype | krb_principal +--+--++--+--- 1 | straylen | | 0 | 0 | and user_id=1 does not appear in user_perms . i.e I am a boring user. The package has been added. koji list-pkgs --tag=dist-centos4 --package=mpich Package Tag Extra Arches Owner --- --- --- mpich dist-centos4 straylen Thanks again for the help. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
mash fails for me with x86_64.
Hi, On F10. i.e mash-0.4.2-3.fc10.noarch createrepo-0.9.6-3.fc10.noarch yum-3.2.21-2.fc10.noarch I want to build a repo out of a koji-tag containing packages built against centos4. mash -c etc/mash.conf -o out centos4 Mash gives an error File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 1305, in lambda repoXML = property(fget=lambda self: self._getRepoXML(), File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 1301, in _getRepoXML raise Errors.RepoError, msg yum.Errors.RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: centos4-x86_64. Please verify its path and try again my centos4.conf contains [centos4] rpm_path = %(arch)s/ repodata_path = %(arch)s/ source_path = SRPMS debuginfo = Falsea multilib = True multilib_method = devel tag = dist-centos4 inherit = True strict_keys = False arches = x86_64 with only arches = i386 all is well , with arches = x86_64 or both it fails as above. It looks very similar to https://bugzilla.redhat.com/show_bug.cgi?id=350391 but that was fixed and released some time ago. Full python message: 2009-05-07 10:35:50 mash: Getting package lists for dist-centos4... 2009-05-07 10:35:50 mash: Sorting packages... 2009-05-07 10:35:50 mash: Checking signatures... 2009-05-07 10:35:50 mash: Writing out files for out/centos4/x86_64/... 2009-05-07 10:35:50 mash: Writing out files for out/centos4/SRPMS... 2009-05-07 10:35:50 mash: Waiting for createrepo to finish... 2009-05-07 10:35:50 mash: Running createrepo for out/centos4/x86_64/... Previous repo file missing: 2009-05-07 10:35:50 mash: Running createrepo for out/centos4/SRPMS... Previous repo file missing: 2009-05-07 10:35:50 mash: Waiting for depsolve and createrepo to finish... 2009-05-07 10:35:50 mash: Resolving multilib for arch x86_64 using method devel Traceback (most recent call last): File /usr/bin/mash, line 82, in module main() File /usr/bin/mash, line 70, in main rc = themash.doMultilib() File /usr/lib/python2.5/site-packages/mash/__init__.py, line 513, in doMultilib pid = self.doDepSolveAndMultilib(arch, repocache) File /usr/lib/python2.5/site-packages/mash/__init__.py, line 464, in doDepSolveAndMultilib yumbase.doTsSetup() File /usr/lib/python2.5/site-packages/yum/depsolve.py, line 86, in doTsSetup return self._getTs() File /usr/lib/python2.5/site-packages/yum/depsolve.py, line 101, in _getTs self._getTsInfo(remove_only) File /usr/lib/python2.5/site-packages/yum/depsolve.py, line 112, in _getTsInfo pkgSack = self.pkgSack File /usr/lib/python2.5/site-packages/yum/__init__.py, line 592, in lambda pkgSack = property(fget=lambda self: self._getSacks(), File /usr/lib/python2.5/site-packages/yum/__init__.py, line 435, in _getSacks self.repos.populateSack(which=repos) File /usr/lib/python2.5/site-packages/yum/repos.py, line 251, in populateSack sack.populate(repo, mdtype, callback, cacheonly) File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 162, in populate if self._check_db_version(repo, mydbtype): File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 223, in _check_db_version return repo._check_db_version(mdtype) File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 1147, in _check_db_version repoXML = self.repoXML File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 1305, in lambda repoXML = property(fget=lambda self: self._getRepoXML(), File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 1301, in _getRepoXML raise Errors.RepoError, msg yum.Errors.RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: centos4-x86_64. Please verify its path and try again 2009-05-07 10:35:51 mash: Depsolve and createrepo finished. mash failed in out/centos4 Any ideas? Steve -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: mash fails for me with x86_64.
On Thu, May 7, 2009 at 4:00 PM, Bill Nottingham nott...@redhat.com wrote: 2009-05-07 10:35:50 mash: Waiting for depsolve and createrepo to finish... 2009-05-07 10:35:50 mash: Resolving multilib for arch x86_64 using method devel Traceback (most recent call last): File /usr/bin/mash, line 82, in module main() File /usr/bin/mash, line 70, in main rc = themash.doMultilib() File /usr/lib/python2.5/site-packages/mash/__init__.py, line 513, in doMultilib pid = self.doDepSolveAndMultilib(arch, repocache) File /usr/lib/python2.5/site-packages/mash/__init__.py, line 464, in doDepSolveAndMultilib yumbase.doTsSetup() File /usr/lib/python2.5/site-packages/yum/depsolve.py, line 86, in doTsSetup return self._getTs() File /usr/lib/python2.5/site-packages/yum/depsolve.py, line 101, in _getTs self._getTsInfo(remove_only) File /usr/lib/python2.5/site-packages/yum/depsolve.py, line 112, in _getTsInfo pkgSack = self.pkgSack File /usr/lib/python2.5/site-packages/yum/__init__.py, line 592, in lambda pkgSack = property(fget=lambda self: self._getSacks(), File /usr/lib/python2.5/site-packages/yum/__init__.py, line 435, in _getSacks self.repos.populateSack(which=repos) File /usr/lib/python2.5/site-packages/yum/repos.py, line 251, in populateSack sack.populate(repo, mdtype, callback, cacheonly) File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 162, in populate if self._check_db_version(repo, mydbtype): File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 223, in _check_db_version return repo._check_db_version(mdtype) File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 1147, in _check_db_version repoXML = self.repoXML File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 1305, in lambda repoXML = property(fget=lambda self: self._getRepoXML(), File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 1301, in _getRepoXML raise Errors.RepoError, msg yum.Errors.RepoError: Cannot retrieve repository metadata (repomd.xml) for repository: centos4-x86_64. Please verify its path and try again 2009-05-07 10:35:51 mash: Depsolve and createrepo finished. mash failed in out/centos4 Any ideas? Ugh. What's the temporary yum.conf that's created look like, if you pull it out? Hi Bill, Thanks this was enough to spot the problem. The relative path -o out was not resolved at a certain point. Using a full path it proceeded. https://bugzilla.redhat.com/show_bug.cgi?id=499645 submitted. Many Thanks Steve You may want to just try rebuilding the F11 version and see if that works for you. Bill -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: how init mock ? there are need some packages about i386 for build x86_64 packages !
2009/5/5 李建 lijian@gmail.com: hello,everybody! I've setup koji server , but i'm doubt how to set repo when some packages on x86_64 need packages on i386 . when i run command on x86_64 system: -- # koji build gtes-11 audit-1.7.7-6.el5_3.2.src.rpm --scratch the root.log have erros : -- DEBUG util.py:280: Executing command: /usr/bin/yum --installroot /var/lib/mock/gtes-11-build-13-15/root/ groupinstall build DEBUG util.py:256: Error: Missing Dependency: libgcc_s.so.1 is needed by package gcc DEBUG util.py:319: Child returncode was: 1 DEBUG backend.py:542: umount -n /var/lib/mock/gtes-11-build-13-15/root/proc = In my project , libgcc_s.so.1 is a file of libgcc-4.1.2-42.i386 。how to add the repo info about libgcc-4.1.2-42.i386 on x86_64 system enviroment ? Sounds like you want to add an external repository? http://fedoraproject.org/wiki/Koji/ExternalRepoServerBootstrap Steve Thanks! -- Jian Lee msn lijian@gmail.com -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: unicode error with mergerepos against CentOS 5.
On Wed, Mar 18, 2009 at 2:23 PM, Steve Traylen st...@traylen.net wrote: On Wed, Mar 18, 2009 at 2:18 PM, Seth Vidal skvi...@fedoraproject.org wrote: On Wed, 18 Mar 2009, Steve Traylen wrote: the metadata you're pulling in was written with a much older createrepo which was less picky about some of the data it wrote back out. So you end up with some pkgs with dodgy data in the changelogs. Mirroring in CentOS5 and then recreating from scratch the repodirs with a Fedora 10 createrepo the the repository is created spitting out some warnings. What ver of yum and createrepo do you have when you're running both createrepo and mergerepo? $ rpm -qf /usr/bin/yum \ /usr/bin/mergerepo \ /usr/bin/createrepo \ /usr/libexec/kojid/mergerepos yum-3.2.21-2.fc10.noarch createrepo-0.9.6-3.fc10.noarch createrepo-0.9.6-3.fc10.noarch koji-builder-1.3.1-1.fc10.noarch Hi Seth, Is there any workaround or anything for this? Steve -sv -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: unicode error with mergerepos against CentOS 5.
On Mon, Mar 16, 2009 at 1:29 PM, Seth Vidal skvi...@fedoraproject.org wrote:
On Mon, 16 Mar 2009, Steve Traylen wrote:
Hi,
$ rpm -qf /usr/libexec/kojid/mergerepos
koji-builder-1.3.1-1.fc10.noarch
Running agaist Fedora repositories all looks well but against CentOS.
/usr/libexec/kojid/mergerepos -a x86_64 \
-o repo \
-r http://swissmirror.silyus.net/centos/5.2/os/i386/
Adding repo: http://swissmirror.silyus.net/centos/5.2/os/i386/
1/400 - Deployment_Guide-or-IN-5.2-9.el5.centos.noarch
2/400 - 1:kde-i18n-Icelandic-3.5.4-1.noarch
328/400 - system-config-users-1.2.51-4.el5.noarch
329/400 - yum-priorities-1.1.10-9.el5.centos.noarch
Traceback (most recent call last):
File /usr/libexec/kojid/mergerepos, line 241, in module
main(sys.argv[1:])
File /usr/libexec/kojid/mergerepos, line 236, in main
merge.write_metadata()
File /usr/libexec/kojid/mergerepos, line 216, in write_metadata
mdgen.doPkgMetadata()
File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line
332, in doPkgMetadata
self.writeMetadataDocs(packages)
File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line
472, in writeMetadataDocs
self.primaryfile.write(po.xml_dump_primary_metadata())
File /usr/lib/python2.5/site-packages/yum/packages.py, line 943,
in xml_dump_primary_metadata
msg += misc.to_unicode(self._dump_format_items())
File /usr/lib/python2.5/site-packages/yum/packages.py, line 809,
in _dump_format_items
msg += self._dump_pco('provides')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position
38: ordinal not in range(128)
Tried with a few other repos as well, e.g the i386 one , the two
merged , ... it fails the same
though apparently at different packages.
the metadata you're pulling in was written with a much older createrepo
which was less picky about some of the data it wrote back out. So you end up
with some pkgs with dodgy data in the changelogs.
Mirroring in CentOS5 and then recreating from scratch the repodirs with a Fedora
10 createrepo the the repository is created spitting out some warnings.
1556/2382 - CentOS/ghostscript-devel-8.15.2-9.1.el5_1.1.i386.rpm
iso-8859-1 encoding on /usr/lib/aspell-0.60/�slenska.alias
1598/2382 - CentOS/avalon-logkit-javadoc-1.2-4jpp.3.i386.rpm
iso-8859-1 encoding on Ville Skytt� ville.skytta at iki.fi - 1:0.2-1jpp
and then the merge fails with.
/usr/libexec/kojid/mergerepos -a i386 \
-o repo \
-r http://skojihub.cern.ch/mirror/centos/5/updates/i386 \
-r http://skojihub.cern.ch/mirror/centos/5/os/i386
Adding repo: http://skojihub.cern.ch/mirror/centos/5/updates/i386
Adding repo: http://skojihub.cern.ch/mirror/centos/5/os/i386
1/2607 - openssl-perl-0.9.8b-10.el5_2.1.i386
2/2607 - 1:cups-1.2.4-11.18.el5_2.3.i386
1353/2607 - ImageMagick-c++-6.2.8.0-4.el5_1.1.i386
1354/2607 - mesa-libGL-6.5.1-7.5.el5.i386
Traceback (most recent call last):
File /usr/libexec/kojid/mergerepos, line 241, in module
main(sys.argv[1:])
File /usr/libexec/kojid/mergerepos, line 236, in main
merge.write_metadata()
File /usr/libexec/kojid/mergerepos, line 216, in write_metadata
mdgen.doPkgMetadata()
File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line
332, in doPkgMetadata
self.writeMetadataDocs(packages)
File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line
472, in writeMetadataDocs
self.primaryfile.write(po.xml_dump_primary_metadata())
File /usr/lib/python2.5/site-packages/yum/packages.py, line 943,
in xml_dump_primary_metadata
msg += misc.to_unicode(self._dump_format_items())
File /usr/lib/python2.5/site-packages/yum/packages.py, line 809,
in _dump_format_items
msg += self._dump_pco('provides')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position
38: ordinal not in range(128)
Have read around a bit on .spec file character and there looks to be some
recommendations but nothing concrete? e.g
http://wiki.mandriva.com/en/Policies/Charset#In_spec_file_itself
Steve
-sv
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
--
Steve Traylen
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: unicode error with mergerepos against CentOS 5.
On Wed, Mar 18, 2009 at 2:18 PM, Seth Vidal skvi...@fedoraproject.org wrote: On Wed, 18 Mar 2009, Steve Traylen wrote: the metadata you're pulling in was written with a much older createrepo which was less picky about some of the data it wrote back out. So you end up with some pkgs with dodgy data in the changelogs. Mirroring in CentOS5 and then recreating from scratch the repodirs with a Fedora 10 createrepo the the repository is created spitting out some warnings. What ver of yum and createrepo do you have when you're running both createrepo and mergerepo? $ rpm -qf /usr/bin/yum \ /usr/bin/mergerepo \ /usr/bin/createrepo \ /usr/libexec/kojid/mergerepos yum-3.2.21-2.fc10.noarch createrepo-0.9.6-3.fc10.noarch createrepo-0.9.6-3.fc10.noarch koji-builder-1.3.1-1.fc10.noarch -sv -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
mergerepos of f10 x86_64 release and updates does not contain perl or some other packages?
Hi, Merging works fine for i386 f10 repositories but for x86_64 the following happens. The merged repository of f10's Everything and updates is created and then using this new repo mock tries to create and installroot. The problem is that this new repository is unable to provide some items, perl, /bin/bash and so the install root fails. Here are the details. The blocklist file following here is empty. $ /usr/libexec/kojid/mergerepos -a x86_64 \ -b /mnt/koji/repos/dist-f10-build/671/x86_64/blocklist \ -o /tmp/koji/tasks/2021/2021/repo \ -g /mnt/koji/repos/dist-f10-build/671/groups/comps.xml \ -r http://mirror.switch.ch/ftp/mirror/fedora/linux/updates/10/x86_64/ \ -r http://mirror.switch.ch/ftp/mirror/fedora/linux/releases/10/Everything/x86_64/os/ Adding repo: http://mirror.switch.ch/ftp/mirror/fedora/linux/updates/10/x86_64/ Adding repo: http://mirror.switch.ch/ftp/mirror/fedora/linux/releases/10/Everything/x86_64/os/ 1/12364 - openswan-doc-2.6.19-1.fc10.x86_64 2/12364 - ssm-0.1-12.fc10.x86_64 ... ... 1112/12364 - 4:perl-5.10.0-56.fc10.x86_64 ... ... 12363/12364 - libextractor-plugins-exiv2-0.5.20b-2.fc10.x86_64 12364/12364 - lwp-devel-2.4-1.fc10.x86_64 Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Starting other db creation: Tue Mar 17 14:49:44 2009 Ending other db creation: Tue Mar 17 14:50:50 2009 Starting filelists db creation: Tue Mar 17 14:51:03 2009 Ending filelists db creation: Tue Mar 17 14:55:37 2009 Starting primary db creation: Tue Mar 17 14:55:40 2009 Ending primary db creation: Tue Mar 17 14:58:28 2009 Sqlite DBs complete The following is then tried from within mock. The chroot's yum.conf is definitely referencing the repository I just created (I've checked carefully and have recreated the repo a few times just to check this is consistent. ) /usr/bin/yum --installroot /var/lib/mock/dist-f10-build-13-671/root/ \ groupinstall build this results in EBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: /usr/bin/perl is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: perl(Getopt::Long) is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: /bin/sh is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: mktemp is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: fedora-release-notes-10.0.0-1.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: /bin/sh is needed by package fedora-release-notes-10.0.0-1.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: /bin/bash is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /bin/bash is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: mktemp is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /bin/sh is needed by package fedora-release-notes-10.0.0-1.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /usr/bin/perl is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /bin/sh is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: perl(Getopt::Long) is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) And indeed. /usr/bin/yum --installroot /var/lib/mock/dist-f10-build-13-671/root list perl Error: No matching Packages to list There are lots of packages in this repository, but no perl it seems. /usr/bin/yum --installroot /var/lib/mock/dist-f10-build-13-671/root list | wc -l shows 3483 packages. Note the main box I am working on is 32 bit F10 box which may be relevant and the reason why only subsequently the i386 build works? Steve -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: mergerepo fails with PCDATA invalid Char value 8
On Sun, Mar 15, 2009 at 6:39 PM, Mike Bonnet mi...@redhat.com wrote: Steve Traylen wrote: On Sun, Mar 15, 2009 at 12:45 PM, Steve Traylen st...@traylen.net wrote: Hi, Got koji basically working for me over the last couple of weeks. Was very keen to try its new external repository support. Starting with a fresh instance I made a tag (dist-slc5) containing two repos. koji add-external-repo -t dist-slc5 -p 10 slc5-64-base http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os koji add-external-repo -t dist-slc5 -p 10 slc5-32-base http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os and then tried to make a koji repo from that. koji regen-repo dist-slc5 This called /usr/libexec/kojid/mergerepos -a i386 -b /mnt/koji/repos/dist-slc5-build/189/i386/blocklist -o /tmp/koji/tasks/556/556/repo \ -g /mnt/koji/repos/dist-slc5-build/189/groups/comps.xml -r http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os/ \ -r http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os/ resulting in as below. Any ideas ? To hopefully answer my own question. Is this because these slc yum repositories do not contain the sqlite files thats that mergerepo makes use of. Looking at CentOS and ScientificLinux neither of these look to make use of the '-d' option to createrepo to generate the sqlite files. Is there a way around this or we have to ask CentOS to generate the sql files? The error occurs when parsing other.xml. I would check your external repos to see if other.xml passes XML validation successfully. Hi Mike, That's exactly the problem. Thanks. It fails later now but I'll look into it first. Steve Of course maybe it is something else entirely? Steve Steve process:19630): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing other.xml error: PCDATA invalid Char value 8 Traceback (most recent call last): File /usr/libexec/kojid/mergerepos, line 241, in module main(sys.argv[1:]) File /usr/libexec/kojid/mergerepos, line 236, in main merge.write_metadata() File /usr/libexec/kojid/mergerepos, line 216, in write_metadata mdgen.doPkgMetadata() File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line 332, in doPkgMetadata self.writeMetadataDocs(packages) File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line 475, in writeMetadataDocs clog_limit=self.conf.changelog_limit)) File /usr/lib/python2.5/site-packages/yum/packages.py, line 959, in xml_dump_other_metadata msg += %s\n/package\n % misc.to_unicode(self._dump_changelog(clog_limit)) File /usr/lib/python2.5/site-packages/yum/packages.py, line 927, in _dump_changelog if not self.changelog: File /usr/lib/python2.5/site-packages/yum/packages.py, line 423, in lambda changelog = property(fget=lambda self: self.returnChangelog()) File /usr/lib/python2.5/site-packages/yum/sqlitesack.py, line 225, in returnChangelog self._loadChangelog() File /usr/lib/python2.5/site-packages/yum/sqlitesack.py, line 202, in _loadChangelog self.sack.populate(self.repo, mdtype='otherdata') File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 184, in populate dobj = repo_cache_function(xml, csum) File /usr/lib/python2.5/site-packages/sqlitecachec.py, line 60, in getOtherdata self.repoid)) TypeError: Parsing other.xml error: PCDATA invalid Char value 8 Steve -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
unicode error with mergerepos against CentOS 5.
Hi,
$ rpm -qf /usr/libexec/kojid/mergerepos
koji-builder-1.3.1-1.fc10.noarch
Running agaist Fedora repositories all looks well but against CentOS.
/usr/libexec/kojid/mergerepos -a x86_64 \
-o repo \
-r http://swissmirror.silyus.net/centos/5.2/os/i386/
Adding repo: http://swissmirror.silyus.net/centos/5.2/os/i386/
1/400 - Deployment_Guide-or-IN-5.2-9.el5.centos.noarch
2/400 - 1:kde-i18n-Icelandic-3.5.4-1.noarch
328/400 - system-config-users-1.2.51-4.el5.noarch
329/400 - yum-priorities-1.1.10-9.el5.centos.noarch
Traceback (most recent call last):
File /usr/libexec/kojid/mergerepos, line 241, in module
main(sys.argv[1:])
File /usr/libexec/kojid/mergerepos, line 236, in main
merge.write_metadata()
File /usr/libexec/kojid/mergerepos, line 216, in write_metadata
mdgen.doPkgMetadata()
File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line
332, in doPkgMetadata
self.writeMetadataDocs(packages)
File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line
472, in writeMetadataDocs
self.primaryfile.write(po.xml_dump_primary_metadata())
File /usr/lib/python2.5/site-packages/yum/packages.py, line 943,
in xml_dump_primary_metadata
msg += misc.to_unicode(self._dump_format_items())
File /usr/lib/python2.5/site-packages/yum/packages.py, line 809,
in _dump_format_items
msg += self._dump_pco('provides')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position
38: ordinal not in range(128)
Tried with a few other repos as well, e.g the i386 one , the two
merged , ... it fails the same
though apparently at different packages.
--
Steve Traylen
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
mergerepo fails with PCDATA invalid Char value 8
Hi, Got koji basically working for me over the last couple of weeks. Was very keen to try its new external repository support. Starting with a fresh instance I made a tag (dist-slc5) containing two repos. koji add-external-repo -t dist-slc5 -p 10 slc5-64-base http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os koji add-external-repo -t dist-slc5 -p 10 slc5-32-base http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os and then tried to make a koji repo from that. koji regen-repo dist-slc5 This called /usr/libexec/kojid/mergerepos -a i386 -b /mnt/koji/repos/dist-slc5-build/189/i386/blocklist -o /tmp/koji/tasks/556/556/repo \ -g /mnt/koji/repos/dist-slc5-build/189/groups/comps.xml -r http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os/ \ -r http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os/ resulting in as below. Any ideas ? Steve process:19630): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing other.xml error: PCDATA invalid Char value 8 Traceback (most recent call last): File /usr/libexec/kojid/mergerepos, line 241, in module main(sys.argv[1:]) File /usr/libexec/kojid/mergerepos, line 236, in main merge.write_metadata() File /usr/libexec/kojid/mergerepos, line 216, in write_metadata mdgen.doPkgMetadata() File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line 332, in doPkgMetadata self.writeMetadataDocs(packages) File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line 475, in writeMetadataDocs clog_limit=self.conf.changelog_limit)) File /usr/lib/python2.5/site-packages/yum/packages.py, line 959, in xml_dump_other_metadata msg += %s\n/package\n % misc.to_unicode(self._dump_changelog(clog_limit)) File /usr/lib/python2.5/site-packages/yum/packages.py, line 927, in _dump_changelog if not self.changelog: File /usr/lib/python2.5/site-packages/yum/packages.py, line 423, in lambda changelog = property(fget=lambda self: self.returnChangelog()) File /usr/lib/python2.5/site-packages/yum/sqlitesack.py, line 225, in returnChangelog self._loadChangelog() File /usr/lib/python2.5/site-packages/yum/sqlitesack.py, line 202, in _loadChangelog self.sack.populate(self.repo, mdtype='otherdata') File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 184, in populate dobj = repo_cache_function(xml, csum) File /usr/lib/python2.5/site-packages/sqlitecachec.py, line 60, in getOtherdata self.repoid)) TypeError: Parsing other.xml error: PCDATA invalid Char value 8 Steve -- Steve Traylen $ /usr/libexec/kojid/mergerepos -a i386 -b /mnt/koji/repos/dist-slc5-build/189/i386/blocklist -o /tmp/koji/tasks/556/556/repo -g /mnt/koji/repos/dist-slc5-build/189/groups/comps.xml -r http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os/ -r http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os/ Adding repo: http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os/ Adding repo: http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os/ (process:19630): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing other.xml error: PCDATA invalid Char value 8 (process:19630): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing other.xml error: PCDATA invalid Char value 8 (process:19630): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing other.xml error: PCDATA invalid Char value 8 (process:19630): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing other.xml error: PCDATA invalid Char value 8 (process:19630): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing other.xml error: PCDATA invalid Char value 8 Traceback (most recent call last): File /usr/libexec/kojid/mergerepos, line 241, in module main(sys.argv[1:]) File /usr/libexec/kojid/mergerepos, line 236, in main merge.write_metadata() File /usr/libexec/kojid/mergerepos, line 216, in write_metadata mdgen.doPkgMetadata() File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line 332, in doPkgMetadata self.writeMetadataDocs(packages) File /usr/lib
Re: how to build srpms on koji?
On Sun, Mar 15, 2009 at 10:55 AM, 陈鲍孜 chenba...@gmail.com wrote: Hi, After long time fighting with koji configuration, I think the service is now running successfully. I added some srpms to it and saw its information on kojiweb. But it seems I can not build the srpms by koji (when I submit a build task to koji, it will open for a while, then it would finally come to be failed). I doubted if I have missed some configurations which connecting koji and mock. Does the build get as far as kojid. Is there anything in /var/log/kojid.log ? This can tell you the mock problem and then where to look for that. Steve -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
