Re: X509 login patches
On 12/14/2009 02:03 PM, Christos Triantafyllidis wrote: Hi all and welcome me to the list :), Welcome, and thanks for the patches! Comments in-line. i'm using koji since a few week and i needed X509 authentication. Unfortunately current support for x509 was limited to: a) Use of the CN part only from the subject DN as the username Although traditionally CN can be the username of the user there are cases (like in our PKI) where CN is just Christos Triantafyllidis and of course many users can have the same name but different DNs. To avoid this but also keep the backwards compatibility i have introduced a new variable to be exported by both apache config (for git-web) and hub.conf (for the rest of the tools) called EnvVarForUserName which defines which variable to use as Username. For my case i have EnvVarForUserName = SSL_CLIENT_S_DN which uses the whole DN as username. koji-hub already supports a DNUsernameComponent option. Rather than introduce a new config option, I think I'd rather see DNUsernameComponent=DN special-cased to mean use the whole DN. I don't see any env. vars other than DN that would be useful for authentication. b) Keep asking the user to provide their pass-phrase many times for the the same operation This leads (IMHO) many users to use password-less certificates. Unfortunately this is not acceptable according to our PKI policy so i added a callback to cache the passphrase within each koji execution. This looks very interesting, thanks. I'll see about testing it locally and merging it. I wonder if this could be extended to integrate with gnome-keyring (or similar) to provide once-per-session login for SSL certificates. I'll look into this. I have created some patches to both this limitations and i have uploaded the to my git repository[1]. Feel free to use/clone them. Best regards, Christos Triantafyllidis [1] http://git.afroditi.hellasgrid.gr/git/grid.auth.gr/koji.git -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: astronomy-bookmarks conflicts with fedora-bookmarks
On 12/14/2009 04:50 PM, Paul W. Frields wrote: On Mon, Dec 14, 2009 at 04:32:56PM -0500, Eric Christensen wrote: When trying to install astronomy-bookmarks I get the error that astronomy-bookmarks conflicts with fedora-bookmarks in PackageKit. I looked at the source (an HTML file) but I can't quite figure out why it would conflict. Does anyone know? [p...@scarlett ~]$ repoquery -q --provides astronomy-bookmarks astronomy-bookmarks = 1-6.fc12 system-bookmarks [p...@scarlett ~]$ repoquery -q --provides fedora-bookmarks fedora-bookmarks = 11-2 system-bookmarks They both provide 'system-bookmarks'. Multiple packages can Provides: the same thing without problems. The issue here is that astronomy-bookmarks explicitly Conflicts: with fedora-bookmarks: http://koji.fedoraproject.org/koji/rpminfo?rpmID=1459223 -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Postgresql Database Error
On 11/18/2009 02:15 AM, peng chen wrote: hello, fedora-buildsys-list: when I requset a build task for pakcage anaconda to koji, one errie error come out. It detailed as follow: pg.DatabaseError: error ' ERROR: new row for relation task violates check constraint task_weight_check ' in ' UPDATE task SET weight=-0.9838856091396 WHERE id = 16296 ' I'm sure that the source rpm of anaconda is OK,because I succed to build it in local mock environment. and the repo is the same as the koji build server. hope you do me a favor sincerely! Does one of the builds have a completion_time earlier than its create_event time? Koji uses the build duration to dynamically calculate the weight of the task. It should probably be checking for a negative result, but it doesn't. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: latest python-cheetah breaks koji.
On 09/30/2009 04:05 AM, Steve Traylen wrote: Hi, I would just submit a bug but unsure to which package to submit in the first instance. On FC11 with Hi With, koji-hub-1.3.1-2.fc11.noarch koji-1.3.1-2.fc11.noarch koji-builder-1.3.1-2.fc11.noarch koji-web-1.3.1-2.fc11.noarch koji-utils-1.3.1-2.fc11.noarch python-cheetah-2.0.1-5.fc11.x86_64 then everything is good. Updating only python-cheetah to python-cheetah-2.2.2-1.fc11.x86_64 then python-web dumps to the browser as below, full traceback attached. Downgrading back to python-cheetah-2.0.1-5.fc11.x86_64 then all is good. File /usr/lib64/python2.6/site-packages/Cheetah/Compiler.py, line 1588, in __init__ source = unicode(source) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 374: ordinal not in range(128) full back trace attached. Sorry about that, my fault. I hadn't finished testing Koji with the new Cheetah before I pushed it. This patch should fix the errors: http://mikeb.fedorapeople.org/0001-handle-all-strings-as-unicode-for-compatibility-with.patch It'll be pushed to the Koji git repo shortly. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Về: Fedora-buildsys-list Digest, Vol 55 , Issue 2
On 09/06/2009 07:34 AM, NGUYEN VAN TAN wrote:
Thank you, I saw kojid log, but I don't understand what it mean.
2009-09-06 14:18:54,311 [INFO] koji.build.TaskManager: open task: {'waiting':
True, 'id': 7, 'weight': 0.10001}
2009-09-06 14:18:54,398 [WARNING] koji.build.TaskManager: TRACEBACK: Traceback
(most recent call last):
File /usr/sbin/kojid, line 1275, in runTask
response = (handler.run(),)
File /usr/sbin/kojid, line 1351, in run
return self.handler(*self.params,**self.opts)
File /usr/sbin/kojid, line 2560, in handler
for f in os.listdir(self.datadir):
OSError: [Errno 2] No such file or directory:
'/tmp/koji/tasks/8/8/repo/repodata'
2009-09-06 14:18:54,450 [WARNING] koji.build.TaskManager: TRACEBACK: Traceback
(most recent call last):
File /usr/sbin/kojid, line 1275, in runTask
response = (handler.run(),)
File /usr/sbin/kojid, line 1351, in run
return self.handler(*self.params,**self.opts)
File /usr/sbin/kojid, line 2560, in handler
for f in os.listdir(self.datadir):
OSError: [Errno 2] No such file or directory:
'/tmp/koji/tasks/9/9/repo/repodata'
Do you have any packages or external repos associated with the tag
you're trying to create a repo for? If not, then createrepo will never
get called, and it won't create the repodata/ subdirectory. Add a
package or external repo to the tag, and newRepo/createrepo tasks should
start succeeding.
2009-09-06 14:19:09,407 [INFO] koji.build.TaskManager: pids: {7: 10867, 8:
10873, 9: 10877}
2009-09-06 14:19:09,417 [INFO] koji.build.TaskManager: open task: {'waiting':
True, 'id': 7, 'weight': 0.10001, 'alert': True}
2009-09-06 14:19:09,418 [INFO] koji.build.TaskManager: Waking up task:
{'waiting': True, 'id': 7, 'weight': 0.10001, 'alert': True}
2009-09-06 14:19:09,424 [INFO] koji.build.TaskManager: Task 8 (pid 10873)
exited with status 0
2009-09-06 14:19:09,582 [WARNING] koji.build.TaskManager: FAULT:
Traceback (most recent call last):
File /usr/sbin/kojid, line 1275, in runTask
response = (handler.run(),)
File /usr/sbin/kojid, line 1351, in run
return self.handler(*self.params,**self.opts)
File /usr/sbin/kojid, line 2517, in handler
results = self.wait(subtasks.values(), all=True, failany=True)
File /usr/sbin/kojid, line 1438, in wait
return dict(session.host.taskWaitResults(self.id,subtasks))
File /usr/lib/python2.5/site-packages/koji/__init__.py, line 1255, in
__call__
return self.__func(self.__name,args,opts)
File /usr/lib/python2.5/site-packages/koji/__init__.py, line 1501, in
_callMethod
raise err
Fault:Fault 1: 'Traceback (most recent call last):
File /usr/sbin/kojid, line 1275, in runTask
response = (handler.run(),)
File /usr/sbin/kojid, line 1351, in run
return self.handler(*self.params,**self.opts)
File /usr/sbin/kojid, line 2560, in handler
for f in os.listdir(self.datadir):
OSError: [Errno 2] No such file or directory:
\'/tmp/koji/tasks/8/8/repo/repodata\'
'
2009-09-06 14:19:09,610 [INFO] koji.build.TaskManager: Expiring subsession 27
(task 8)
2009-09-06 14:19:09,666 [INFO] koji.build.TaskManager: Task 9 (pid 10877)
exited with status 0
2009-09-06 14:19:09,694 [INFO] koji.build.TaskManager: Expiring subsession 28
(task 9)
2009-09-06 14:19:25,813 [INFO] koji.build.TaskManager: pids: {7: 10867}
2009-09-06 14:19:25,943 [INFO] koji.build.TaskManager: Task 7 (pid 10867)
exited with status 0
2009-09-06 14:19:25,973 [INFO] koji.build.TaskManager: Expiring subsession 26
(task 7)
I also saw kojira log, but I can't find the what problem is.
2009-09-06 13:09:23,652 [INFO] koji.repo.manager: Found repo 24, state=INIT
2009-09-06 13:10:07,760 [INFO] koji.repo.manager: Problem: newRepo task 116 for
tag 2 is FAILED
2009-09-06 13:12:19,724 [INFO] koji.repo.manager: Created newRepo task 121 for
tag 2 (dist-foo-build)
2009-09-06 13:12:25,373 [INFO] koji.repo.manager: Found repo 25, state=INIT
2009-09-06 13:12:32,349 [INFO] koji.repo.manager: Problem: newRepo task 121 for
tag 2 is FAILED
2009-09-06 14:07:52,959 [INFO] koji: Entering main loop
2009-09-06 14:14:26,191 [INFO] koji.repo.manager: Created newRepo task 1 for
tag 2 (dist-foo-build)
2009-09-06 14:14:36,261 [INFO] koji.repo.manager: Found repo 1, state=INIT
2009-09-06 14:15:06,624 [INFO] koji.repo.manager: Problem: newRepo task 1 for
tag 2 is FAILED
2009-09-06 14:15:06,654 [INFO] koji.repo.manager: Created newRepo task 4 for
tag 2 (dist-foo-build)
2009-09-06 14:15:21,807 [INFO] koji.repo.manager: Problem: newRepo task 4 for
tag 2 is FAILED
2009-09-06 14:15:21,868 [INFO] koji.repo.manager: Found repo 2, state=INIT
2009-09-06 14:18:24,378 [INFO] koji.repo.manager: Created newRepo task 7 for
tag 2 (dist-foo-build)
2009-09-06 14:18:43,487 [INFO] koji.repo.manager: Found repo 3, state=INIT
2009-09-06 14:19:11,220 [INFO] koji.repo.manager: Problem: newRepo task 7 for
tag 2 is FAILED
2009-09-06 14:21:41,670 [INFO]
Re: static linking and LGPL libraries
On 09/04/2009 03:10 AM, Hans de Goede wrote: On 09/03/2009 09:22 PM, Tom spot Callaway wrote: On 09/03/2009 02:25 PM, Hans de Goede wrote: Note that we have the same problem with any package which does static linking against an lgpl library (such as glibc). This is (one of the big reasons) why we only permit static linking with explicit approval from FESCo. Still we do allow it for certain packages and every such package has the potential LGPL problem Notting described. I personally think it would be a good idea to include root.log inside the srpm of packages, this way people can truely use their GPL rights and exactly re-create the package by using all the same packages in a chroot as used during the original build. We don't keep every Koji build indefinitely. We need to garbage-collect older builds or our storage requirements would grow without bound. Rebuilding a package using *exactly* the same NVRs present in the buildroot is not practical, and after a few weeks, generally not possible. -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: glibc error reports go to the bit bucket in koji
On 08/18/2009 06:14 PM, Tom Lane wrote: I've been poking away at the mysql crashes I mentioned a few days ago, and have just realized something that explains why I've been at such a loss to interpret the failure reports from koji. It seems that what has been getting triggered is glibc's malloc-error aborts, the ones that look like *** glibc detected *** /home/tgl/rpmwork/BUILD/mysql-5.1.37/client/.libs/lt-mysqlslap: free(): invalid pointer: 0x01dc6eb0 *** === Backtrace: = /lib64/libc.so.6[0x309a275a96] /home/tgl/rpmwork/BUILD/mysql-5.1.37/libmysql_r/.libs/libmysqlclient_r.so.16(my_thread_end+0x62)[0x7fb1910b9612] /home/tgl/rpmwork/BUILD/mysql-5.1.37/client/.libs/lt-mysqlslap(run_task+0x2df)[0x4033ef] /lib64/libpthread.so.0[0x309ae0686a] /lib64/libc.so.6(clone+0x6d)[0x309a2de39d] === Memory map: 0040-00408000 r-xp fd:01 1522370 /home/tgl/rpmwork/BUILD/mysql-5.1.37/client/.libs/lt-mysqlslap 00608000-0060a000 rw-p 8000 fd:01 1522370 /home/tgl/rpmwork/BUILD/mysql-5.1.37/client/.libs/lt-mysqlslap .. etc etc .. **The build logs from koji do not contain this rather critical information**. I haven't dug into the glibc sources, but what it looks like on my own machine is that these reports go to /dev/tty not to stderr (and thus not into any log file). I will not presume to question the sanity of the /dev/tty default, but surely this is a *completely* undesirable behavior within the koji environment. Can't we fix things so that such reports show up in the build.log? This is more of a mock issue, since mock is responsible for capturing the output of the build process and directing it to build.log, which koji simply stores. Is this reproduceable in mock on a local machine? How would mock go about capturing output sent directly to /dev/tty? Could we get away with hard/sym-linking it to /dev/stderr? Does this have the potential to break other things? -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Messaging SIG - proposal for our notification infrastructure
On 08/04/2009 06:20 PM, John Palmieri wrote: Hey everyone. I put up a proposal[1] that describes a publish/subscribe setup for the infrastructure wide notification system. I haven't quite gotten to the publish side of things because the QMF docs get a little hazy there but the meat of the proposal is there and I wanted to get feedback sooner than later. An event/notification system is important to the work I need to do going forward. I specifically avoided method invocation and properties/statistics as they can be added in a later round if we feel we need them. I do feel statistics might be nice (for instance keeping track of information that is expensive to do via a query but cheap to update based on events) but they are a bonus that we don't need right away. Thanks for writing this up, I'm glad this is finally gaining some momentum, and I'm going to be working on adding support for this to Koji soon. In addition to the event model you outline, I think we should also look at how we can support synchronous communication (method calls) via the bus. One of the big advantages of the bus is having a single transport and data exchange format, rather than having to teach each application how to speak xml-rpc, json, soap, etc. http://qpid.apache.org/qmf-protocol.html has some interesting notes about communication patterns. The unsolicited-indication looks like our event use-case. Request-response would be a normal method call. Query-indication looks like something in-between, and would be useful for getting information about a long-running process (koji watch-task comes to mind). To enable two-way communication we'll need some kind of adapter framework that sits on the bus and converts method calls on the bus to requests to the back-end services. Ideally this layer will be generic enough to be used by many/all of the different services used in the infrastructure. It could even be a single instance which registers multiple objects on the bus and proxies their methods to the separate backend systems. ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Re: Trouble with koji
On 07/29/2009 06:09 PM, Jochen Schmitt wrote: Hallo, I'm trying to make a scratch build for kaya on rawhide to fix a FTBFS. Unfortunately, I have got the lollowing error message: DEBUG util.py:256: No Package Found for ghc-editline But from my view of point this package should exist. It may be nice, if anyone can help me. You may find the build at: http://koji.fedoraproject.org/koji/taskinfo?taskID=1563211 The build of ghc-editline creates 3 subpackages ghc-editline-devel ghc-editline-doc ghc-editline-prof but no ghc-editline base package. http://koji.fedoraproject.org/koji/buildinfo?buildID=121279 So there is no package providing ghc-editline. I don't know enough about Haskell packaging to know whether this is expected or not. -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: Trouble with koji
On 07/29/2009 07:37 PM, Jochen Schmitt wrote: Am 29.07.2009 12:21, schrieb Mike Bonnet: The build of ghc-editline creates 3 subpackages ghc-editline-devel ghc-editline-doc ghc-editline-prof Thank you for your hint. Now it's works for Rawhide, but not for F-11. On F-11 I got DEBUG util.py:256: No Package Found for ghc-editline-devel The build you may find at: http://koji.fedoraproject.org/koji/taskinfo?taskID=1563427 It looks like ghc-editline was only added to Fedora after F11 GA. It has been built for F11 but never pushed through bodhi, so it never made it into the Koji buildroots for F11 updates. You'll need to create and update via bodhi, or request a buildroot override by emailing rel-...@fedoraproject.org explaining why it's required (for example, so dependent packages can be built and the entire set pushed out as one update). -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: using yumdownloader to fetch sources from koji
On 07/16/2009 05:46 AM, Marcus Moeller wrote: Hi all, is there a way to add http://kojipkgs.fedoraproject.org/packages/ as yum repository, as I wanted to make use of yumdownloader to fetch some SRPMs from there. Using: koji download-build --arch=src N-V-R might be easier. There's no way to add the /packages/ directory as a yum repository (nor would we want to encourage that). -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: art of packaging: fluxbox-pulseaudio
On 07/08/2009 02:55 PM, Andreas Bierfert wrote: On Wed, 08 Jul 2009 11:50:18 -0700 Jesse Keating jkeat...@redhat.com wrote: On Wed, 2009-07-08 at 20:44 +0200, Andreas Bierfert wrote: It is not only an empty file. It requires the necessary PA packages and triggers automatic PA loading on session starts. Please see [1] and [2] for some more background information. That subpackage should likely be noarch. Just been thinking the same. Will be fixed asap. While you're at it, could you convert that specfile (specifically the changelog) from iso8859-1 to utf-8? createrepo complains about non-ascii non-utf-8 encodings. Thanks, Mike -- fedora-devel-list mailing list fedora-devel-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-devel-list
Re: lower N-V-R used in buildroot
Dan Horák wrote: Hi, in koji for Fedora/s390x I have 2 builds for wxGTK package https://s390.koji.fedoraproject.org/koji/packageinfo?packageID=7367 wxGTK-2.8.9-4.fc11 wxGTK-2.8.10-1.fc11 the higher NVR build is older but the lower NVR is used in the buildroot used to build a package (like wxPython) that requires wxGTK https://s390.koji.fedoraproject.org/koji/buildrootinfo?buildrootID=32535 https://s390.koji.fedoraproject.org/koji/taskinfo?taskID=74417 Could someone explain me what's wrong? Koji uses the latest package in a tag to populate the repodata/buildroots, where latest equals most recently tagged. $ koji --server http://s390.koji.fedoraproject.org/kojihub list-tag-history --tag dist-f11 --package wxGTK Sun Apr 5 11:52:54 2009: Tagged wxGTK-2.8.10-1.fc11 with dist-f11 [still active] Thu Apr 9 19:08:22 2009: Tagged wxGTK-2.8.9-4.fc11 with dist-f11 [still active] wxGTK-2.8.9-4.fc11 was tagged most recently, so it's considered the latest. To fix the issue, either untag wxGTK-2.8.9-4.fc11 or untag-and-retag wxGTK-2.8.10-1.fc11 (which would then become the latest). This behavior is intentional and useful, but it can cause unexpected behavior like this if you don't realize how Koji populates buildroots. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: How to remove the large directory after build ?
李建 wrote: Hello,everybody! My koji server has setup ok. but the mock dir is more and more large. How to remove it ? I mean if koji has this feature,I doesn't to do it myself. Is anyone can help me ? my mock directory info : 651Mgtes11.2-build-10-9 18M gtes11.2-build-13-10 5.6Mgtes11.2-build-15-15 549Mgtes11.2-build-1-6 5.5Mgtes11.2-build-16-15 18M gtes11.2-build-18-18 18M gtes11.2-build-19-18 579Mgtes11.2-build-21-20 560Mgtes11.2-build-22-20 3.5Ggtes11.2-build-23-26 548Mgtes11.2-build-2-6 567Mgtes11.2-build-3-6 583Mgtes11.2-build-5-9 562Mgtes11.2-build-6-9 670Mgtes11.2-build-9-9 6.2Mgtes11.3-build-24-28 6.1Mgtes11.3-build-25-28 637Mgtes11.3-build-26-30 609Mgtes11.3-build-28-30 3.5Ggtes11.3-build-30-30 3.0Ggtes11.3-build-31-30 1.8Ggtes11.3-build-32-32 1.8Ggtes11.3-build-34-32 583Mgtes11.3-build-35-33 562Mgtes11.3-build-37-33 888Mgtes11.3-build-39-34 852Mgtes11.3-build-40-34 408Mmoblin2-build-41-38 417Mmoblin2-build-42-39 == kojid manages the cleanup up the /var/lib/mock directory for builds it runs. Buildroots for successful builds will be cleaned up almost immediately. Buildroots for unsuccessful builds will be cleaned up after 4 hours, to give you a chance to debug the failure or copy relevant data out of the buildroot for later analysis. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Which RPM does mergerepo prefer?
Jitesh Shah wrote: Hi, I have attached an external repo to a build tag (Just trying out some hand-built RPMs). Now, if an RPM is present *both* in the build tag as well as the external repo, which one does mergerepos pick? It picks the rpm in the build tag (the local rpm). Is the decision based on version comparison (the greater version wins), or does the one in the build tag always get preference over the external repo (irrespective of the version)? The one in the build tag always wins. This is consistent with normal Koji tag inheritance, which is a depth-first, first-match-wins search. Koji never compares NVRs. If it is the latter, how do I tell mergerepos to prefer external repo over the build tag? (or atleast use version comparisons rather than preferring build tag) If you want to use the version from the external repo, untag any builds of that package from the local tag and let the external repo version be picked up as the first match. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: ActionNotAllowed: admin permission required
Tom Stage wrote:
Hi all
Well I have to admit that I am in the same boat as the last thread with the
same Subject, this config is also with SSL configured and it seems to be ok
and running good, I can log in to the web interface.
I have Koji installed on a Fedora 10 x86_64 system, and I have followed the
HowTo at http://fedoraproject.org/wiki/Koji/ServerHowTo and I to cant seem
to execute the following commands as an example:
System info:
Uname -a
Linux koji 2.6.27.21-170.2.56.fc10.x86_64 #1 SMP Mon Mar 23 23:08:10 EDT
2009 x86_64 x86_64 x86_64 GNU/Linux
Rpm -qa
koji-builder-1.3.1-1.fc10.noarch
koji-utils-1.3.1-1.fc10.noarch
koji-1.3.1-1.fc10.noarch
koji-web-1.3.1-1.fc10.noarch
koji-hub-1.3.1-1.fc10.noarch
SSL certificates created after the instructions in the howto, with one
exception, since this is a single host installation I have only created 3
types of certificates. The 1st one is the signing certificate. The 2nd one
is the certificates for the host, and used by all the koji services. The 3rd
one is the user certificates.
[r...@koji ~]koji call getLoggedInUser
{'id': 1, 'krb_principal': None, 'name': 'admin', 'status': 0, 'usertype':
0}
[r...@koji ~]koji add-user kojira
ActionNotAllowed: admin permission required
[r...@koji ~]koji add-host koji.dvos.dk x86_64
ActionNotAllowed: admin permission required
My users in the users table in postgres looks like this:
Id namepasswordstatus usertypekrb_principal
1 admin 0 0
2 koji.dvos.dk0 1
My permissions table looks like this:
Id name
1 admin
2 build
3 repo
I am confused and don't understand what I am doing wrong, and I am willing
to post my configuration files as well.
Any help is appreciated.
To grant a permission to a user you need to insert into the user_perms
table. The user_id column references the id of the users table, and the
perm_id references the id of the permissions table. In your case,
granting the admin user the admin permission would be accomplished
by running:
insert into user_perms (user_id, perm_id) values (1, 1);
After that, you can grant other permissions by using the koji
grant-permission command.
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: One build for multiple platforms?
Steve Traylen wrote:
On Thu, May 14, 2009 at 8:08 PM, Mike Bonnet mi...@redhat.com wrote:
Steve Traylen wrote:
Hi,
I thinking that the answer is that it is not currently possible but is
there any arrangement of configuration
to allow a build on say centos4 and centos5 concurrently.
A build tartget that has a fork to two buildroots and destination
tags. Both would need to work
for the overall task to work.
If you mean one koji build results in two builds being created in Koji,
then no that is not currently possible.
This sounds like something that could easily be handled with a Makefile
target though.
Create your separate dist-centos4 and dist-centos5 build/dest tags and
targets. Have a make build generate the appropriate SCM URL and call:
koji build --nowait dist-centos4 $SCMURL
koji build --nowait dist-centos5 $SCMURL
This works of course. I was hoping to force the dists to stay
in step though. builds are only tagged for el5 and el4 or not at all.
Rather like currently both i386 and x86_64 must both work to get either.
I can force the submitters to build on fc10 as well el4 and 5 to future proof
ourself.
This could be accomplished with koji build --skip-tag. Then koji
watch-task on the two taskIDs, and if that returns 0 you can then koji
tag-pkg the two builds into their dest tags. It would probably be more
robust to code this up by using the xmlrpc api directly, so you can
check task success/failure and get the build NVR from the task ID
directly, rather than parsing the output of koji taskinfo.
Assuming you're using %{?dist} in your specfiles and have the
buildsys-macros defined correctly (and uniquely) in the -build tags, this
will create 2 separate builds in different tags from the same sources.
An item relevant to this. When building say a -el4.src.rpm on el5
koji build dist-centos5 foobar-1.5.2.el4.src.rpm
will always fail because the resulting foobar-1.5.2.el5.src.rpm does
not name match foobar-1.5.2.el4.src.rpm
Is this check useful? It requires one to create to src.rpms before they can
be submitted to dist-centos4 and 5. I can't for instance just grab a
src.rpm package
from fc10 and submit it.
We only do this check for real (non --scratch) builds, and yes it is
useful. The NVR of the build object in the database comes from that
srpm. It would be very confusing if a build called foo-1.0-1.el4
generated an rpm named foo-1.0-1.el5.
(a cheeky p.s, did you get the slides I sent. no comment is just fine)
Yes, haven't had a change to look them over yet though, I'll do that today.
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: ActionNotAllowed: policy violation - but why?
Steve Traylen wrote: On Fri, May 8, 2009 at 11:52 AM, Steve Traylen st...@traylen.net wrote: Hi, I was reliably building on a tag before but now receive ActionNotAllowed: policy violation A little bit more. ActionNotAllowed: policy violation -- all :: deny What version of Koji are you running? I believe there was a bug in earlier versions that caused a missing policy entry in /etc/koji-hub/hub.conf to result in denials of everything. That should be fixed in 1.3.1. Alternately you could add a policy entry to allow building from srpm into the dist-centos4 tag: [policy] build_from_srpm = tag dist-centos4 :: allow has_perm admin :: allow all :: deny Steve and can't seem shake it or understand why for a particular package Is is possible to get an explanation? http://skoji.cern.ch/koji/taskinfo?taskID=2918 This is following a build as CN=straylen koji build --nowait dist-centos4 ../SRPMS/mpich-1.2.7p1-2.el4.src.rpm My permissions. id | name | password | status | usertype | krb_principal +--+--++--+--- 1 | straylen | | 0 |0 | and user_id=1 does not appear in user_perms . i.e I am a boring user. The package has been added. koji list-pkgs --tag=dist-centos4 --package=mpich Package Tag Extra Arches Owner --- --- --- mpich dist-centos4 straylen Thanks again for the help. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: ActionNotAllowed: policy violation - but why?
Steve Traylen wrote: On Fri, May 8, 2009 at 4:04 PM, Mike Bonnet mi...@redhat.com wrote: Steve Traylen wrote: On Fri, May 8, 2009 at 11:52 AM, Steve Traylen st...@traylen.net wrote: Hi, I was reliably building on a tag before but now receive ActionNotAllowed: policy violation A little bit more. ActionNotAllowed: policy violation -- all :: deny What version of Koji are you running? I believe there was a bug in earlier versions that caused a missing policy entry in /etc/koji-hub/hub.conf to result in denials of everything. That should be fixed in 1.3.1. Alternately you could add a policy entry to allow building from srpm into the dist-centos4 tag: [policy] build_from_srpm = tag dist-centos4 :: allow has_perm admin :: allow all :: deny Hi Mike, I'm running stock FC10. koji-hub-1.3.1-1.fc10.noarch koji-1.3.1-1.fc10.noarch koji-web-1.3.1-1.fc10.noarch koji-utils-1.3.1-1.fc10.noarch koji-builder-1.3.1-1.fc10.noarch I've not had had any [policy] in the hub.conf file up to now and things have been okay, i.e I could build from cvs and svn for instance which I did yesterday. It's just building from srpm that has been blocked but I have not tried that in a while so that may have been the before the recent upgrade. Actually I was wrong, if no policy is present the default policy forbids building from srpm by anyone but an admin (this is for consistency with previous versions of Koji). But this is now overrideable with custom policy, whereas it was hard-coded before. Certainly making a very open policy [policy] build_from_srpm = tag dist-centos4 :: allow has_perm admin :: allow all :: all and then things proceed, I'll tune that now. More generally now about policies. Do you have some description on these and what can be set? If nothing exists if you can give me something brief then I'll try and write something up for the wiki. Unfortunately the policy stuff is not well-documented at the moment, but we're working on fixing that. It is actually a very powerful mechanism that allows you to control what types of source repositories you can build from, setup elaborate building and tagging policy, etc. We'll get some basic documentation onto the fedoraproject.org wiki soon, and any help expanding on it at that point would be appreciated. As it happens I'm giving a presentation in a weeks time to my colleagues on mock, koji and mash and certainly any content (e.g diagrams) I produce I'll write up in a generic way for inclusion in documentation. That'd be great! Thanks again Steve Steve and can't seem shake it or understand why for a particular package Is is possible to get an explanation? http://skoji.cern.ch/koji/taskinfo?taskID=2918 This is following a build as CN=straylen koji build --nowait dist-centos4 ../SRPMS/mpich-1.2.7p1-2.el4.src.rpm My permissions. id | name | password | status | usertype | krb_principal +--+--++--+--- 1 | straylen | | 0 |0 | and user_id=1 does not appear in user_perms . i.e I am a boring user. The package has been added. koji list-pkgs --tag=dist-centos4 --package=mpich Package Tag Extra Arches Owner --- --- --- mpich dist-centos4 straylen Thanks again for the help. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Make sources?
Brian Kosick wrote: Hi All, So I think that I have successfully upgraded koji from 1.2.6 to 1.3.1 by doing this. I just have a few issues to work out 1) install the koji 1.3.1 rpms 2) update the db schema pgsql -h kojihost koji koji /usr/share/doc/koji-1.3.1/docs/schema-upgrade-1.2-1.3.sql 3) started kojid and kojira When I did my first build, I got stuck with koji not having a mock group srpm-build I setup the mock group with koji add-group dist-el5-build srpm-build koji add-group-pkg dist-el5-build srpm-build pkg1 pkg2 pk3 I then added that group to my build dist with koji call addGroupList dist-mxl-el5-build srpm-build When I do a koji list-groups dist-mxl-el5-build i get koji list-groups dist-mxl-el5-build build [dist-mxl-el5-build] snip for brevity srpm-build [dist-el5-build] automake: None, default [dist-el5-build] bash: None, default [dist-el5-build] buildsys-macros: None, default [dist-el5-build] bzip2: None, default [dist-el5-build] bzip2-devel: None, default [dist-el5-build] coreutils: None, default [dist-el5-build] cpio: None, default [dist-el5-build] diffutils: None, default [dist-el5-build] elfutils: None, default [dist-el5-build] elfutils-libelf: None, default [dist-el5-build] file: None, default [dist-el5-build] gcc: None, default [dist-el5-build] gcc-c++: None, default [dist-el5-build] glibc: None, default [dist-el5-build] glibc-common: None, default [dist-el5-build] glibc-devel: None, default [dist-el5-build] glibc-headers: None, default [dist-el5-build] gzip: None, default [dist-el5-build] info: None, default [dist-el5-build] libselinux: None, default [dist-el5-build] libsemanage: None, default [dist-el5-build] libsepol: None, default [dist-el5-build] libtool-ltdl: None, default [dist-el5-build] make: None, default [dist-el5-build] patch: None, default [dist-el5-build] perl: None, default [dist-el5-build] policycoreutils: None, default [dist-el5-build] python: None, default [dist-el5-build] readline: None, default [dist-el5-build] readline-devel: None, default [dist-el5-build] redhat-release: None, default [dist-el5-build] redhat-rpm-config: None, default [dist-el5-build] rpm-build: None, default [dist-el5-build] rpm-libs: None, default [dist-el5-build] sed: None, default [dist-el5-build] selinux-policy: None, default [dist-el5-build] shadow-utils: None, default [dist-el5-build] sqlite: None, default [dist-el5-build] tar: None, default [dist-el5-build] unzip: None, default [dist-el5-build] which: None, default [dist-el5-build] zip: None, default [dist-el5-build] zlib-devel: None, default [dist-el5-build] Wow, that's way more in your srpm-build group than necessary. srpm building is a fairly simple process, and doesn't require a lot in the buildroot. For reference, this is the Fedora srpm-build group: $ koji list-groups dist-f12-build [snip] srpm-build [dist-f9] bash: None, default [dist-f9-build] curl: None, default [dist-f9-build] cvs: None, default [dist-f9-build] fedora-release: None, default [dist-f9-build] gnupg: None, default [dist-f9-build] make: None, default [dist-f9-build] redhat-rpm-config: None, default [dist-f9-build] rpm-build: None, default [dist-f9-build] shadow-utils: None, default [dist-f9-build] Of course, any dependencies of those packages will also be installed, but using a smaller base package set will make your buildSRPMFromSCM tasks complete significantly faster. It appears that the repos have regenned and now when I try to do a build, I'm getting DEBUG util.py:280: Executing command: ['make', 'sources'] DEBUG util.py:256: make: *** No rule to make target `sources'. Stop. DEBUG util.py:319: Child returncode was: 2 Has a new make command sources been created similar to make srpm? If so what does it do and what does koji expect back? Brian -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: mergerepos of f10 x86_64 release and updates does not contain perl or some other packages?
Steve Traylen wrote: Hi, Merging works fine for i386 f10 repositories but for x86_64 the following happens. The merged repository of f10's Everything and updates is created and then using this new repo mock tries to create and installroot. The problem is that this new repository is unable to provide some items, perl, /bin/bash and so the install root fails. Here are the details. The blocklist file following here is empty. $ /usr/libexec/kojid/mergerepos -a x86_64 \ -b /mnt/koji/repos/dist-f10-build/671/x86_64/blocklist \ -o /tmp/koji/tasks/2021/2021/repo \ -g /mnt/koji/repos/dist-f10-build/671/groups/comps.xml \ -r http://mirror.switch.ch/ftp/mirror/fedora/linux/updates/10/x86_64/ \ -r http://mirror.switch.ch/ftp/mirror/fedora/linux/releases/10/Everything/x86_64/os/ Adding repo: http://mirror.switch.ch/ftp/mirror/fedora/linux/updates/10/x86_64/ Adding repo: http://mirror.switch.ch/ftp/mirror/fedora/linux/releases/10/Everything/x86_64/os/ 1/12364 - openswan-doc-2.6.19-1.fc10.x86_64 2/12364 - ssm-0.1-12.fc10.x86_64 ... ... 1112/12364 - 4:perl-5.10.0-56.fc10.x86_64 ... ... 12363/12364 - libextractor-plugins-exiv2-0.5.20b-2.fc10.x86_64 12364/12364 - lwp-devel-2.4-1.fc10.x86_64 Saving Primary metadata Saving file lists metadata Saving other metadata Generating sqlite DBs Starting other db creation: Tue Mar 17 14:49:44 2009 Ending other db creation: Tue Mar 17 14:50:50 2009 Starting filelists db creation: Tue Mar 17 14:51:03 2009 Ending filelists db creation: Tue Mar 17 14:55:37 2009 Starting primary db creation: Tue Mar 17 14:55:40 2009 Ending primary db creation: Tue Mar 17 14:58:28 2009 Sqlite DBs complete The following is then tried from within mock. The chroot's yum.conf is definitely referencing the repository I just created (I've checked carefully and have recreated the repo a few times just to check this is consistent. ) /usr/bin/yum --installroot /var/lib/mock/dist-f10-build-13-671/root/ \ groupinstall build this results in EBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: /usr/bin/perl is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: perl(Getopt::Long) is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: /bin/sh is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: mktemp is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: fedora-release-notes-10.0.0-1.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: /bin/sh is needed by package fedora-release-notes-10.0.0-1.noarch (build) DEBUG util.py:256: redhat-rpm-config-9.0.3-3.fc10.noarch from build has depsolving problems DEBUG util.py:256:-- Missing Dependency: /bin/bash is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /bin/bash is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: mktemp is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /bin/sh is needed by package fedora-release-notes-10.0.0-1.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /usr/bin/perl is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: /bin/sh is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) DEBUG util.py:256: Error: Missing Dependency: perl(Getopt::Long) is needed by package redhat-rpm-config-9.0.3-3.fc10.noarch (build) And indeed. /usr/bin/yum --installroot /var/lib/mock/dist-f10-build-13-671/root list perl Error: No matching Packages to list There are lots of packages in this repository, but no perl it seems. /usr/bin/yum --installroot /var/lib/mock/dist-f10-build-13-671/root list | wc -l shows 3483 packages. Note the main box I am working on is 32 bit F10 box which may be relevant and the reason why only subsequently the i386 build works? You can't use a x86_64 chroot on a i386 box, nothing in the chroot will run. I suspect rpm is preventing installation of x86_64 packages on a i386 host. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: buildSRPMfromSCM issue
Jitesh Shah wrote: Hi all, I've been trying out some stuff on my own local koji instance. Recently, I added a new builder and all the builds using SCMs started to fail (although, scratch builds on SRPMs work just fine). buildSRPMfromSCM fails. The checkouts go through fine and make sources also works. Following is the tail of the root.log (after make sources) DEBUG util.py:251: 3 2266k 73 1670k0 0 239k 0 0:00:09 0:00:06 0:00:03 310k 87 2266k 87 1976k0 0 247k 0 0:00:09 0:00:07 0:00:02 310k 100 2266k 100 2266k0 DEBUG util.py:251: DEBUG util.py:251: 0 DEBUG util.py:251: DEBUG util.py:251: DEBUG util.py:251: DEBUG util.py:251: 253k 0 0:00:08 0:00:08 --:--:-- 307k DEBUG util.py:251: -rw-rw-r-- 1 mockbuild mockbuild 2321402 Jun 18 2007 libidn-0.6.14.tar.gz DEBUG util.py:312: Child returncode was: 0 DEBUG backend.py:484: umount -n /var/lib/mock/dist-f10-build-139-697/root/proc DEBUG util.py:273: Executing command: umount -n /var/lib/mock/dist-f10-build-139-697/root/proc DEBUG util.py:312: Child returncode was: 0 DEBUG backend.py:484: umount -n /var/lib/mock/dist-f10-build-139-697/root/sys DEBUG util.py:273: Executing command: umount -n /var/lib/mock/dist-f10-build-139-697/root/sys DEBUG util.py:312: Child returncode was: 0 DEBUG util.py:99: kill orphans end of root.log The task exits with FAILED: BuildError: error building srpm, mock exited with status 2; see root.log for more information But, there is nothing suspicious in root.log (yum succeeds, so does checkout and make sources and there are no errors after that too). state.log and kojid.log also give out nothing. I have enabled full debugging from kojihub.conf. That error means mock --buildsrpm failed. I'm not sure what status 2 is, it probably means a failure in rpmbuild in the chroot. There should be more information in root.log, I'm not sure why there isn't. Does building the srpm by hand work? -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: mergerepo fails with PCDATA invalid Char value 8
Steve Traylen wrote: On Sun, Mar 15, 2009 at 12:45 PM, Steve Traylen st...@traylen.net wrote: Hi, Got koji basically working for me over the last couple of weeks. Was very keen to try its new external repository support. Starting with a fresh instance I made a tag (dist-slc5) containing two repos. koji add-external-repo -t dist-slc5 -p 10 slc5-64-base http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os koji add-external-repo -t dist-slc5 -p 10 slc5-32-base http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os and then tried to make a koji repo from that. koji regen-repo dist-slc5 This called /usr/libexec/kojid/mergerepos -a i386 -b /mnt/koji/repos/dist-slc5-build/189/i386/blocklist -o /tmp/koji/tasks/556/556/repo \ -g /mnt/koji/repos/dist-slc5-build/189/groups/comps.xml -r http://linuxsoft.cern.ch/cern/slc5X/x86_64/yum/os/ \ -r http://linuxsoft.cern.ch/cern/slc5X/i386/yum/os/ resulting in as below. Any ideas ? To hopefully answer my own question. Is this because these slc yum repositories do not contain the sqlite files thats that mergerepo makes use of. Looking at CentOS and ScientificLinux neither of these look to make use of the '-d' option to createrepo to generate the sqlite files. Is there a way around this or we have to ask CentOS to generate the sql files? The error occurs when parsing other.xml. I would check your external repos to see if other.xml passes XML validation successfully. Of course maybe it is something else entirely? Steve Steve process:19630): GLib-WARNING **: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: Parsing other.xml error: PCDATA invalid Char value 8 Traceback (most recent call last): File /usr/libexec/kojid/mergerepos, line 241, in module main(sys.argv[1:]) File /usr/libexec/kojid/mergerepos, line 236, in main merge.write_metadata() File /usr/libexec/kojid/mergerepos, line 216, in write_metadata mdgen.doPkgMetadata() File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line 332, in doPkgMetadata self.writeMetadataDocs(packages) File /usr/lib/python2.5/site-packages/createrepo/__init__.py, line 475, in writeMetadataDocs clog_limit=self.conf.changelog_limit)) File /usr/lib/python2.5/site-packages/yum/packages.py, line 959, in xml_dump_other_metadata msg += %s\n/package\n % misc.to_unicode(self._dump_changelog(clog_limit)) File /usr/lib/python2.5/site-packages/yum/packages.py, line 927, in _dump_changelog if not self.changelog: File /usr/lib/python2.5/site-packages/yum/packages.py, line 423, in lambda changelog = property(fget=lambda self: self.returnChangelog()) File /usr/lib/python2.5/site-packages/yum/sqlitesack.py, line 225, in returnChangelog self._loadChangelog() File /usr/lib/python2.5/site-packages/yum/sqlitesack.py, line 202, in _loadChangelog self.sack.populate(self.repo, mdtype='otherdata') File /usr/lib/python2.5/site-packages/yum/yumRepo.py, line 184, in populate dobj = repo_cache_function(xml, csum) File /usr/lib/python2.5/site-packages/sqlitecachec.py, line 60, in getOtherdata self.repoid)) TypeError: Parsing other.xml error: PCDATA invalid Char value 8 Steve -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: how to build srpms on koji?
陈鲍孜 wrote: I checked the /var/log/kojid.log. it is said: 2009-03-14 17:03:14,371 [WARNING] koji.build.TaskManager: TRACEBACK: Traceback (most recent call last): File /usr/sbin/kojid, line 1275, in runTask response = (handler.run(),) File /usr/sbin/kojid, line 1351, in run return self.handler(*self.params,**self.opts) File /usr/sbin/kojid, line 2560, in handler for f in os.listdir(self.datadir): OSError: [Errno 2] No such file or directory: '/tmp/koji/tasks/230/230/repo/repoda 2009/3/15 Steve Traylen st...@traylen.net If your build tag has no builds associated or inherited, and no external repos configured, then no repodata will be created and the task will fail this way when it tries to upload the repodata. Tag a package into the build tag, or enable an external repo. On Sun, Mar 15, 2009 at 10:55 AM, 陈鲍孜 chenba...@gmail.com wrote: Hi, After long time fighting with koji configuration, I think the service is now running successfully. I added some srpms to it and saw its information on kojiweb. But it seems I can not build the srpms by koji (when I submit a build task to koji, it will open for a while, then it would finally come to be failed). I doubted if I have missed some configurations which connecting koji and mock. Does the build get as far as kojid. Is there anything in /var/log/kojid.log ? This can tell you the mock problem and then where to look for that. Steve -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Steve Traylen -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Password resets
Toshio Kuratomi wrote: Mike McGrath wrote: So holy crap does the planet hate it when you ask people to reset their passwords. In particular though, they hated the following: 1. Kittens 2. Password Expiration is confusing and does not imply account expiration. Some may have ignored the warning because they did not understand what the consequences were. 3. Mail aliases going away. This one's legit and accounts for the only data loss we actually had. 4. fedorapeople space going away and not coming back automatically. Possible implementation here: https://fedorahosted.org/fedora-infrastructure/ticket/1244#comment:1 5. Password resets could be introducing less secure passwords. This one's hard for me to quantify. If you use a strong password the first time, what's the likelihood that each reset will bring some number of users to use an insecure password? What's the likelihood of someone using an insecure password to use a more secure password next time (? This can be partially mitigated by using a password strength checker but it was pointed out to me that a strength checker 1) doesn't catch things like BIRTHDATE + WIFESNAME + FIRSTPET 2) Strength checkers often aren't as devious as someone trying to crack passwords. #2 is a bug in the strength checker but we're likely to have to continuously work on the upstream software in order to keep things secure. Without the reward of knowing how much security we're gaining. #1... I don't have a solution for. I'm going to disable password reset/account expiration until at least 3 of the 4 above are done. Please hate me a little less now. Thoughts? Would not doing a password expiration but just an account expiration be okay? I think that we can cover a pretty broad swathe of contributors with something that ties into people logging into fas (because we use json to log people in to web services including the wiki and they need to login to get a certificate to use koji/lookaside). We'd just have to expire accounts on a longer interval than the ssl certs... like 6 months for certs and 7 months for accounts. +1 Even if they were required to log in to the FAS web UI as an indication that their account was still active, I think that would be preferable to forced password resets. Thoughts on implementing alternate means of checking activity here: https://fedorahosted.org/fedora-infrastructure/ticket/1237 ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Re: koji 1.3.0
Brian Schubert wrote: Hi, Is there an easy way to convert the database from Koji 1.2.6 to work with this new release? Importing the initial packages took a very long time, and as such, starting with a fresh database is undesirable. Or has the import speed improved drastically with this release? There is an upgrade script (/usr/share/doc/koji-1.3.1/docs/schema-upgrade-1.2-1.3.sql) to upgrade an existing database the 1.3 format. Note that the external repos support may allow you to avoid an initial import altogether, in many cases. There is documentation forthcoming on how to setup external repos. In the meantime you can ask questions in #koji on Freenode. Thanks, -Brian Schubert Mike McLean wrote: The tarball is posted here: https://fedorahosted.org/koji/wiki/KojiRelease (created from koji-1.3.0 tag) Major Changes -- Support for external repos new cli commands createrepo tasks now have a higher weight Support for noarch subpackages Support larger hashes Handle different kinds of rpm signatures [mitr] (#127) support file digests other than md5 in the api and web UI Hub configuration via hub.conf use a hub config file instead of PythonOptions scan for handlers once at startup instead of each call now honors KojiDir Remove huge tables from db - rpmfiles, rpmdeps and changelog tables dropped - data pulled from rpms as needed Build srpms in chroots bump the weight of buildSRPMFromSCM Some web ui theming support SiteName option (the name that shows up in the title) move explicit image sizes to css make welcome message customizable Improved security add an auth token to defend against CSRF include the current time in the cookie some changes to the web UI to defend against XSS Hub policies configured in hub.conf enable verbose policy errors without full debug current policies: tag: controls tag, untag, move operations build_from_srpm: controls whether such builds are allowed build_from_repo_id: controls whether such builds are allowed Plugin support kojihub and kojid now have limited plugin support Other Changes - Python-2.6-isms use hashlib if available use subprocess instead of popen2 Builder improved task cleanup check the load average before taking a task make use of createrepo --update optional choose a better arch for noarch builds update the buildArch task weight based on the average duration of a build of the package set %distribution the same way we set %vendor and %packager cleanup before re-taking a freed or reassigned task indicate which log people should look in for build failures make use of --skip-stat configurable use same repo for all buildArch subtasks more fully honor topdir option Web UI show summary and description for rpms and builds group rpms more sensibly (make the build log links correct) remove the dirtyness indicator from the buildrootinfo page (never used) enable displaying of only the latest builds in a tag use ids instead of names in the urls fix the Watch logs feature in the web UI to work over SSL cache compiled Cheetah templates update the web UI to conform to XHTML 1.0 Transitional tasks page: rework view selector use kojiweb.publisher (new location) Hub NotifyOnSuccess option honor KojiDir option DisableNotifications option don't blow up when the database contains older base64 encoded task data make a latest link when a new repo completes add createdBefore= and createdAfter= parameters to listBuilds() fix LoginCreatesUser check in resetBuild use CANCELED state instead of FAILED report offline status if: db connection fails there are errors on startup preserve old extra_arches during package list updates Command line new subcommand: search new subcommand: regen-repo new subcommand: remove-tag show package filters correctly in taginfo allow list-tagged to query at an event added --non-latest to untag-pkg subcommand added --old-user flag to set-pkg-owner-global subcommand show buildroot info in rpminfo output show arch in list-buildroot output handle chain-build cases where the build tag is the same as the dest tag Utils don't start kojira by default (#96) fix timestamp checks when deleting repos package koji-gc added purge option to koji-gc added koji-shadow utility for shadow builds Changes related to shadow builds koji-shadow utility allow creation of repos from a specified event allow building from a specific repo id (subject to policy) Miscellaneous add a strict option to multiCall() handle empty multicalls sensibly return a placeholder object while in a multicall enable building Koji
Re: kojira repo generation
Thomas Hatch wrote: I keep having problems with it telling me the system is locked until I run a restart, but service kojid status keeps returning the same error service kojid status kojid dead but subsys locked kojid also seems to be dying but the logs yield no real data I think I have a problem in my configs: What is the output of openssl x509 -noout -subject -in /etc/pki/koji/kojibuilder1.pem The CN component needs to match the hostname you added with koji add-host, in your case koji.bcinfra.net. Also, that same certificate may not be used to authenticate any other services or users to the system. You can also run /usr/sbin/kojid --force-lock --verbose --fg as root to run kojid in the foreground and see what errors are reported. kojid.conf: [kojid] ; The number of seconds to sleep between tasks ; sleeptime=15 ; The maximum number of jobs that kojid will handle at a time ; maxjobs=10 ; The minimum amount of free space (in MBs) required for each build root ; minspace=8192 ; The directory root where work data can be found from the koji hub ; topdir=/mnt/koji ; The directory root for temporary storage workdir=/tmp/koji ; The directory root for mock mockdir=/var/lib/mock ; The user to run as when doing builds mockuser=kojibuilder ; The vendor to use in rpm headers ; vendor=Koji ; The packager to use in rpm headers ; packager=Koji ; The _host string to use in mock ; mockhost=koji-linux-gnu ; The URL for the xmlrpc server server=http://sunlight.pp.bcinfra.net/kojihub user=koji.bcinfra.net ; The URL for the packages tree pkgurl=http://sunlight.pp.bcinfra.net/pkg/packages ; A space-separated list of hostname:repository[:use_common] tuples that kojid is authorized to checkout from (no quotes). ; Wildcards (as supported by fnmatch) are allowed. ; If use_common is specified and is one of false, no, or 0 (without quotes), then kojid will not attempt to checkout ; a common/ dir when checking out sources from the source control system. Otherwise, it will attempt to checkout a common/ ; dir, and will raise an exception if it cannot. ;allowed_scms=scm.example.com:/cvs/example git.example.org:/example svn.example.org:/users/*:no ; The mail host to use for sending email notifications smtphost=sunlight.pp.bcinfra.net ; The From address used when sending email notifications from_addr=Koji Build System k...@koji.bcinfra.net ;configuration for SSL athentication ;client certificate cert = /etc/pki/koji/kojibuilder1.pem ;certificate of the CA that issued the client certificate ca = /etc/pki/koji/koji_ca_cert.crt ;certificate of the CA that issued the HTTP server certificate serverca = /etc/pki/koji/koji_ca_cert.crt On Thu, Feb 26, 2009 at 10:32 AM, Jeffrey Ollie j...@ocjtech.us wrote: On Thu, Feb 26, 2009 at 11:29 AM, Thomas Hatch thatc...@gmail.com wrote: I run koji list-hosts --channel=createrepo and get: Hostname Enb Rdy Load/Cap Arches Last Update koji.bcinfra.net Y N0.0/8.0 i386,x86_64 - Seems it is enabled and in the channel, but not ready? Is kojid running? That's the service that does the actual building... -- Jeff Ollie Marcus to Franklin in Babylon 5: A Late Delivery from Avalon -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: attribute errors when koji login
陈鲍孜 wrote: Hello, After fighting with my carelessness, I finished configuring koji and made it run. Thanks a lot. Now, here comes another problem. When I click the login link on the kojiweb, it returns an error such as below: Traceback (most recent call last): File /usr/lib64/python2.4/site-packages/mod_python/apache.py, line 299, in HandlerDispatch result = object(req) File /usr/lib64/python2.4/site-packages/mod_python/publisher.py, line 213, in handler published = publish_object(req, object) File /usr/lib64/python2.4/site-packages/mod_python/publisher.py, line 412, in publish_object return publish_object(req,util.apply_fs_data(object, req.form, req=req)) File /usr/lib64/python2.4/site-packages/mod_python/util.py, line 439, in apply_fs_data return object(**args) File /usr/share/koji-web/scripts/index.py, line 155, in login _redirectBack(req, dest, forceSSL=True) File /usr/share/koji-web/scripts/index.py, line 134, in _redirectBack page = _getBaseURL(req) + '/' + page File /usr/share/koji-web/scripts/index.py, line 117, in _getBaseURL return req.construct_url(base) AttributeError: 'mp_request' object has no attribute 'construct_url' I searched google. There are some results, but still not clear for me. Is it a bug or something due to some wrong configuration? I was run koji on CentOS with EPEL packages. What version of Koji are you using? That error has been fixed in Koji git for a while, and is available in the latest release, Koji 1.3.1. You'll need to run it on CentOS 5. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] Add compatibility back in for older versions of Yum.
James Antill wrote:
On Tue, 2009-02-17 at 10:29 -0600, Jeffrey C. Ollie wrote:
We would like for Koji to remain compatible with the version of Yum
that is shipped with RHEL 5.
Signed-off-by: Jeffrey C. Ollie j...@ocjtech.us
---
builder/mergerepos | 11 +++
1 files changed, 7 insertions(+), 4 deletions(-)
diff --git a/builder/mergerepos b/builder/mergerepos
index defb8c1..cfa3a8d 100755
--- a/builder/mergerepos
+++ b/builder/mergerepos
@@ -95,10 +95,13 @@ class RepoMerge(object):
self.mdconf.verbose = True
self.mdconf.changelog_limit = 3
self.yumbase = yum.YumBase()
-self.yumbase.preconf.fn = '/dev/null'
-self.yumbase.preconf.init_plugins = False
-self.yumbase.preconf.debuglevel = 2
-self.yumbase._getConfig()
+if hasattr(self.yumbase, 'preconf'):
+self.yumbase.preconf.fn = '/dev/null'
+self.yumbase.preconf.init_plugins = False
+self.yumbase.preconf.debuglevel = 2
+self.yumbase._getConfig()
This line should be deleted, the first line after the if will turn the
preconf into the conf.
Thanks, I've made that change.
+else:
+self.yumbase._getConfig('/dev/null', init_plugins=False,
debuglevel=2)
self.yumbase.conf.cachedir = tempfile.mkdtemp()
self.yumbase.conf.cache = 0
self.archlist = arches
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji feature proposals
On Tue, 2009-01-06 at 21:51 +0100, Oliver Falk wrote: Hi Mike! Mike Bonnet schrieb: I've just created tickets for a few Koji features that I've been wanting to implement for a while (as well as updated an old one), and I'm planning to devote some time to in the near future. If you have any comments on these features feel free to post to the tickets, or talk to me at FUDCon this weekend. Just figured people might want to see the direction that Koji is headed. The future is now! :) [ ... ] drop the rpmfiles and rpmdeps tables: https://fedorahosted.org/koji/ticket/124 -1 Only if you provide the same functionality using another approach :-) Yes, the plan is to query the information directly from the rpms rather than from the database. The content on the rpminfo page in the web UI should not change at all from the user perspective. *I* do use it quite often; Find out which file belongs to which pkg(s). However, this was quite slow and now doesn't even seem to work in koji.fpo. :-( Hmmm, it should be. In what way is it not working? noarch subpackage support: https://fedorahosted.org/koji/ticket/125 Duh? We do have already (at least one) packages that build arch-specific and noarch pkgs - kernel or do we use some *hack* in the kernel.spec? We use a hack in the kernel specfile and in the build system. The noarch subpackage support in rpm is much more generic and flexible, and we need to support it without build system hacks. And regarding your point: '... different arches build noarch subpackage with different contents'. Well, then it's definitly not *noarch*, is't it? :-) True, but it's still possible, and we may need to check for this case and handle is appropriately (possibly by failing the build). -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji feature proposals
On Tue, 2009-01-06 at 15:21 -0600, Jason L Tibbitts III wrote: OF == Oliver Falk oli...@linux-kernel.at writes: OF And regarding your point: '... different arches build noarch OF subpackage with different contents'. Well, then it's definitly not OF *noarch*, is't it? :-) It is quite possible for the contents to differ by, say, date, or by timestamps being included in plain text output. Why would that render the output arch-specific? I'm not so much worried about that level of difference as I am of say different file lists from noarch rpms built on different hosts, or maybe different endianness of data files. There is some set of post-build checks we may want to run on these noarch subpackages to ensure they are in fact noarch, and that their content is sane. This noarch subpackage feature is new, and there are things Koji can and probably should be doing to make sure it's being used correctly. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Supporting EPEL Builds in Koji
Picking up this thread again, sorry about the long delay. I'd like to come to consensus on the approach here, hammer out any remaining details at FUDCon this weekend, and hopefully get this implemented by the end of January. Time to really get rid of plague! On Mon, 2008-10-06 at 15:14 -0400, Mike McLean wrote: Mike Bonnet wrote: On Fri, 2008-07-18 at 11:38 -0400, Mike McLean wrote: Mike Bonnet wrote: On Thu, 2008-07-17 at 13:54 -0400, Mike McLean wrote: If the remote_repo_url data is going to be inherited (and I tend to think it should be), then I think it should be in a separate table. ... I don't have any problem with this, though it does mean we'll need to duplicate quite a bit of the inheritance-walking code, ... Walking inheritance is just a matter of determining the inheritance order and scanning data on the parent tags in sequence. ... Sorry, I was referring to walking tag_inheritance. I'd rather have one place that walks the inheritance hierarchy and aggregates data from it, than two places that are doing almost the same thing. We're talking about inherently different data. External repos to be merged in are quite different from builds in the system. Yes, I see the issue here. Since remote repos won't have their packages filtered out (by mergerepo) until after all packages in the local inheritance hierarchy are placed in the repo, they don't really follow the existing inheritance rules. Ok, you've convinced me. A separate table that stores a priority-ordered list of remote repos associated with each tag will probably be easier to manage. The lists will be aggregated when walking the tag hierarchy and passed to mergerepo in (priority, inheritance) order for proper filtering (based on srpm name, first match wins). Each tag has a set of builds associated with it. We walk the inheritance hierarchy, aggregating the builds from each tag in the hierarchy into a flat list, and then pass that list to createrepo. We would do essentially the same thing for external repos. When walking the hierarchy, if a tag has an external repo associated with it, we would append that repo url to a flat list, and pass that list to mergerepo. In both cases we're working with collections of packages that are associated with a tag, just in different formats. Sure, we can do this with one call to readFullInheritance, and traverse both the build table and external repo table from the given order. Yes, that makes sense. In discussing this with Jesse, I think we want external repos to be inherited. This is probably the easiest way to deal with having multiple external repos getting pulled in to a single buildroot, which is essential for Fedora (think F9 GA and F9 Updates). The idea was that, by convention, we would have external-repo-only tags, with only a single external repo associated with it and no packages/builds associated. These external-repo-only tags could then be inserted into the build hierarchy where appropriate. An ordered list of external repos could then be constructed by performing the current depth-first search of the inheritance hierarchy. The ordered list would then be passed to mergerepo, which would ensure that packages in repos earlier in the list supersede packages (by srpm name) in repos later in the list. This would preserve the first-match-wins inheritance policy that Koji currently implements, and that admins expect. For example: dist-custom-build ├─dist-custom └─dist-f9-updates-external └─dist-f9-ga-external would result mergerepo creating a single repo that would only contain packages from dist-f9-ga-external if they did not exist in the Koji-generated repo (dist-custom-build + dist-custom), dist-f9-updates-external, or the blacklist of blocked packages. This is consistent with how Koji package inheritance currently works, and I think is the most intuitive approach. It is similar, but different in potentially confusing ways. External repos do not have build structure, so we can't really have the same sort of inheritance behavior with a combination of external repo tags and normal tags. We order the external repos in inheritance order, but ultimately those repos are merged with the internal one in a way that does not honor inheritance in the way that the admin might expect. Using tags to represent external repos fails intuition because external repos are very much not like tags. When we get to supporting external koji systems, we can do something like this, but for external repos the bolted-on nature needs to be clear. This is why I'd prefer to have the data a little more removed. Ok, we're agreed on this. I see all that, and I'm almost convinced. The flipside is that by default all the code will treat these external rpms the same as the local ones, which will not be correct for a number of cases. Personally I'd
Re: caching packages on koji builder
On Wed, 2008-11-05 at 10:55 +0100, Dan Horák wrote: Hello, is it possible to somehow cache the packages that are going from koji hub to the builder to create a build root? There is no NFS connection between the builder and hub in s390's koji and so each build requires to download 100 MB via XMLRPC(?) and this is slow. Will squid help here? Koji builders have never downloaded packages via XMLRPC. All downloading is done by mock/yum, via http (previously nfs). You could potentially use squid locally to cache downloaded packages. You'd configure pkgurl in koji.conf to point to your local squid instance at http://localhost:8080/koji/packages; or something similar, and configure squid to pull from the actual http location where /mnt/koji/packages is being served. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: caching packages on koji builder
On Wed, 2008-11-05 at 09:49 -0600, Jason L Tibbitts III wrote: MB == Mike Bonnet [EMAIL PROTECTED] writes: MB Koji builders have never downloaded packages via XMLRPC. All MB downloading is done by mock/yum, via http (previously nfs). Well, mock can cache all sorts of things these days. If there are multiple builders at one location then having a single squid cache for them all might be nice, but mock's caching would still help to avoid having to hit the network. Actually mock's caching doesn't really help us. It's all done per-buildroot, and since every build is run in a different buildroot, the caches would never be reused. For this reason Koji disables caching in the mock configs it writes out. A global (per-machine) rpm cache might be useful for reducing network bandwidth. However, because mock/yum would have to lock this global cache while interacting with it, it would become a bottleneck when running concurrent builds. The best approach currently is probably a local squid cache. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Supporting EPEL Builds in Koji
On Fri, 2008-07-18 at 11:38 -0400, Mike McLean wrote: Mike Bonnet wrote: On Thu, 2008-07-17 at 13:54 -0400, Mike McLean wrote: If the remote_repo_url data is going to be inherited (and I tend to think it should be), then I think it should be in a separate table. I'd like to reserve tag_config for data that is local to individual tags. This will also make it easier to represent multiple remote repos. I don't have any problem with this, though it does mean we'll need to duplicate quite a bit of the inheritance-walking code, or make it configurable as to which inheritance it's walking. This new table would also have to be versioned, the same way the tag_config table is. Walking inheritance is just a matter of determining the inheritance order and scanning data on the parent tags in sequence. Currently, nothing scans tag_config in this way because no data in tag_config is inherited. (Well, in a sense tag_changed_since_event() does walk tag_config, but that's a little different.) Sorry, I was referring to walking tag_inheritance. I'd rather have one place that walks the inheritance hierarchy and aggregates data from it, than two places that are doing almost the same thing. Each tag has a set of builds associated with it. We walk the inheritance hierarchy, aggregating the builds from each tag in the hierarchy into a flat list, and then pass that list to createrepo. We would do essentially the same thing for external repos. When walking the hierarchy, if a tag has an external repo associated with it, we would append that repo url to a flat list, and pass that list to mergerepo. In both cases we're working with collections of packages that are associated with a tag, just in different formats. We need to figure out how we'll deal with multiplicity for the external repos. If tag A uses repo X and inherits from tag B which uses repo Y, then does tag A use both X and Y, or does the X entry override it? A (+repo X) +- B (+repo Y) My inclination is that it should override, because I think we'll want some way to do override that that mechanism seems easiest. In discussing this with Jesse, I think we want external repos to be inherited. This is probably the easiest way to deal with having multiple external repos getting pulled in to a single buildroot, which is essential for Fedora (think F9 GA and F9 Updates). The idea was that, by convention, we would have external-repo-only tags, with only a single external repo associated with it and no packages/builds associated. These external-repo-only tags could then be inserted into the build hierarchy where appropriate. An ordered list of external repos could then be constructed by performing the current depth-first search of the inheritance hierarchy. The ordered list would then be passed to mergerepo, which would ensure that packages in repos earlier in the list supersede packages (by srpm name) in repos later in the list. This would preserve the first-match-wins inheritance policy that Koji currently implements, and that admins expect. For example: dist-custom-build ├─dist-custom └─dist-f9-updates-external └─dist-f9-ga-external would result mergerepo creating a single repo that would only contain packages from dist-f9-ga-external if they did not exist in the Koji-generated repo (dist-custom-build + dist-custom), dist-f9-updates-external, or the blacklist of blocked packages. This is consistent with how Koji package inheritance currently works, and I think is the most intuitive approach. Also, I think we'll probably want to allow multiple external repos per tag, something which will be much easier to represent in an external table. We can include an explicit priority field to make a sane uniqueness condition (and to provide a clear ordering for the repo merge). As outlined above, I'd prefer to keep it to one external repo per tag, along with repo inheritance. I think this is easier from a management perspective, and more consistent with the way Koji currently works. Ordering for mergerepo will be represented by the location of the tag in the inheritance hierarchy. With a 1-to-1 tag-external repo mapping, it then makes sense to store the external repo url in the tag_config table. The big win here is that the methods and tools that query rpminfo for information about what was present in the buildroot at build time -snip- I see all that, and I'm almost convinced. The flipside is that by default all the code will treat these external rpms the same as the local ones, which will not be correct for a number of cases. Obviously, part of this will involve changing code to behave differently for the external ones, I'm just worried about how much we might have to change, or what we might miss. Personally I'd prefer adding a few special cases to the existing code, rather than maintain a whole heap of almost-but-not-quite-the-same code to manage external rpms. I think
Re: [PATCH] make authtype a config option
On Mon, 2008-08-11 at 22:54 -0500, Dennis Gilmore wrote: the attached patch adds a config option that can be in a config file or on the command line forcing the use of one authentication type. it is useful if a hub supports more than one authentication type. or using different hubs that support different authentications methods. Ive tested with noauth, kerberos, and ssl. Applied. Thanks for the patch! -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji CLI Auth problem
On Wed, 2008-07-16 at 11:06 +0800, Linul wrote:
HI:
I'm using CentOS 5.2 for my Koji Server, but now I have a problem
about Koji CLI auth.
According the wiki document in
http://fedoraproject.org/wiki/Koji/ServerHowTo , I setup my Koji-hub、
Koji-web、postgresql
, and have a koji web interface.
I also setup my CA Center，and configure the kojiweb.conf、
kojihub.conf、/etc/koji.conf.
But when i execute the koji command with no username and password, the
messages is：
Error: [('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL
routines', 'SSL_CTX_use_PrivateKey_file', 'PEM lib')]
Your client certificate file (indicated by cert in the config file)
needs to contain both the certificate and private key. Your private key
is missing.
why?
thanks.
/etc/koji.conf：
[koji]
;configuration for koji cli tool
;url of XMLRPC server
;server = http://koji.fedoraproject.org/kojihub
server = http://koji.ossii.com.tw/kojihub
;url of web interface
;weburl = http://koji.fedoraproject.org/koji
weburl = http://koji.ossii.com.tw/koji
;url of package download site
;pkgurl = http://koji.fedoraproject.org/packages
pkgurl = http://koji.ossii.com.tw/packages
;path to the koji top directory
topdir = /mnt/koji
;configuration for SSL athentication
;client certificate
;cert = ~/.fedora.cert
cert = /etc/kojid/kojiadmin.crt
;certificate of the CA that issued the client certificate
;ca = ~/.fedora-upload-ca.cert
ca = /etc/kojid/kojiadmin.key
;certificate of the CA that issued the HTTP server certificate
;serverca = ~/.fedora-server-ca.cert
serverca = /etc/httpd/conf.d/ssl/ossiikojica.crt
kojihub.conf：
Directory /usr/share/koji-hub
SetHandler mod_python
PythonHandler kojixmlrpc
PythonOption DBName koji
PythonOption DBUser kevin
PythonOption DBHost 127.0.0.1
PythonOption KojiDir /mnt/koji
# Kerberos auth configuration
# PythonOption AuthPrincipal [EMAIL PROTECTED]
# PythonOption AuthKeytab /etc/koji.keytab
# PythonOption ProxyPrincipals [EMAIL PROTECTED]
# format string for host principals (%s = hostname)
# PythonOption HostPrincipalFormat compile/[EMAIL PROTECTED]
# end Kerberos auth configuration
# SSL client certificate auth configuration
# the client username is the common name of the subject of
their client certificate
PythonOption DNUsernameComponent CN
# separate multiple DNs with |
# PythonOption ProxyDNs /C=US/ST=Massachusetts/O=Example
Org/OU=Example User/CN=example/[EMAIL PROTECTED]
PythonOption ProxyDNs /C=TW/ST=Taiwan/O=OSSII/OU=Koji Hub
Server/CN=OSSII Koji Server CA/[EMAIL PROTECTED]
# end SSL client certificate auth configuration
PythonOption LoginCreatesUser On
PythonOption KojiWebURL http://koji.ossii.com.tw/koji
# The domain name that will be appended to Koji usernames
# when creating email notifications
PythonOption EmailDomain example.com
# PythonOption KojiDebug On
# PythonOption KojiTraceback extended
# sending tracebacks to the client isn't very helpful for
debugging xmlrpc
PythonDebug Off
# autoreload is mostly useless to us (it would only reload
kojixmlrpc.py)
PythonAutoReload Off
/Directory
# uncomment this to enable authentication via SSL client certificates
Location /kojihub
SSLOptions +StdEnvVars
/Location
# these options must be enabled globally (in ssl.conf)
SSLVerifyClient require
SSLVerifyDepth 10
kojiweb.conf：
Alias /koji /usr/share/koji-web/scripts/
Directory /usr/share/koji-web/scripts/
# Config for the publisher handler
SetHandler mod_python
PythonHandler mod_python.publisher
# General settings
PythonDebug On
PythonOption KojiHubURL http://koji.ossii.com.tw/kojihub
PythonOption KojiWebURL http://koji.ossii.com.tw/koji
PythonOption KojiPackagesURL
http://koji.ossii.com.tw/koji/packages
PythonOption WebPrincipal koji/[EMAIL PROTECTED]
PythonOption WebKeytab /etc/httpd.keytab
PythonOption WebCCache /var/tmp/kojiweb.ccache
PythonOption WebCert /etc/httpd/conf.d/ssl/kojiweb.crt
PythonOption ClientCA /etc/httpd/conf.d/ssl/kojiweb.key
PythonOption KojiHubCA /etc/httpd/conf.d/ssl/ossiikojica.crt
PythonOption LoginTimeout 72
# This must be changed before deployment
PythonOption Secret CHANGE_ME
PythonPath sys.path + ['/usr/share/koji-web/lib']
PythonCleanupHandler kojiweb.handlers::cleanup
PythonAutoReload Off
/Directory
Location /koji/login
SSLOptions +StdEnvVars
/Location
# these options must be enabled globally (in ssl.conf)
SSLVerifyClient require
SSLVerifyDepth 10
Alias /koji-static/ /usr/share/koji-web/static/
Directory /usr/share/koji-web/static/
Options None
AllowOverride None
Re: [Fwd: Cron [EMAIL PROTECTED] /var/lib/pgsql/vacstat.py check]
On Wed, 2008-07-09 at 11:33 -0700, Toshio Kuratomi wrote: Since the plan is to move koji to db3 within the week I'd like to hold off on this. The dump and reload to move to the new server should be more effective than a manual vacuum. Note that you still need to analyze all the tables after a dump/reload to regenerate database statistics. Otherwise postgres may choose some less-than-optimal query plans. -Toshio Original Message Subject: Cron [EMAIL PROTECTED] /var/lib/pgsql/vacstat.py check Date: Wed, 9 Jul 2008 13:20:05 GMT From: [EMAIL PROTECTED] (Cron Daemon) To: [EMAIL PROTECTED] Traceback (most recent call last): File /var/lib/pgsql/vacstat.py, line 650, in ? Commands[command](opts) File /var/lib/pgsql/vacstat.py, line 150, in test_all test_transactions(opts) File /var/lib/pgsql/vacstat.py, line 147, in test_transactions raise XIDOverflowWarning, '\n'.join(overflows) __main__.XIDOverflowWarning: Used over half the transaction ids for koji. Please schedule a vacuum of that entire database soon: sudo -u postgres vacuumdb -zvd koji ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Re: Disk Space Issues - cvs-int/build hosts - (includes general note on Nagios Disk notifications)
On Tue, 2008-05-13 at 08:58 -0500, Dennis Gilmore wrote: On Tuesday 13 May 2008, Nigel Jones wrote: Hey guys, I seem to have deleted the nagios notification that I want to mention in particular, but as I noted in SOP/Nagios the %ages noted in the e-mails are what is left, so inode=99% doesn't mean that 99% of the inodes are used, it means 99% are still free. Anyway, what this means, is that when nagios has been complaining about cvs-int recently, in particular the fact that /git has reached WARNING. After a bit of hunting around, I found /repo/pkgs using 168GiB of the 192GiB available, which is understandable, Fedora has got huge. Problem here, is that there are a LOT of old tarballs in that folder, which leaves me wondering if we should do a spring clean ~1 mo after release. snip We already have plans to move the lookaside cache to the netapp. at least that was the last plan i was aware of. Diskspace isn't cheap, so I like delete old tarballs, I also like this option because it's not like they disappear completely, they should be in the src.rpm's already on archive.fp.o and if we accidentally delete 1 or 2 that are still needed, well grab it from src.rpm... This leads on to my second item... xenbuilder2 has run out of diskspace in /, it's down to 32M, thankfully koji has disabled it so it's safe for now, but wouldn't it be nice to throw say a 50GB partition dedicated solely for /var/lib/mock /mnt/build? Yes, yes, I know money, but once again, builds are getting bigger so 'it'd be nice'. xenbuilder1 and hammer2 are the oldest boxes we have in the buildsystem. closely followed by ppc1 xenbuilder2 is also quite an old box now. im cleaning up /var/lib/mock on xenbuilder2 now. failed builds dont get cleaned up automatically. we probably should reap them more often. Buildroots for failed Koji builds do get cleaned up after 4 hours (to give someone time to diagnose the problem if desired). However, xenbuilder2 is also running plague, which never cleans up buildroots as far as I know. ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Re: koji clone-tag?
On Mon, 2008-04-21 at 20:54 -0400, Doug Chapman wrote: We are working on building a beta release of our ia64 F9 bits. I have been digging through the wiki for general release engineering processes and found this: http://fedoraproject.org/wiki/ReleaseEngineering/SOP/FreezingRawhide?highlight=%28ReleaseEngineering/SOP/%29 which shows using the command: koji clone-tag dist-f8 f8-final however, I find that the clone-tag command does not actually exist. Was it replaced by something else? We are looking for a way to take the latest version of all packages and tag them with a tag which will be used to build the beta release. It does exist, what version of Koji are you using? It requires admin permissions, so it'll only show up in koji help --admin. For usage, use koji clone-tag --help. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji hidden packages proposal
On Fri, 2008-03-14 at 19:55 -0500, Dennis Gilmore wrote: On Friday 14 March 2008, Mike Bonnet wrote: I've written up a brief proposal about how hidden packages may be supported in Koji. The objective of this is to enable building EPEL packages in Koji. I wrote this up fairly quickly, and I'm sure I haven't thought through all the issues, but I wanted to get the ball rolling. Let me know if you have any questions/comments/ideas/issues relating to this proposal. http://people.redhat.com/mikeb/koji/koji-hidden-packages-proposal.html Thanks, Mike the tree will have to be nothing like /mnt/koji/packages instead it will have to be like http://download.fedora.redhat.com/pub/fedora/linux/ so that you can use rsync and repo mirroring tools to sync the content and keep trees in sync. ill leave it up to Seth to explain more but hit micro repository option he is working on would allow us to pull the existing repodata intothe new repodata. I'm not really sure what you're talking about. *No one* will be mirroring the hidden packages in the case we're talking about (building EPEL in Koji), these will be RHEL binaries, and only available to the builders. We really only need to have a command that can suck the repodata into the database. so we know about the packages. We have a command that can suck data about packages in to the database. It's koji import. The proposal is designed to work within the framework of that we already have in Koji, to try to minimize the number of changes and thus the time it'll take to implement. The micro repository option Seth has talked about sounds interesting, but it doesn't really help with the issue of making packages available to Koji builders without making them available to the world. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Koji hidden packages proposal
On Fri, 2008-03-14 at 19:17 -0600, Stephen John Smoogen wrote: On Fri, Mar 14, 2008 at 6:55 PM, Dennis Gilmore [EMAIL PROTECTED] wrote: On Friday 14 March 2008, Mike Bonnet wrote: I've written up a brief proposal about how hidden packages may be supported in Koji. The objective of this is to enable building EPEL packages in Koji. I wrote this up fairly quickly, and I'm sure I haven't thought through all the issues, but I wanted to get the ball rolling. Let me know if you have any questions/comments/ideas/issues relating to this proposal. http://people.redhat.com/mikeb/koji/koji-hidden-packages-proposal.html Thanks, Mike the tree will have to be nothing like /mnt/koji/packages instead it will have to be like http://download.fedora.redhat.com/pub/fedora/linux/ so that you can use rsync and repo mirroring tools to sync the content and keep trees in sync. ill leave it up to Seth to explain more but hit micro repository option he is working on would allow us to pull the existing repodata intothe new repodata. We really only need to have a command that can suck the repodata into the database. so we know about the packages. Another big issue will be that EL is split in so many 'interesting' ways. You need to be able to either have the build system know of every seperate channels from RHN for each variation EL-3,4,5 or have RHN somehow give a conglomerate channel to you of all the packages in one big bucket. The code from DAG's mrepo sort of does this but would need some work on getting it to play nicely We're not going to be pulling packages straight from RHN, we'll be manually importing the packages. The proposal allows for multiple alternate directory trees, so it will be possible to have a tree to which we import the RHEL-4 binaries, a tree for RHEL-5 binaries, etc. When a new RHEL update is released, it'll be up to an admin to download the new packages and import them into the proper tree. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Koji hidden packages proposal
I've written up a brief proposal about how hidden packages may be supported in Koji. The objective of this is to enable building EPEL packages in Koji. I wrote this up fairly quickly, and I'm sure I haven't thought through all the issues, but I wanted to get the ball rolling. Let me know if you have any questions/comments/ideas/issues relating to this proposal. http://people.redhat.com/mikeb/koji/koji-hidden-packages-proposal.html Thanks, Mike -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: another issue to fix with the FAS2 switch: Kojis ssl certificate
On Tue, 2008-03-11 at 21:52 +0100, Till Maas wrote: On Tue March 11 2008, Dennis Gilmore wrote: On Tuesday 11 March 2008, Till Maas wrote: Hiyas, now that everyone needs to change his password, can we now also deploy the new certifcate for koji? This will make it possible to verify whether or not one can trust the certificate for koji and the ticket[1] is now 7 months old, i.e. about a full Fedora release cycle. Therefore I guess there won't be a better time than now. Regards, Till [1] https://fedorahosted.org/fedora-infrastructure/ticket/88 No, Because it will break user certs. To make it work would require that users all get entirely new server cert files. We need to redo our entire CA system. We also need to consider the ramifications for Secondary arches, deploying a new CA would require each and every Secondary arch to purchase a cert from the same CA. or somebody to purchase a cert that covered *.koji.fedoraproject.org from the same CA. we are looking at deploying the hub on a separate box from the frontend which would allow us to do what you are wanting but would not look after secondary arches. How about making the hub (I assume this is only used by automated processes and not manually) listen on a different port than 443? Then the web interface could use the new well know certificate. The automated processes the internal ones, where imho using a own ca does not hurt. Also using a different port should be only a matter of configuring it once. The secondary arch instances could then use a cacert[0] certificate, which are free and are trusted by some browsers already for the web interface. The Koji cli communicates with the hub for all operations, so it would require everyone to update their Koji config. In addition, I'm sure running ssl on a non-standard port would mess with some people's proxy/firewall setups. I don't think this is the best solution. ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Re: rebuilding from old cvs tags
On Wed, 2008-02-27 at 16:15 -0600, Matt Domsch wrote: On Tue, Feb 26, 2008 at 06:33:12PM -0600, Dennis Gilmore wrote: Ive considered the idea of having a web app to make srpms on demand for people also. It would most likely be needed there also. But it doesnt help with the historical data we dont have. I started the 'correspondingsource' project on fedorahosted.org today exactly for such a webapp. We have just over 85k tags for all the packages in CVS; some going back to 2004, some only to the F7 days. Not sure yet how much history was lost during the Core import. We've talked about immutable tags before. While that would be nice, I'd be fine with having koji add a tag in its own namespace, either at package checkout pre-build, or on succcessful build, either way is fine by me. These tags would not be the same tags as 'make tag' creates, so users can force-tag if they really feel the need, but we would more than strongly discourage force tagging on the koji namespace tags. I'm not really comfortable having an automated system (Koji) modifying the SCM. Giving the Koji builders credentials to modify the SCM in a secure way (without making those credentials available to the world) might be tricky. It also dramatically increases the risk in the event that a builder is compromised. Right now all Koji access to the SCM is read-only, and it should probably stay that way. I think immutable tags are the answer here. We already kind of assume tags don't change once they're built, we might as well enforce it. I know force-tag is convenient, but how much harder is it really to bump the revision number instead? -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: rebuilding from old cvs tags
On Thu, 2008-02-28 at 11:58 -0500, Bill Nottingham wrote: Mike Bonnet ([EMAIL PROTECTED]) said: If a transient build error (build system hiccup) prevented the build from completing, you should be able to rebuild from the same tag. Yes, I know there are some failure modes where this does not work, but we're working on addressing those, and they should be the minority of cases. True, but if it's something like 'a build dependent package is broken', you're unlikely to remember to just re-push the task two days later when it's fixed. I don't see how disabling force-tagging has any effect on this. Remember you can rebuild with the same tag if the build fails. In any case, turning off force tag would lead to either: - needless release changelog inflation Do we require people to add a changelog entry for *every* increment of the release number? If you're just bumping the release for a rebuild, it seems reasonable to not bother with the changelog entry. or - usage of scratch builds just to test that it builds everywhere If the build fails because of a problem with a dependent package, you can rebuild from the same tag once the dependent package is fixed. If the build fails because of a problem with the spec file or source, you need to change something to make it build successfully. Bumping the release and adding a changelog entry seems appropriate in this case. I'd argue that even *now* we should be discouraging force-tag in this case. Actually, do we even know if disabling force tag can be worked around? Not sure what you're asking here. There's no reason force-tag is required for any part of the package maintenance workflow. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: rebuilding from old cvs tags
On Thu, 2008-02-28 at 13:45 -0500, Jesse Keating wrote: On Thu, 28 Feb 2008 18:24:59 + Paul Howarth [EMAIL PROTECTED] wrote: I've used it after I've done a make tag whilst forgetting to commit changes first, so the tag was applied to the wrong version of the spec file etc. Force tagging is useful for correcting this error. Couldn't that be solved by making 'make tag' abort if there are unchecked in files in the dir, particularly .spec ? It already should. make tag uses cvs tag -c. $ cvs -H tag Usage: cvs tag [-bcdFflR] [-r rev|-D date] tag [files...] snip -c Check that working files are unmodified. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: rebuilding from old cvs tags
On Thu, 2008-02-28 at 15:21 -0500, Bill Nottingham wrote: Mike Bonnet ([EMAIL PROTECTED]) said: Actually, do we even know if disabling force tag can be worked around? Not sure what you're asking here. There's no reason force-tag is required for any part of the package maintenance workflow. I mean, even if you remove force-tag from the Makefile, you can still move the tag yourself. Which I'm fairly sure you can do. We can make tags really immutable by adding a taginfo script to Fedora CVS, and exit 1 when a mov operation (tag -F) is requested. This is essentially the way we prevent the same tag from being applied on multiple branches today. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Dazed and confused....
On Wed, 2008-02-06 at 23:02 +, Bryce wrote: Just playing around with a miniature version of koji at home, building off my own box (i386/x86_64) using koji-1.2.5 Issue 1. The kernel, because my build host is defined as arch i386, x86_64 when I pass a kernel build in, it passes --target i386 which leads to the usual train wreck of no i386 config how do I pass in i686 specifically for the kernel? You need to set the extra arches that a package gets built for, in addition to the arches defined for that tag. Use: koji set-pkg-arches i686 dist-foo-build kernel In Fedora Koji the i386 build builds a kernel-headers package. If that's not working for you, you can avoid the i386 build altogether by putting ExcludeArch: i386 into the spec file. Issue 2. One of the fun items in xen is that when you're building the 64bit version there is an odd dependency on pulling in a 32bit glibc-devel by adding /usr/include/gnu/stubs-32.h to the dependency list, when building in x86_64 that dependancy cannot get satisfied. How can I beat the tag into realizing it can pull from the i386 repo to fulfill that requirement? Fedora uses the glibc32 and glibc64 packages to provide these dependencies. A bit of a hack, but it works: http://cvs.fedoraproject.org/viewcvs/rpms/glibc32/devel/ http://cvs.fedoraproject.org/viewcvs/rpms/glibc64/devel/ Koji won't let you pull i386 packages into a x86_64 build environment, or vice versa. The build environments are single-arch only. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
[PATCH] add --unpriv option to drop privileges when running a command with --chroot
This patch adds a --unpriv option that will cause privileges to be dropped before running a command with --chroot. This can be used to more closely simulate the environment used when running rpmbuilds. 0001-optionally-drop-privileges-when-running-a-command-wi.patch Description: application/mbox -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] add --unpriv option to drop privileges when running a command with --chroot
On Thu, 2008-01-24 at 16:04 -0500, Mike Bonnet wrote: On Thu, 2008-01-24 at 15:42 -0500, Mike Bonnet wrote: This patch adds a --unpriv option that will cause privileges to be dropped before running a command with --chroot. This can be used to more closely simulate the environment used when running rpmbuilds. Let me try that again... Ok, the attachments are getting stripped off for some reason, trying inline... From 85e14d38aec32cf20d7f2bbdc77044d41c32a0a2 Mon Sep 17 00:00:00 2001 From: Mike Bonnet [EMAIL PROTECTED] Date: Thu, 24 Jan 2008 15:37:15 -0500 Subject: [PATCH] optionally drop privileges when running a command with --chroot --- docs/mock.1 |3 +++ py/mock.py |8 +++- 2 files changed, 10 insertions(+), 1 deletions(-) diff --git a/docs/mock.1 b/docs/mock.1 index beaf2fb..38c3233 100644 --- a/docs/mock.1 +++ b/docs/mock.1 @@ -137,6 +137,9 @@ Change directory where config files are found \fB\-\-rpmbuild_timeout=\fR\fISECONDS\fP Fail build if rpmbuild takes longer than 'timeout' seconds .TP +\fB\-\-unpriv\fR +Drop privileges before running command when using --chroot +.TP \fB\-q\fR, \fB\-\-quiet\fR Be quiet. .TP diff --git a/py/mock.py b/py/mock.py index 4a589bc..f422a33 100755 --- a/py/mock.py +++ b/py/mock.py @@ -150,6 +150,8 @@ def command_parse(config_opts): dest=rpmbuild_timeout, type=int, default=None, help=Fail build if rpmbuild takes longer than 'timeout' seconds ) +parser.add_option(--unpriv, action=store_true, default=False, + help=Drop privileges before running command when using --chroot) # verbosity parser.add_option(-v, --verbose, action=store_const, const=2, @@ -532,7 +534,11 @@ def main(ret): chroot._resetLogging() try: chroot._mountall() -chroot.doChroot(args, shell=shell) +if options.unpriv: +chroot.doChroot(args, shell=shell, +uid=chroot.chrootuid, gid=chroot.chrootgid) +else: +chroot.doChroot(args, shell=shell) finally: chroot._umountall() -- 1.5.3.3 -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
[PATCH] do mounts when running mock --chroot
Here's a small patch to handle mounting things into the chroot when using the --chroot command, the same way we do when using --rebuild. --- mock.py.orig2008-01-16 13:42:21.0 -0500 +++ mock.py.mounts 2008-01-22 17:10:24.0 -0500 @@ -507,7 +507,12 @@ log.info(Running in chroot: %s % args) chroot.tryLockBuildRoot() chroot._resetLogging() -chroot.doChroot(args) + +try: +chroot._mountall() +chroot.doChroot(args) +finally: +chroot._umountall() elif options.mode == 'installdeps': if len(args) == 0: -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: [PATCH] Create the dev/full device, some packages use it during make check.
On Thu, 2008-01-03 at 09:43 -0500, Jesse Keating wrote: There are a few packages which use /dev/full to generate nospace left or other such return values for the use in make check. We should create that in mock. Here is a simple patch that adds it. Shouldn't that be os.makedev(1, 7)? $ ls -la /dev/full crw-rw-rw- 1 root root 1, 7 2007-12-29 17:34 /dev/full -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: RHEL 5.1
On Thu, 2007-11-08 at 16:35 -0500, rob myers wrote:
I know RHEL is a bit off topic for this list, but bear with me. I
import all the RHEL5 bits into koji and then use koji to build rpms for
RHEL5. The release of RHEL 5.1 gave me a few kinks to work out:
The RHEL5 supplementary channel includes some noarch packages which do
not come with source rpms. This breaks the assumption in mash that
excludearch and exclusive arch can safely be initialized on SRPMS only.
I know that mash is not designed for this use case, but I thought it
might be worth pointing out. The patch I used is below.
diff --git a/mash/__init__.py b/mash/__init__.py
index b02d6c8..6e7a496 100644
--- a/mash/__init__.py
+++ b/mash/__init__.py
@@ -209,6 +209,18 @@ class Mash:
# now deal with noarch
for pkg in noarch:
for target_arch in self.config.arches:
+
+# if excludearch is not set this build likely has no src.rpm
+# so set excludearch and exclusivearch from the binary
+if pkg['build_id'] fg
not in excludearch:
+path =
os.path.join(koji.pathinfo.build(builds_hash[pkg['build_id']]),
koji.pathinfo.rpm(pkg))
+fn = open(path, 'r')
+hdr = koji.get_rpm_header(fn)
+excludearch[pkg['build_id']] = hdr['EXCLUDEARCH']
+exclusivearch[pkg['build_id']] = hdr['EXCLUSIVEARCH']
+fn.close()
+continue
+
if (excludearch[pkg['build_id']] and
has_any(masharch.compat[target_arch], excludearch[pkg['build_id']])) or \
(exclusivearch[pkg['build_id']] and not
has_any(masharch.compat[target_arch], exclusivearch[pkg['build_id']])):
print Excluding %s.%s from %s due to
EXCLUDEARCH/EXCLUSIVEARCH % (pkg['name'], pkg['arch'], target_arch)
Another problem was that I had already imported RHEL 5.1 beta, which has
the same rpms in RHEL 5.1 signed with a different key- one that I do not
have access to. Since I wanted the rpms written out with the release
key, I added a function take the 5.1 release sighdrs and add them to the
previously imported RHEL 5.1 beta rpms. Would anyone else ever need this
functionality? The patch I used is below.
diff --git a/hub/kojihub.py b/hub/kojihub.py
index f9ba39a..c5a1261 100644
--- a/hub/kojihub.py
+++ b/hub/kojihub.py
@@ -4706,6 +4706,15 @@ class RootExports(object):
context.session.assertPerm('sign')
return add_rpm_sig(an_rpm, base64.decodestring(data))
+def addRPMSigByPath(self, an_rpm, path):
+Store a signature header for an rpm
+
+an_rpm: N-V-R.a (for example)
+path: the path to the signed rpm
+
+context.session.assertPerm('sign')
+return add_rpm_sig(an_rpm, koji.rip_rpm_sighdr(path))
+
findBuildID = staticmethod(find_build_id)
getTagID = staticmethod(get_tag_id)
getTag = staticmethod(get_tag)
Comments?
Does the koji import-sig command not do what you want?
--
Fedora-buildsys-list mailing list
Fedora-buildsys-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: RFC - Patch - koji SCM generalization
On Tue, 2007-10-23 at 14:13 -0400, rob myers wrote: attached is a patch that attempts to generalize checking out the components used to build an SRPM. this patch supports CVS, GIT, and SVN, but only CVS and SVN have been tested. the idea is to provide the infrastructure for different SCM systems to be configured at run-time so that users can choose their favorite system. is there a better approach? did i miss something obvious? general comments? It looks pretty good, but I wonder if you need to make scmtype a config option? I'd rather have Koji support all scmtypes all the time and have some other method for restricting what URLs are valid locations to download the source from. Maybe a list of valid hostnames or hostname/path pairs? -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: mock-0.7.2-1.fc6.1
On Wed, 2007-09-12 at 16:52 -0500, Mike McGrath wrote: Anyone know any reason why we shouldn't upgrade mock? Our last upgrade borked a bunch of stuff. I'm under the impression that has all been fixed but thought I'd ask. I haven't really been following mock development recently. Let me take a look at the changelog and see if anything looks like it might cause issues with Koji. I'll get back to you tomorrow. ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Re: Security issue: Can't build mediawiki - F7 thinks it's F8
On Mon, 2007-08-06 at 15:00 +0200, Axel Thimm wrote: On Mon, Aug 06, 2007 at 07:51:00AM -0500, Dennis Gilmore wrote: Once upon a time Monday 06 August 2007, Jesse Keating wrote: On Mon, 6 Aug 2007 14:18:36 +0200 Axel Thimm [EMAIL PROTECTED] wrote: Typo? Not exactly. Expression mismatch. The tag was /applied/ on the devel/ branch, so when cvs is asked for that tag, it tries to pull it from devel/ and bad things happen. in devel, koji gets a little confused when making the srpm for you. You'll most likely need to bump/tag on F-7 then you can build and it will get the proper .fc7 tag to it. Hm, this sounds more like a bug that should be fixed in koji. Wouldn't that apply to any kind of branching CVS, e.g. koji inherits bad tags to branches? This bug only surfaces if the tagsing and building is intermitted by the branch, but consider adding new archs to Fedora, it will hit all packages (as the build for the new arch will be after the branching, unless new archs are limited to devel). I'm not entirely sure how this is going to be handled. It probably does need looking into, something deep in the cvs branching scripts we use. My cvs-fu isn't nearly that strong :( probably need to call make tag after creating the branch. The problem was that make tag has been created before the branch, but the make build afterwards. It is also not possible to rerun make tag. The problem here is actually that the tag created on devel/ didn't include a branch file (that file doesn't get created until the branch-creation scripts are run). In the absence of a branch file, Makefile.common assumes you're on devel/ and expands the %dist tag to the values defined for devel in common/branches (currently .fc8). Basically, tagging before the branch point and building after it is not supported. After the branch point, any builds need to bump the revision and run make tag before they can build. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: Presto in F8
On Thu, 2007-07-19 at 16:36 -0700, John Poelstra wrote: Hi, I took an action item at the FESCo meeting today to check with the infrastructure team to make sure you are in alignment and can service the needs of the Presto feature proposed for F8. http://fedoraproject.org/wiki/Releases/FeaturePresto If there are any strong reasons why this feature should not be approved for F8, please add comments to the bottom of http://fedoraproject.org/wiki/Releases/FeaturePresto by 2007-07-23. Is support for generating/storing the deltarpms going to be added to Koji, or is it going to be a separate process? If it's a part of Koji, has someone committed to doing the required engineering work? ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Re: DB Upgrade complete
On Wed, 2007-07-11 at 18:09 -0400, Toshio Kuratomi wrote: On Tue, 2007-07-10 at 10:09 -0500, Mike McGrath wrote: Mike Bonnet wrote: On Tue, 2007-07-10 at 09:56 -0500, Mike McGrath wrote: With autovacuum on do we still have to run the nightly vacuum jobs? We'll have to keep an eye on the heavily-updated tables, but autovacuum combined with the max_fsm_pages increase should remove the need for the nightly vacuums. Sounds good, I'll leave them in the cron.d file but comment them out and mention why. We probably want to change to a monthly maintenance schedule where we run a vacuum full on the dbs. If things are configured correctly, postgres 8.1 should never require a full vacuum. Dead tuples get garbage-collected by the auto-vacuum process and the space is reused for new tuples. I'd rather avoid a window where all access to tables is locked out by the vacuum full. I also noticed that shmmax hasn't been increased from the default (32MB). Do we need to do that to match the shared cache buffer (512MB)? $ /sbin/sysctl kernel.shmmax kernel.shmmax = 33554432 Yes, there's a note about that in the diff. It needs to get changed or postgresql won't start with the modified config. ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Re: RFC: utility of 'orphansbuild' patch to mock-helper (BZ#221351)?
On Tue, 2007-07-10 at 11:53 -0500, Clark Williams wrote: Jan Kratochvil has submitted a patch to mock that adds the 'orphanskill' command to mock-helper (a setuid root program used by mock). The patch traverses the /proc directory, looking for tasks with a root link that matches the chroot currently in use, and sends a SIGKILL to each matching task. As far as I can tell this is only useful to the GDB build. The testsuite for GDB seems to have some either abnormal terminations or so other oddity that leaves jobs hanging. I've looked at the C code and it looks well written, without obvious security holes. I've mixed feelings regarding adding the command. Michael and I have been fairly resistant to adding things to mock-helper, on the general principle that adding features to a setuid root program is fraught with peril. I see the utility of the code, but I'm torn as to whether the 'orphanskill' command is sufficiently useful to the general community. So, that's the question. Is 'orphanskill' worth adding to mock? GDB is not the only build that leaves orphaned processes lying around. I've seen similar behavior when building gcc, glibc, and mysql, to name a few. The problems are usually caused by test suites called during the build process, and leaving them around after a build has completed (or failed) can tie up system resources or in some cases cause subsequent builds to fail. Just as mock cleans up the filesystem after a build, it should probably be cleaning up the process list as well. I'd be in favor of adding this patch. Koji could certainly make use of it. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
Re: DB Upgrade complete
On Tue, 2007-07-10 at 09:56 -0500, Mike McGrath wrote: Mike Bonnet wrote: #--- -#autovacuum = off # enable autovacuum subprocess? +autovacuum = on # enable autovacuum subprocess? #autovacuum_naptime = 60# time between autovacuum runs, in secs #autovacuum_vacuum_threshold = 1000 # min # of tuple updates before # vacuum With autovacuum on do we still have to run the nightly vacuum jobs? We'll have to keep an eye on the heavily-updated tables, but autovacuum combined with the max_fsm_pages increase should remove the need for the nightly vacuums. ___ Fedora-infrastructure-list mailing list Fedora-infrastructure-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-infrastructure-list
Re: [PATCH] kojid - BuildNotificationTask traceback
On Sun, 2007-04-22 at 13:22 +0200, Michael Schwendt wrote: --- kojid~ 2007-04-09 19:23:38.0 +0200 +++ kojid 2007-04-22 13:18:09.0 +0200 @@ -2118,7 +2118,7 @@ if changelog: changelog = Changelog:\r\n%s % changelog -from_addr = self.from_addr +from_addr = options.from_addr to_addrs = ', '.join(recipients) subject = self.subject_templ % locals() message = self.message_templ % locals() Thanks, patch applied. -- Fedora-buildsys-list mailing list Fedora-buildsys-list@redhat.com https://www.redhat.com/mailman/listinfo/fedora-buildsys-list
