Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Josh Boyer (jwbo...@gmail.com) said: You'd want to do it something like that. kernel-minimal as you say but with a Provides: kernel, kernel-common as you say. I'd introduce a third metapackage just kernel that requires both of those and implicitly Provides: kernel. Most people would just get the kernel metapackage when a transaction asks for something to provide kernel, but if you explicitly ask for kernel-minimal you'd get just the minimal. This would all be done from one kernel spec and built out at the same time. We've got a lot of new infrastructure coming for kernel builds and we don't want to make things even more complicated by having to do multiple rpm build runs. All of this can probably already be done with a new 'flavor' in the existing kernel.spec. I really wouldn't do the common/minimal split though. It just makes it more complicated for not a whole lot of gain. The idea that Dave, Justin, and Kevin all had simlutaneously about doing a 'kernel-virtguest' might be worthwhile if someone wants to spend time poking at a config, etc. That also works with the normal paradigm where all the variants provide 'kernel' for RPM dependency purposes; if you try to have a kernel-minimal that provides 'kernel' while also having a 'kernel' package that requires 'kernel-minimal', things get a bit more strange. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 10:33:27AM -0400, Bill Nottingham wrote: All of this can probably already be done with a new 'flavor' in the existing kernel.spec. I really wouldn't do the common/minimal split though. It just makes it more complicated for not a whole lot of gain. The idea that Dave, Justin, and Kevin all had simlutaneously about doing a 'kernel-virtguest' might be worthwhile if someone wants to spend time poking at a config, etc. That also works with the normal paradigm where all the variants provide 'kernel' for RPM dependency purposes; if you try to have a kernel-minimal that provides 'kernel' while also having a 'kernel' package that requires 'kernel-minimal', things get a bit more strange. I'm open to this idea, but I think it's nicer if one can go from the reduced selection to the full just by adding in the right package, not changing or removing things. Unlike PAE or etc., I don't think we'd actually build anything differently (would we?). -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 10:39 AM, Matthew Miller mat...@fedoraproject.org wrote: On Thu, Oct 18, 2012 at 10:33:27AM -0400, Bill Nottingham wrote: All of this can probably already be done with a new 'flavor' in the existing kernel.spec. I really wouldn't do the common/minimal split though. It just makes it more complicated for not a whole lot of gain. The idea that Dave, Justin, and Kevin all had simlutaneously about doing a 'kernel-virtguest' might be worthwhile if someone wants to spend time poking at a config, etc. That also works with the normal paradigm where all the variants provide 'kernel' for RPM dependency purposes; if you try to have a kernel-minimal that provides 'kernel' while also having a 'kernel' package that requires 'kernel-minimal', things get a bit more strange. I'm open to this idea, but I think it's nicer if one can go from the reduced selection to the full just by adding in the right package, not changing or removing things. Unlike PAE or etc., I don't think we'd actually build anything differently (would we?). Of course we would. The entire point is to reduce the size, and the only way to reduce the size is to build it with different config options. And we're not talking about going from kernel-virtguest to kernel by installing kernel-everythingnotinvirtguest. That's still going down the split the kernel up into a bunch of subpackages route which just creates more work for the maintainers. At the moment though, all of this is just talk anyway. If something like this is to happen, someone actually has to do work. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 10:44:58AM -0400, Josh Boyer wrote: I'm open to this idea, but I think it's nicer if one can go from the reduced selection to the full just by adding in the right package, not changing or removing things. Unlike PAE or etc., I don't think we'd actually build anything differently (would we?). Of course we would. The entire point is to reduce the size, and the only way to reduce the size is to build it with different config options. And we're not talking about going from kernel-virtguest to kernel by installing kernel-everythingnotinvirtguest. That's still going down the split the kernel up into a bunch of subpackages route which just creates more work for the maintainers. We already have kernel-modules-extra. I think the idea would be to add kernel-modules-virt and kernel-modules-normal to that, at most. (Or, put the virt modules in kernel, and just add one more subpackage, kernel-modules-normal.) There's already code in the spec file for dealing with modules-extra, so it's mostly a matter of extending that slightly -- not doing something entirely new, *and* not going down the alarmist slope of a horde of subpackages. At the moment though, all of this is just talk anyway. If something like this is to happen, someone actually has to do work. Start with an idea, discuss it, come up with a plan, find resources for that plan, and then implement. Sometimes things happen the other way around, but only when we happen to be lucky, and it often has consequences like extra ongoing work with no support. So, just talk is an important place to start. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 11:34:00AM -0400, Josh Boyer wrote: On Thu, Oct 18, 2012 at 11:15 AM, Matthew Miller mat...@fedoraproject.org wrote: On Thu, Oct 18, 2012 at 10:44:58AM -0400, Josh Boyer wrote: At the moment though, all of this is just talk anyway. If something like this is to happen, someone actually has to do work. Start with an idea, discuss it, come up with a plan, find resources for that plan, and then implement. Sometimes things happen the other way around, but only when we happen to be lucky, and it often has consequences like extra ongoing work with no support. So, just talk is an important place to start. OK. So here's a more blunt response. I'm really against splitting the modules up into more subpackages, regardless of how many it is. I will not spend any time looking at how to do that. I won't spend time discussing further plans to do something I don't feel is maintainable. If you find someone willing to, great. Post patches to the kernel list and we'll review them. But I'm telling you now that it will be an uphill battle. Just to be clear on this, if someone feels it is worthwhile and wants to step up and do the work, it can't just be someone who will send patches to change the current kernel and leave. It has to be someone willing to sign up to maintain the solution in the rawhide kernel. Modules change, get added or removed, and/or have deps change pretty much every single release. 3.7 just moved the nat modules. This is why we are saying that it is much easier to do a different build of another flavor than to split out the modules into more subpackages. Justin -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Thu, Oct 18, 2012 at 10:56:21AM -0500, Justin M. Forbes wrote: I'm really against splitting the modules up into more subpackages, regardless of how many it is. I will not spend any time looking at how to do that. I won't spend time discussing further plans to do something I don't feel is maintainable. If you find someone willing to, great. Post patches to the kernel list and we'll review them. But I'm telling you now that it will be an uphill battle. Just to be clear on this, if someone feels it is worthwhile and wants to step up and do the work, it can't just be someone who will send patches to change the current kernel and leave. It has to be someone willing to sign up to maintain the solution in the rawhide kernel. Modules change, get added or removed, and/or have deps change pretty much every single release. 3.7 just moved the nat modules. This is why we are saying that it is much easier to do a different build of another flavor than to split out the modules into more subpackages. Head nods to both of you. I'm happy to live in reality rather than utopia and this is helpful feedback. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Thu, 11.10.12 01:48, Lennart Poettering (mzerq...@0pointer.de) wrote: On Wed, 10.10.12 16:50, Kevin Fenzi (ke...@scrye.com) wrote: My laptop started acting up last tuesday, I should see whats in the logs from then I'd like to run a daily report on my logs These two are much better implemented via explicit time seeks. The journal APIs support that just fine, journalctl currently doesn't. However it's trivial to add that based on the lower level APIs, the only thing that stopped me from doing that so far is that for that we'd have to come up with a nice way to parse calendar timestamps, and I want to be careful about that. that said the idea is to have two command line args to journalctl where you can pass things such as: $ journalctl --start-time=2012-10-01 ... $ journalctl --start-time=-5d ... $ journalctl --start-time=2012-01-01 --end-time=2012-05-02 ... A quick update: This is implemented now, but I called it --since= and --until=. I'll push this into F18 as well, sicne it's actually a minor change only, and just too useful. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Tue, Oct 16, 2012 at 9:07 AM, Bill Nottingham nott...@redhat.com wrote: Peter Robinson (pbrobin...@gmail.com) said: I wonder... could we make linux-firmware optional? I would expect many virt env's don't need any firmware to work... (but of course I could be wrong). It use to be optional, I know on the olpc xo-1 it use to be optional and there should be no firmware needed for an average VM. I'd also love to see it broken down for various profiles because most desktops don't need enterprise storage controllers, most servers don't need wifi and most ARM platforms don't need most of the stuff in there but do need a few ARM only firmware packages. However, if you go down that route, the kernel should be the same way, the firmware should be separate subpackages, and requires should be done at the module - firmware level by generating it from the MODULE_FIRMWARE tags. (Unless you're relying on packagekit to install your firmware, which if you're going that minimal seems to have missed the forest for the trees somewhere.) I'm not understanding what you're proposing. Are you suggesting: 1) We have further split up module sub-packages that carry their own firmware requires (e.g. kernel-module-iwlwifi requires iwlwifi-firmware) or 2) Even more firmware subpackages split out of linux-firmware. If you're suggesting 1, I'd be really really opposed to that. It would make packaging in kernel.spec even more of a nightmare than it already is. If you're suggesting 2, I don't see the point. The kernel will install and even with per-module dependencies generated (somehow), it'll still install all of the various -firmware packages because the modules will be getting installed. Or maybe you mean something else, and my jet-lagged and coffee deprived brain just isn't following. I actually hope this is the case, because neither of the above 2 options sound that great to me... josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Tue, 09.10.12 23:24, Lennart Poettering (mzerq...@0pointer.de) wrote: I am not generally against adding time-based rotation, but really, this is much less of a necessity than other things the journal provides, which syslog does not: for example per-service rate limits, and unfakable meta-data for log messages. I mean, really, how can we ship a syslog where every random user can fake messages, say they are from a privileged process and offer no way how to detect that? To settle this discussion as well I've now implemented time-based rotation for the journal as well, and this will also hit F18 soonishly. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 17, 2012 at 02:13:35PM +0200, Lennart Poettering wrote: This is implemented now, but I called it --since= and --until=. I'll push this into F18 as well, sicne it's actually a minor change only, and just too useful. Thanks Lennart. This is great stuff. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Josh Boyer (jwbo...@gmail.com) said: However, if you go down that route, the kernel should be the same way, the firmware should be separate subpackages, and requires should be done at the module - firmware level by generating it from the MODULE_FIRMWARE tags. (Unless you're relying on packagekit to install your firmware, which if you're going that minimal seems to have missed the forest for the trees somewhere.) I'm not understanding what you're proposing. Are you suggesting: 1) We have further split up module sub-packages that carry their own firmware requires (e.g. kernel-module-iwlwifi requires iwlwifi-firmware) or 2) Even more firmware subpackages split out of linux-firmware. If you're suggesting 1, I'd be really really opposed to that. It would make packaging in kernel.spec even more of a nightmare than it already is. If you're suggesting 2, I don't see the point. The kernel will install and even with per-module dependencies generated (somehow), it'll still install all of the various -firmware packages because the modules will be getting installed. Both - if people want firmware packages split out of linux-firmware, it doesn't make sense unless the drivers (similar in size) are also split, and if you were to do so, you'd need to start adding the driver - firmware package dependency mapping. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 10:47:34AM -0400, Bill Nottingham wrote: If you're suggesting 1, I'd be really really opposed to that. It would make packaging in kernel.spec even more of a nightmare than it already is. [...] Both - if people want firmware packages split out of linux-firmware, it doesn't make sense unless the drivers (similar in size) are also split, and if you were to do so, you'd need to start adding the driver - firmware package dependency mapping. Basically: it's hard, but the only way we're going to get to a reasonably-small minimal image, so if that's important (and I believe it is), it's something Fedora should invest resources in. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 10:51 AM, Matthew Miller mat...@fedoraproject.org wrote: On Wed, Oct 17, 2012 at 10:47:34AM -0400, Bill Nottingham wrote: If you're suggesting 1, I'd be really really opposed to that. It would make packaging in kernel.spec even more of a nightmare than it already is. [...] Both - if people want firmware packages split out of linux-firmware, it doesn't make sense unless the drivers (similar in size) are also split, and if you were to do so, you'd need to start adding the driver - firmware package dependency mapping. What the hell did you drink today, Bill? Basically what you're suggesting is that Fedora move to a kmod model for everything. Which means you'd have to install all of them by default anyway or the kernel team would be swamped with bugs like I installed Fedora and my random USB device doesn't work! Seriously, no. There are 3 kernel people. There's no way we have time to sit around closing bugs with stuff like Install kernel-module-foo. Basically: it's hard, but the only way we're going to get to a reasonably-small minimal image, so if that's important (and I believe it is), it's something Fedora should invest resources in. No. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 4:51 PM, Matthew Miller mat...@fedoraproject.org wrote: On Wed, Oct 17, 2012 at 10:47:34AM -0400, Bill Nottingham wrote: If you're suggesting 1, I'd be really really opposed to that. It would make packaging in kernel.spec even more of a nightmare than it already is. [...] Both - if people want firmware packages split out of linux-firmware, it doesn't make sense unless the drivers (similar in size) are also split, and if you were to do so, you'd need to start adding the driver - firmware package dependency mapping. Basically: it's hard, it is a mess. but the only way we're going to get to a reasonably-small minimal image, not true. so if that's important (and I believe it is), I believe it isn't. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote: Basically: it's hard, it is a mess. but the only way we're going to get to a reasonably-small minimal image, not true. Given that the kernel is currently a full quarter of the current image, I think it has to be. so if that's important (and I believe it is), I believe it isn't. And that's fine. We'll need to come to a group decision eventually but we don't need to right now. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 11:37:29AM -0400, Josh Boyer wrote: What the hell did you drink today, Bill? Basically what you're suggesting is that Fedora move to a kmod model for everything. Which means you'd have to install all of them by default anyway or the kernel team would be swamped with bugs like I installed Fedora and my random USB device doesn't work! The default desktop case would pull in everything, I think. Splitting it up would be not just for fun but in order to target smaller cloud and virt images. Because we can reasonably define the targets there, it doesn't need to be completely all-out kmod model for everything approach at all. Seriously, no. There are 3 kernel people. There's no way we have time to sit around closing bugs with stuff like Install kernel-module-foo. Yeah; it is reasonable to say that we can't do this without more resources. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 5:46 PM, Matthew Miller mat...@fedoraproject.org wrote: On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote: Basically: it's hard, it is a mess. but the only way we're going to get to a reasonably-small minimal image, not true. Given that the kernel is currently a full quarter of the current image, I think it has to be. No you could also use a different kernel image; build your own kernel; use a compressed filesystem, don't use a kernel at all and some container based approach instead of full virt for your cloud instances (you could even base them on a btrfs subvolume and save more space that way). Outside of the cloud use case the disk space added by modules and firmware does not matter a bit (so I am ignoring this cases). So there are lots of other ways to achieve what you want without splitting the kernel into hundreds of sub packages. So while it is a way it is not the only way. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 05:59:55PM +0200, drago01 wrote: Given that the kernel is currently a full quarter of the current image, I think it has to be. No you could also use a different kernel image; build your own kernel; use a compressed filesystem, don't use a kernel at all and some container based approach instead of full virt for your cloud instances (you could even base them on a btrfs subvolume and save more space that way). Outside of the cloud use case the disk space added by modules and firmware does not matter a bit (so I am ignoring this cases). So there are lots of other ways to achieve what you want without splitting the kernel into hundreds of sub packages. So while it is a way it is not the only way. As reluctant as I am to introduce new kernel packages, an ultra-minimal kernel package for use in cloud environments may make more sense than splitting up the one-size-fits-all packages into hundreds of sub-packages. But even this option is a lot of work, and isn't a panacea. With virtualised environments supporting pci/usb passthrough, where do you draw the line on what hardware to support in a hypothetical kernel-cloud package ? Dave -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 05:59:55PM +0200, drago01 wrote: On Wed, Oct 17, 2012 at 5:46 PM, Matthew Miller mat...@fedoraproject.org wrote: On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote: Basically: it's hard, it is a mess. but the only way we're going to get to a reasonably-small minimal image, not true. Given that the kernel is currently a full quarter of the current image, I think it has to be. No you could also use a different kernel image; build your own kernel; [I'll treat these two the same, because they amount to the same thing] It's a considerable amount of work for everyone if people building minimal images have to use a different kernel. By using the same kernel as everyone else, it means that bug reports against the normal kernel package are relevant, and it means that the regular kernel gets more testing. Also it's a lot of work to compile another kernel, when we've already got a team of (apparently 3) people doing this. use a compressed filesystem, Every minimal image I've ever seen has been compressed to the max. don't use a kernel at all and some container based approach instead of full virt for your cloud instances Unfortunately containers don't work for every application out there. Obviously *if* you can use a container, then you should, and you probably are already. Outside of the cloud use case the disk space added by modules and firmware does not matter a bit (so I am ignoring this cases). It's partly disk space, it's more often the time taken to copy these images over the network (eg. when users download minimal images, or when they are PXE-booted). So there are lots of other ways to achieve what you want without splitting the kernel into hundreds of sub packages. I don't think we're talking hundreds of sub packages. Most people seem to be discussing a split into between 2 and 5 packages. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 6:58 PM, Richard W.M. Jones rjo...@redhat.com wrote: On Wed, Oct 17, 2012 at 05:59:55PM +0200, drago01 wrote: On Wed, Oct 17, 2012 at 5:46 PM, Matthew Miller mat...@fedoraproject.org wrote: On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote: Basically: it's hard, it is a mess. but the only way we're going to get to a reasonably-small minimal image, not true. Given that the kernel is currently a full quarter of the current image, I think it has to be. No you could also use a different kernel image; build your own kernel; [I'll treat these two the same, because they amount to the same thing] No the one means we ship a kernel-foo (like kernel-minimal or whatever we call it) and the other is you build your own kernel. It's a considerable amount of work for everyone if people building minimal images have to use a different kernel. If it is all about using kernel-minimal (or whatever it is called) instead of kernel there is no extra work for the ones that build minimal images at all. By using the same kernel as everyone else, it means that bug reports against the normal kernel package are relevant, and it means that the regular kernel gets more testing. They can be built from the same srpm (same version, same patches applied just some modules stripped). Also it's a lot of work to compile another kernel, when we've already got a team of (apparently 3) people doing this. I'd argue it is less work then building a subpackage for every kernel module. use a compressed filesystem, Every minimal image I've ever seen has been compressed to the max. The image itself might be the installed system isn't ... which is the think I was talking about (i.e this would allow you to save disk space after installation). The smaller image would just save bandwidth for the initial download and/or space on mirrors. don't use a kernel at all and some container based approach instead of full virt for your cloud instances Unfortunately containers don't work for every application out there. Obviously *if* you can use a container, then you should, and you probably are already. That might be true but I can't think of such applications right now. Couldn't the applications you have in mind be modified / fixed to work in such an environment? Outside of the cloud use case the disk space added by modules and firmware does not matter a bit (so I am ignoring this cases). It's partly disk space, it's more often the time taken to copy these images over the network (eg. when users download minimal images, or when they are PXE-booted). In that case you should place the image on your internal network. I doubt anyone uses a cloud infrastructure where you download the images over the internet using a 56k modem. So there are lots of other ways to achieve what you want without splitting the kernel into hundreds of sub packages. I don't think we're talking hundreds of sub packages. Most people seem to be discussing a split into between 2 and 5 packages. People where talking about splitting each module (driver) into its own package. This are far more then 5. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 6:34 PM, drago01 drag...@gmail.com wrote: On Wed, Oct 17, 2012 at 6:58 PM, Richard W.M. Jones rjo...@redhat.com wrote: On Wed, Oct 17, 2012 at 05:59:55PM +0200, drago01 wrote: On Wed, Oct 17, 2012 at 5:46 PM, Matthew Miller mat...@fedoraproject.org wrote: On Wed, Oct 17, 2012 at 05:38:55PM +0200, drago01 wrote: Basically: it's hard, it is a mess. but the only way we're going to get to a reasonably-small minimal image, not true. Given that the kernel is currently a full quarter of the current image, I think it has to be. No you could also use a different kernel image; build your own kernel; [I'll treat these two the same, because they amount to the same thing] No the one means we ship a kernel-foo (like kernel-minimal or whatever we call it) and the other is you build your own kernel. It's a considerable amount of work for everyone if people building minimal images have to use a different kernel. If it is all about using kernel-minimal (or whatever it is called) instead of kernel there is no extra work for the ones that build minimal images at all. By using the same kernel as everyone else, it means that bug reports against the normal kernel package are relevant, and it means that the regular kernel gets more testing. They can be built from the same srpm (same version, same patches applied just some modules stripped). Also it's a lot of work to compile another kernel, when we've already got a team of (apparently 3) people doing this. I'd argue it is less work then building a subpackage for every kernel module. use a compressed filesystem, Every minimal image I've ever seen has been compressed to the max. The image itself might be the installed system isn't ... which is the think I was talking about (i.e this would allow you to save disk space after installation). The smaller image would just save bandwidth for the initial download and/or space on mirrors. don't use a kernel at all and some container based approach instead of full virt for your cloud instances Unfortunately containers don't work for every application out there. Obviously *if* you can use a container, then you should, and you probably are already. That might be true but I can't think of such applications right now. Couldn't the applications you have in mind be modified / fixed to work in such an environment? Outside of the cloud use case the disk space added by modules and firmware does not matter a bit (so I am ignoring this cases). It's partly disk space, it's more often the time taken to copy these images over the network (eg. when users download minimal images, or when they are PXE-booted). In that case you should place the image on your internal network. I doubt anyone uses a cloud infrastructure where you download the images over the internet using a 56k modem. So there are lots of other ways to achieve what you want without splitting the kernel into hundreds of sub packages. I don't think we're talking hundreds of sub packages. Most people seem to be discussing a split into between 2 and 5 packages. People where talking about splitting each module (driver) into its own package. This are far more then 5. Not necessarily, I was the person wanting it split down and only to groups of packages like usb for usb devices, arm for arm only firmware, storage for enterprise storage, wifi etc. Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 07:34:22PM +0200, drago01 wrote: If it is all about using kernel-minimal (or whatever it is called) instead of kernel there is no extra work for the ones that build minimal images at all. It really depends on what 'kernel-minimal' is. If it's the same kernel (identical vmlinuz) with groups of modules, then I'm assuming this is the same as what everyone else is proposing. But more dangerous if this is a recompile of the kernel (maybe same source, different configs). Then inevitably we're testing different codepaths. [...] Unfortunately containers don't work for every application out there. Obviously *if* you can use a container, then you should, and you probably are already. That might be true but I can't think of such applications right now. Couldn't the applications you have in mind be modified / fixed to work in such an environment? LXC isn't a secure alternative to full virtualization -- try poking random /sys files on your container -- and even now OpenVZ isn't quite upstream. Also clouds are existing ecosystems. For example I can't call up Amazon and ask that they reimplement their service on top of LXC instead of Xen. In that case you should place the image on your internal network. Well, no. I don't colocate with everyone who might want to download a minimal image. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones libguestfs lets you edit virtual machines. Supports shell scripting, bindings from many languages. http://libguestfs.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Once upon a time, Richard W.M. Jones rjo...@redhat.com said: It really depends on what 'kernel-minimal' is. If it's the same kernel (identical vmlinuz) with groups of modules, then I'm assuming this is the same as what everyone else is proposing. I would think the only sane way would be to just change the packaing, not actually build multiple kernels (or even multiple packages with kernels). For example, a kernel-minimal that has the kernel and the core modules loaded in most installs (e.g. filesystems like ext4 and NFS, dm, network support like ipv6 and iptables, and virtio-type drivers), a kernel-common that has the rest of the current contents of kernel (and probably obsoletes kernel), and then the current kernel-modules-extras. There will always be requests to move modules from -common to -minimal, and it shouldn't be a big fight (I would bet most requests would be pretty obvious). That already exists some for -modules-extras. -- Chris Adams cmad...@hiwaay.net Systems and Network Administrator - HiWAAY Internet Services I don't speak for anybody but myself - that's enough trouble. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On 10/17/2012 11:32 AM, Chris Adams wrote: I would think the only sane way would be to just change the packaing, not actually build multiple kernels (or even multiple packages with kernels). For example, a kernel-minimal that has the kernel and the core modules loaded in most installs (e.g. filesystems like ext4 and NFS, dm, network support like ipv6 and iptables, and virtio-type drivers), a kernel-common that has the rest of the current contents of kernel (and probably obsoletes kernel), and then the current kernel-modules-extras. There will always be requests to move modules from -common to -minimal, and it shouldn't be a big fight (I would bet most requests would be pretty obvious). That already exists some for -modules-extras. You'd want to do it something like that. kernel-minimal as you say but with a Provides: kernel, kernel-common as you say. I'd introduce a third metapackage just kernel that requires both of those and implicitly Provides: kernel. Most people would just get the kernel metapackage when a transaction asks for something to provide kernel, but if you explicitly ask for kernel-minimal you'd get just the minimal. This would all be done from one kernel spec and built out at the same time. We've got a lot of new infrastructure coming for kernel builds and we don't want to make things even more complicated by having to do multiple rpm build runs. -- Jesse Keating Fedora -- Freedom² is a feature! -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 01:32:23PM -0500, Chris Adams wrote: There will always be requests to move modules from -common to -minimal, and it shouldn't be a big fight (I would bet most requests would be pretty obvious). That already exists some for -modules-extras. That's why I suggest defining the specific targets (for example, EC2, KVM) for the core package. The requirements may grow and change slightly, but in general it provides a simple test, with no fighting needed. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 11:38:13AM -0700, Jesse Keating wrote: I'd introduce a third metapackage just kernel that requires both of those and implicitly Provides: kernel. Most people would just get the kernel metapackage when a transaction asks for something to provide kernel, but if you explicitly ask for kernel-minimal you'd get just the minimal. +1 to this -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, 17 Oct 2012 14:40:39 -0400 Matthew Miller mat...@fedoraproject.org wrote: On Wed, Oct 17, 2012 at 11:38:13AM -0700, Jesse Keating wrote: I'd introduce a third metapackage just kernel that requires both of those and implicitly Provides: kernel. Most people would just get the kernel metapackage when a transaction asks for something to provide kernel, but if you explicitly ask for kernel-minimal you'd get just the minimal. +1 to this I think we should really avoid calling this minimal as we have seen that means different things to different folks. if someone wanted to explore this route, I would say call it 'kernel-virt-guest' or something similar. Establish exactly what virt env's are targeted by it, do some test runs to show that it helps anyone at all, and then sell it to the kernel maintainers. ;) Sounds like it could be a nice f19 feature if there are folks interested enough to work on it. kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Am 17.10.2012 18:52, schrieb Dave Jones: With virtualised environments supporting pci/usb passthrough, where do you draw the line on what hardware to support in a hypothetical kernel-cloud package ? with vmxnet3, vmw_pvscsi, vmw_balloon to support vSphere (all included in the upstream kernel) and the drivers for basic KVM guests + all the iptabales (nat, conntrack, rectent, multiport..) you are sipporting a really wide range of minimized-size and full functional setups 99 out of 100 doe snot passthru physical hardware signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, 2012-10-17 at 11:38 -0700, Jesse Keating wrote: On 10/17/2012 11:32 AM, Chris Adams wrote: I would think the only sane way would be to just change the packaing, not actually build multiple kernels (or even multiple packages with kernels). For example, a kernel-minimal that has the kernel and the core modules loaded in most installs (e.g. filesystems like ext4 and NFS, dm, network support like ipv6 and iptables, and virtio-type drivers), a kernel-common that has the rest of the current contents of kernel (and probably obsoletes kernel), and then the current kernel-modules-extras. There will always be requests to move modules from -common to -minimal, and it shouldn't be a big fight (I would bet most requests would be pretty obvious). That already exists some for -modules-extras. You'd want to do it something like that. kernel-minimal as you say but with a Provides: kernel, kernel-common as you say. I'd introduce a third metapackage just kernel that requires both of those and implicitly Provides: kernel. Most people would just get the kernel metapackage when a transaction asks for something to provide kernel, but if you explicitly ask for kernel-minimal you'd get just the minimal. This would all be done from one kernel spec and built out at the same time. We've got a lot of new infrastructure coming for kernel builds and we don't want to make things even more complicated by having to do multiple rpm build runs. Random worry about this: would this work OK with yum's keep the last 3 kernels around functionality? -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On 10/17/2012 01:46 PM, David Malcolm wrote: Random worry about this: would this work OK with yum's keep the last 3 kernels around functionality? That's obviously something that would have to be tested if this is attempted. I'm not signing up for this work, I was just making a suggestion on how to structure the rpms that fell out of the kernel spec. -- Jesse Keating Fedora -- Freedom² is a feature! -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 2:38 PM, Jesse Keating jkeat...@redhat.com wrote: On 10/17/2012 11:32 AM, Chris Adams wrote: I would think the only sane way would be to just change the packaing, not actually build multiple kernels (or even multiple packages with kernels). We already build multiple kernels. All from the same source, but still multiple kernels with different config options. E.g. PAE, debug, etc. For example, a kernel-minimal that has the kernel and the core modules loaded in most installs (e.g. filesystems like ext4 and NFS, dm, network support like ipv6 and iptables, and virtio-type drivers), a kernel-common that has the rest of the current contents of kernel (and probably obsoletes kernel), and then the current kernel-modules-extras. There will always be requests to move modules from -common to -minimal, and it shouldn't be a big fight (I would bet most requests would be pretty obvious). That already exists some for -modules-extras. You'd want to do it something like that. kernel-minimal as you say but with a Provides: kernel, kernel-common as you say. I'd introduce a third metapackage just kernel that requires both of those and implicitly Provides: kernel. Most people would just get the kernel metapackage when a transaction asks for something to provide kernel, but if you explicitly ask for kernel-minimal you'd get just the minimal. This would all be done from one kernel spec and built out at the same time. We've got a lot of new infrastructure coming for kernel builds and we don't want to make things even more complicated by having to do multiple rpm build runs. All of this can probably already be done with a new 'flavor' in the existing kernel.spec. I really wouldn't do the common/minimal split though. It just makes it more complicated for not a whole lot of gain. The idea that Dave, Justin, and Kevin all had simlutaneously about doing a 'kernel-virtguest' might be worthwhile if someone wants to spend time poking at a config, etc. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Wed, Oct 17, 2012 at 3:43 PM, Reindl Harald h.rei...@thelounge.net wrote: Am 17.10.2012 18:52, schrieb Dave Jones: With virtualised environments supporting pci/usb passthrough, where do you draw the line on what hardware to support in a hypothetical kernel-cloud package ? with vmxnet3, vmw_pvscsi, vmw_balloon to support vSphere (all included in the upstream kernel) and the drivers for basic KVM guests + all the iptabales (nat, conntrack, rectent, multiport..) you are sipporting a really wide range of minimized-size and full functional setups And xen and whatever it needs. I mean, if you're targeting virt and cloud, then you can't ignore EC2 and that is all xen. And hyper-v. Can't exclude them either. And whatever Rusty's virt thing is. And on and on. About the only virt/cloud thing you can exclude is e.g. virtualbox and that's only because they're not in the upstream kernel. Maybe one day they'll realize the error of their ways and then you have to include that too. 99 out of 100 doe snot passthru physical hardware Maybe today, but it's an ever increasing target. I don't think it can be entirely dismissed. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
one may say disk storage is nothing these days iw ould say: mulitply it with 20, 50, 100 virtual machines on really expensive SAN-storage where disk space is cheap is not true And I would say : get an entreprisey deduping san -- Nicolas Mailhot -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, Oct 15, 2012 at 02:36:20PM -0400, john.flor...@dart.biz wrote: From: Bill Nottingham nott...@redhat.com Jesse Keating (jkeat...@j2solutions.net) said: Well, we do currently have the minimal environment, which boils down to @core + the couple things anaconda forces (authconfig, system-config-firewall-base, kernel, bootloader). You can get to that via kickstart with just: %packages @core %end But it's not close to what some of these people want out of a minimal install. For reference: @core + kernel: Install 38 Packages (+157 Dependent packages) Total download size: 128 M Installed size: 506 M But hey, I just want something smaller! systemd + util-linux + bash + initscripts + passwd + yum: Install 7 Packages (+132 Dependent packages) Total download size: 106 M Installed size: 446 M But hey, I don't need to install packages or want python! systemd+ util-linux + bash + initscripts + passwd: Install 6 Packages (+108 Dependent packages) Total download size: 94 M Installed size: 401 M Bill, thanks for that excellent report! It shows me that even if you strip away some of the conveniences, you really don't save that much over our normal minimal install. Very enlightening. If you remove files that are also duplicated on the host (assuption: host distro + version = guest distro + version) then you can make things MUCH smaller :-) $ febootstrap --names bash systemd yum $ ll -h * -rw-rw-r--. 1 rjones rjones 4.6M Oct 16 08:44 base.img -rw-rw-r--. 1 rjones rjones 630K Oct 16 08:44 hostfiles # To reconstruct the image before boot: $ febootstrap-supermin-helper -f cpio base.img hostfiles x86_64 kernel.out initrd.out $ ll -h kernel.out initrd.out -rw-r--r--. 1 rjones rjones 293M Oct 16 08:46 initrd.out lrwxrwxrwx. 1 rjones rjones 33 Oct 16 08:46 kernel.out - /boot/vmlinuz-3.6.1-1.fc18.x86_64 Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Matthew Miller (mat...@fedoraproject.org) said: On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote: But hey, I don't need to install packages or want python! systemd+ util-linux + bash + initscripts + passwd: Install 6 Packages (+108 Dependent packages) Total download size: 94 M Installed size: 401 M Of which one quarter is the kernel and the other quarter is glibc locale support, right? Or more: 122659574 kernel 117821428 glibc-common 35623360linux-firmware 14233540coreutils 13845828glibc I wonder... could we make linux-firmware optional? I would expect many virt env's don't need any firmware to work... (but of course I could be wrong). It use to be optional, I know on the olpc xo-1 it use to be optional and there should be no firmware needed for an average VM. I'd also love to see it broken down for various profiles because most desktops don't need enterprise storage controllers, most servers don't need wifi and most ARM platforms don't need most of the stuff in there but do need a few ARM only firmware packages. Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
Peter Robinson (pbrobin...@gmail.com) said: I wonder... could we make linux-firmware optional? I would expect many virt env's don't need any firmware to work... (but of course I could be wrong). It use to be optional, I know on the olpc xo-1 it use to be optional and there should be no firmware needed for an average VM. I'd also love to see it broken down for various profiles because most desktops don't need enterprise storage controllers, most servers don't need wifi and most ARM platforms don't need most of the stuff in there but do need a few ARM only firmware packages. However, if you go down that route, the kernel should be the same way, the firmware should be separate subpackages, and requires should be done at the module - firmware level by generating it from the MODULE_FIRMWARE tags. (Unless you're relying on packagekit to install your firmware, which if you're going that minimal seems to have missed the forest for the trees somewhere.) I wouldn't be against that, but I also don't have the time to look at that right now. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: modules, firmware, kernel size (was Re: systemd requires HTTP server and serves QR codes)
On Tue, Oct 16, 2012 at 09:07:56AM -0400, Bill Nottingham wrote: I wonder... could we make linux-firmware optional? However, if you go down that route, the kernel should be the same way, the firmware should be separate subpackages, and requires should be done at the module - firmware level by generating it from the MODULE_FIRMWARE tags. [...] I wouldn't be against that, but I also don't have the time to look at that right now. I think this would be an excellent candidate for the Features process (and maybe F19 or F20). -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
Am 16.10.2012 09:19, schrieb Nicolas Mailhot: one may say disk storage is nothing these days iw ould say: mulitply it with 20, 50, 100 virtual machines on really expensive SAN-storage where disk space is cheap is not true And I would say : get an entreprisey deduping san for production under load not really a good decision even if, save ressources is always a good idea over the long signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
one may say disk storage is nothing these days iw ould say: mulitply it with 20, 50, 100 virtual machines on really expensive SAN-storage where disk space is cheap is not true And I would say : get an entreprisey deduping san for production under load not really a good decision even if, save ressources is always a good idea over the long That depends. The process of actually de-duping is some what intensive depending on the process but if you clone off a template it's not and you get other advantages like improved use of cache and less I/O on the disk array so it's very dependent. I generally see more improvement than loss. But that is getting off topic as if there's less OS it uses less space whether it's de-duped or not and that is not a bad thing. Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install [was Re: systemd requires HTTP server and serves QR codes]
On 10/16/2012 01:39 PM, Peter Robinson wrote: one may say disk storage is nothing these days iw ould say: mulitply it with 20, 50, 100 virtual machines on really expensive SAN-storage where disk space is cheap is not true And I would say : get an entreprisey deduping san for production under load not really a good decision even if, save ressources is always a good idea over the long That depends. The process of actually de-duping is some what intensive depending on the process but if you clone off a template it's not and you get other advantages like improved use of cache and less I/O on the disk array so it's very dependent. I generally see more improvement than loss. Moreover, I was under the impression that BTRFS is going to do de-duping internally and transparently, because it keeps block checksums anyway so detecting duplicate data comes cheap. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Jesse Keating (jkeat...@j2solutions.net) said: Well, we do currently have the minimal environment, which boils down to @core + the couple things anaconda forces (authconfig, system-config-firewall-base, kernel, bootloader). You can get to that via kickstart with just: %packages @core %end But it's not close to what some of these people want out of a minimal install. For reference: @core + kernel: Install 38 Packages (+157 Dependent packages) Total download size: 128 M Installed size: 506 M But hey, I just want something smaller! systemd + util-linux + bash + initscripts + passwd + yum: Install 7 Packages (+132 Dependent packages) Total download size: 106 M Installed size: 446 M But hey, I don't need to install packages or want python! systemd+ util-linux + bash + initscripts + passwd: Install 6 Packages (+108 Dependent packages) Total download size: 94 M Installed size: 401 M Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote: But hey, I don't need to install packages or want python! systemd+ util-linux + bash + initscripts + passwd: Install 6 Packages (+108 Dependent packages) Total download size: 94 M Installed size: 401 M Of which one quarter is the kernel and the other quarter is glibc locale support, right? -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
From: Bill Nottingham nott...@redhat.com Jesse Keating (jkeat...@j2solutions.net) said: Well, we do currently have the minimal environment, which boils down to @core + the couple things anaconda forces (authconfig, system-config-firewall-base, kernel, bootloader). You can get to that via kickstart with just: %packages @core %end But it's not close to what some of these people want out of a minimal install. For reference: @core + kernel: Install 38 Packages (+157 Dependent packages) Total download size: 128 M Installed size: 506 M But hey, I just want something smaller! systemd + util-linux + bash + initscripts + passwd + yum: Install 7 Packages (+132 Dependent packages) Total download size: 106 M Installed size: 446 M But hey, I don't need to install packages or want python! systemd+ util-linux + bash + initscripts + passwd: Install 6 Packages (+108 Dependent packages) Total download size: 94 M Installed size: 401 M Bill, thanks for that excellent report! It shows me that even if you strip away some of the conveniences, you really don't save that much over our normal minimal install. Very enlightening. -- John Florian -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Matthew Miller (mat...@fedoraproject.org) said: On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote: But hey, I don't need to install packages or want python! systemd+ util-linux + bash + initscripts + passwd: Install 6 Packages (+108 Dependent packages) Total download size: 94 M Installed size: 401 M Of which one quarter is the kernel and the other quarter is glibc locale support, right? Or more: 122659574 kernel 117821428 glibc-common 35623360linux-firmware 14233540coreutils 13845828glibc ... Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, 15 Oct 2012 15:24:09 -0400 Bill Nottingham nott...@redhat.com wrote: Matthew Miller (mat...@fedoraproject.org) said: On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote: But hey, I don't need to install packages or want python! systemd+ util-linux + bash + initscripts + passwd: Install 6 Packages (+108 Dependent packages) Total download size: 94 M Installed size: 401 M Of which one quarter is the kernel and the other quarter is glibc locale support, right? Or more: 122659574 kernel 117821428 glibc-common 35623360linux-firmware 14233540coreutils 13845828glibc I wonder... could we make linux-firmware optional? I would expect many virt env's don't need any firmware to work... (but of course I could be wrong). kevin signature.asc Description: PGP signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Am 15.10.2012 21:34, schrieb Kevin Fenzi:
On Mon, 15 Oct 2012 15:24:09 -0400
Bill Nottingham nott...@redhat.com wrote:
Matthew Miller (mat...@fedoraproject.org) said:
On Mon, Oct 15, 2012 at 02:19:03PM -0400, Bill Nottingham wrote:
But hey, I don't need to install packages or want python!
systemd+ util-linux + bash + initscripts + passwd:
Install 6 Packages (+108 Dependent packages)
Total download size: 94 M
Installed size: 401 M
Of which one quarter is the kernel and the other quarter is glibc
locale support, right?
Or more:
122659574 kernel
117821428 glibc-common
35623360linux-firmware
14233540coreutils
13845828glibc
I wonder... could we make linux-firmware optional?
I would expect many virt env's don't need any firmware to work...
(but of course I could be wrong)
you are right
the dependency was introduced not so long ago
a bugreport was closed with WONT FIX
i went the road below
[root@buildserver:~]$ cat /rpmbuild/SPECS/linux-firmware-dummy.spec
%global checkout 06c8f81
Summary: metapackage to satisfy kernel-dependencies on vmware-servers
Name: linux-firmware-dummy
Version: 20120206
Release: 0.1.git%{checkout}%{?dist}
BuildArch: noarch
Group: System Environment/Libraries
URL: http://www.thelounge.net/
License: GPL
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
Provides: linux-firmware = %{version}
Provides: kernel-firmware = %{version}
%description
metapackage to satisfy kernel-dependencies on vmware-servers
%install
rm -rf ${RPM_BUILD_ROOT}
%files
%clean
rm -rf ${RPM_BUILD_ROOT}
%changelog
* Wed Apr 11 2012 Reindl Harald h.rei...@thelounge.net
- initial build
signature.asc
Description: OpenPGP digital signature
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Kevin Fenzi (ke...@scrye.com) said: 122659574 kernel 117821428 glibc-common 35623360linux-firmware 14233540coreutils 13845828glibc I wonder... could we make linux-firmware optional? I would expect many virt env's don't need any firmware to work... (but of course I could be wrong). It depends. It includes firmware for wired NICs as well as other things, so it depends on what hardware your virtual environment is deciding to emulate. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
minimal install [was Re: systemd requires HTTP server and serves QR codes]
On Mon, Oct 15, 2012 at 03:24:09PM -0400, Bill Nottingham wrote: Total download size: 94 M Installed size: 401 M Of which one quarter is the kernel and the other quarter is glibc locale support, right? Or more: 122659574 kernel 117821428 glibc-common 35623360linux-firmware 14233540coreutils 13845828glibc So, if a minimal image is a very high priority, this could be shrunk. Handwaving aside the difficulty for a moment, if the default kernel split out some of the drivers, maybe that could get down to 60MB. Leave out linux-firmware And glibc-common is almost _entirely_ locale and i18n. Because we still want to be _Fedora_, not a tiny-linux distro, let's leave coreutils and glibc as-is. Still, though, we've shaved off 200+ MB. With this, the three versions of minimal you give come down to about: @core + kernel: 300MB systemd [...] yum: 240MB 20% savings systemd + not yum: 195M 35% savings Chew away at the dependencies and at the size of some of the other packages (python 2to3, I'm looking at you), and we could get the middle option down below 200MB. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Mon, Oct 15, 2012 at 03:43:11PM -0400, Bill Nottingham wrote: It depends. It includes firmware for wired NICs as well as other things, so it depends on what hardware your virtual environment is deciding to emulate. Whatever hardware support is needed to run out-of-box in KVM, Xen, VirtualBox, and, sigh, VMware. If that doesn't also get us EC2 and Rackspace, make it lightweight to add whatever is needed there. That'd just be for _minimal_, of course. Default install would include everything still. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Am 15.10.2012 21:43, schrieb Bill Nottingham: Kevin Fenzi (ke...@scrye.com) said: 122659574 kernel 117821428 glibc-common 35623360linux-firmware 14233540coreutils 13845828glibc I wonder... could we make linux-firmware optional? I would expect many virt env's don't need any firmware to work... (but of course I could be wrong). It depends. It includes firmware for wired NICs as well as other things, so it depends on what hardware your virtual environment is deciding to emulate surely, it depends but the hard dependency is a little too much install it as default at fedora setup is fine so such wired NICs are supported but as example on a VMware platfrom these days you are using vmxnet3 and vmw_pvscsi what means that all virtual hardware is supported from the upstream kernel signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
Am 16.10.2012 01:50, schrieb Kevin Fenzi: On Mon, 15 Oct 2012 19:11:19 -0400 Matthew Miller mat...@fedoraproject.org wrote: On Mon, Oct 15, 2012 at 03:38:36PM -0700, Adam Williamson wrote: I wonder... could we make linux-firmware optional? [...] I'd agree with Harald here. A hard dep seems excessive, just including the package in the @core group (so people who want something *really* minimal can at least take it out) would seem better. I think this is already the case, actually. I just removed it from my F17 test vm and there were no deps and nothing immediately broke. Yeah, I guess I didn't look closely... it's actually a Requires(pre): linux-firmware = blahblahversion So, once the kernel has been installed, you can safely remove it. (If you like) correct me but this leads to re-install it on each kernel-update signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Oct 9, 2012, at 7:14 PM, Jesse Keating wrote: Anaconda isn't going to do that unless there is rpm support to re-docify yourself. To accomplish this right now, every package would have to split out a -docs subpackage with all the docs in it. Anaconda /might/ do what you want in the future, by way of kickstart commands, but that's not something we're going to expose in the UI. Infrastructure Server option comes pretty close, I think. 800MB vs 1.1G, no-GUI, but does appear to have docs. Chris Murphy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Fri, Oct 12, 2012 at 9:29 PM, Bill Nottingham nott...@redhat.com wrote:
Konstantin Ryabitsev (i...@fedoraproject.org) said:
So, in other words, all our existing log analysis tools have to be
modified if they are to be of any use in Fedora 18?
Right, you'll have to port them to understand CEE from updated rsyslog. HTH,
HAND. - note: THIS IS A JOKE.
FWIW - the current plan for Lumberjack/CEE is to keep
/var/log/{messages,secure etc} unmodified, and store the full data in
a separate file. We can easily change this if the logging users don't
think this is the right thing to do, and users who require maximum
logging performance an obviously use a customized information - but
keeping the files and not requiring a flag day for everyone to convert
their tools immediately sounds like a good default.
Mirek
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Sat, Oct 13, 2012 at 1:36 AM, Lennart Poettering mzerq...@0pointer.de wrote: On Fri, 12.10.12 15:29, Bill Nottingham (nott...@redhat.com) wrote: And we've got a lot of technology going around. journald - that's technology. rsyslog - that's technology. libumberlog ceelog - that's technology. THis really makes me wonder where CEE actually belongs in this. Is anybody using this currently? What area is this supposed to cover that is not already covered by the journal or rsyslog? Is there really room for another format besides BSD syslog and journal records? Given that the (udp AND tcp) syslog is the primary multi-platform log transfer protocol in the UNIX world, we need to be able to take Linux log data, including data originally generated by applications using the journal API, and transport it using the syslog protocol. To be really useful, the syslog representation needs not to loose data (e.g. only including the MESSAGE field is not good enough). So, we need a structured representation compatible with the syslog protocol in any case, and Lumberjack/CEE provide one. (And as soon as there is a structured representation compatible with syslog, it is something non-systemd platforms, like Debian or other UNIXes, can use as well.) old rsyslog (pre-Lumberjack/CEE) doesn't cover the structured representation requirement. journal format and protocol don't cover the syslog protocol compatibility requirement. If people want CEE format logs, or plain text logs, maybe journald should grow those as output formats. To me it appears that CEE isn't widely accepted so far (heck, not even properly defined as multiple different vocabularies for fields are floating around), and I am bit unsure where it really fits in the big picture. I am a bit conservative in adding output formatting for CEE if it isn't clear that there is a need for CEE, that it's going to stick around for long and we actually have people using this. The larger vision of CEE is to have a multi-platform event dictionary and using the same event format to represent the same kind of event (so that e.g. and 'user log can be identified and parsed without knowing what specific piece of code generated it). I'm personally not sure how much that is achievable, or how much of the problem space it can cover.[1]. In any case, complying with this would require either modification of applications, or writing a CEE-specific log message translator; it's not something we can magically get by establishing a representation or protocol, or by only converting the structure of the data that currently arrives in the journal without looking at the content. Using the Lumberjack/CEE representation natively would probably make the application modification/translator implementation simpler (e.g. the current proposals rely on nesting in the structure and other syntax that is prohibited in the journal). But as you say, these specifications are not finalized yet. Mirek [1] http://carolina.mff.cuni.cz/~trmac/blog/2011/structured-logging/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
Konstantin Ryabitsev (i...@fedoraproject.org) said: Not sure I can parse this, but IIUC you are wondering whether logwatch is compatible with the journal. Not to my knowledge, no. But adding this should be fairly easy as the output of journalctl is a pixel-perfect copy of the original format, so where it works on /var/log/messages it should simply work on the output of journalctl and all should be good. Note however that with the capabilities of the journal it might be interesting to add journal support to logwatch that goes beyond mere compatibility. For example, tests such as look for messages which are claimed to come from PID xyz but actually came from uvw and suchlike would be really interesting to have. That information is not available in the /var/log/messages format however... So, in other words, all our existing log analysis tools have to be modified if they are to be of any use in Fedora 18? Right, you'll have to port them to understand CEE from updated rsyslog. HTH, HAND. - note: THIS IS A JOKE. MORE SERIOUSLY There are a lot of usage cases that people want from their logging. 1) Administrators want their plain-text logs that they know and love (or at least know and have gotten accustomed to) that they can use their normal unix tools and their homegrown custom shell/awk/perl/python/whatever scripts for parsing. (For the purposes of this discussion, consider logwatch one of those homegrown things, as it basically is that writ large.) 2) System management authors would love to have a mechanism where they can subscribe to particular alerts as they come in, without having to subscribe to all messages, or try and parse the unstructured text of syslog 3) Application developers might want to be able to express stuff they log in a more structured fashion rather than just: (function:line) bad juju happened here in frobnitz 4) Administrators want to be able to do things like 'show me everything sshd did/logged about', or 'show me what happened last Thursday, because I can never get the hang of them.' 5) Standards People want to have messages in the new CEE format, so they can use their new CEE tools on them and merge some of their homegrown tools. 6) Meanwhile, you've got the poor audit logger over there on the side doing its own thing, and there are users who Really Like those audit logs. And we've got a lot of technology going around. journald - that's technology. rsyslog - that's technology. libumberlog ceelog - that's technology. What we've got to do is take the usage cases we have, and the technology we have, and get a coherent solution that covers them. And it's certainly not clear at this point what that solution would be. If people want CEE format logs, or plain text logs, maybe journald should grow those as output formats. Or maybe rsyslog should produce those formats. Maybe rsyslog should grow a journald plugin, so instead of duplicating some of journald's code for associating entries with pid/exec/etc., it can read the already annotated journal stream and add its own metadata spit out whatever formats it wants. (Maybe it already does this!) Maybe rsyslog or journald should take over audit logging in some way. But the point is, there's a lot of work in this space going on on all sides (take ceelog, liumberlog, and journald - all relatively new bits of technology touching portions of this space). And at this point I'd say it's way too early to say that Fedora Shall Be XYZ, or to conversly say that Fedora Shall Not Be XYZ. A full plan for hitting all the usage cases we might want just isn't known. (Although it would be a lot easier to get there if y'all would stop shouting AT PAST each other.) So no, you don't need to change anything for Fedora 18. rsyslog is there by default, journald is there too if you want to look at that. And until we actually have a Plan, rather than just Technology, I'm not sure why you'd say that Fedora will do XYZ in F-19 either. Well, you can probably say that Fedora 19 won't ship with sysklogd by default; that should be safe. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Fri, 12.10.12 15:29, Bill Nottingham (nott...@redhat.com) wrote: Heya, And we've got a lot of technology going around. journald - that's technology. rsyslog - that's technology. libumberlog ceelog - that's technology. THis really makes me wonder where CEE actually belongs in this. Is anybody using this currently? What area is this supposed to cover that is not already covered by the journal or rsyslog? Is there really room for another format besides BSD syslog and journal records? So, what's our story here with CEE? If people want CEE format logs, or plain text logs, maybe journald should grow those as output formats. To me it appears that CEE isn't widely accepted so far (heck, not even properly defined as multiple different vocabularies for fields are floating around), and I am bit unsure where it really fits in the big picture. I am a bit conservative in adding output formatting for CEE if it isn't clear that there is a need for CEE, that it's going to stick around for long and we actually have people using this. Or maybe rsyslog should produce those formats. Maybe rsyslog should grow a journald plugin, so instead of duplicating some of journald's code for associating entries with pid/exec/etc., it can read the already annotated journal stream and add its own metadata spit out whatever formats it wants. (Maybe it already does this!) Yes, this would certainly be useful. If rsyslog wants access to the full data stream systemd generates then using our C APIs is a good choice, it will get all meta data, and can process them the way they want. Maybe rsyslog or journald should take over audit logging in some way. Since the audit logs contain a lot of useful data we definitely want to acquire auditing as another input for the journal. In fact, Eric has been working on kernel support to allow the journal to get a copy of the audit stream without interfering with auditd. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Wed, Oct 10, 2012 at 08:43:22AM -0400, Matthew Miller wrote: On Wed, Oct 10, 2012 at 07:17:58PM +0800, Daniel Veillard wrote: libxml2 takes up 5.2M, of which 3.8M is docs It really should go in -devel, I agree ! Check it out -- we've accomplished something with this thread. :) Done, fixed in rawhide, base package install size is now down to 1.6 MB, the docs were already in the -devel package so I kept them there. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ dan...@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install set tuning [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 08:16:49AM -0500, Michael Cronenworth wrote: Alexander Larsson wrote: Honestly, we should be building glib2 with --disable-fam, since glib will prefer the inotify notification module anyway (it has prio 20 and fam prio 10). It looks[1] like Matthias was watching this thread. Yay! [1] http://pkgs.fedoraproject.org/cgit/glib2.git/commit/?id=3d798c3fdcbab4255323c570fcfc0090bd2980df Let's celebrate one less use of gamin ! Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ dan...@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install set tuning [was Re: systemd requires HTTP server and serves QR codes]
On 10/9/12 12:34 PM, Adam Jackson wrote: On 10/9/12 9:18 AM, Lennart Poettering wrote: From the list of packages this minimal set still installs, that I'd really like to see gone: chkconfig gamin info systemd-sysv chkconfig seems like it could have the 'alternatives' bit split off. I've not investigated this in detail. Took a look this morning, it appears alternatives is almost completely orthogonal to the rest of the package. Naturally the bit that ruins everything is translations since the locale data is all mushed together among alternatives chkconfig and ntsysv, but that's straightforward to detangle. (Not that I've done so.) That'd drop the size on disk from ~700k for chkconfig to probably around 400k for just alternatives. Not a huge win, but not terrible either, and certainly Correct for a systemd world. Bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=865496 Better, though, is looking at what's pulling it in, a Requires(post) in openssh-server (actually for %triggerun and %triggerpostun, but rpm doesn't know how to Requires for those), and in particular for upgrading from versions of openssh-server older than what shipped in F16 Gold. And, the scriptlets are properly immunized against /sbin/chkconfig not existing, which means strictly we could drop the Requires(post) for it already. Bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=865498 - ajax -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On 10/09/2012 09:42 PM, Matthew Miller wrote: On Tue, Oct 09, 2012 at 11:59:08AM -0400, Simo Sorce wrote: In current versions .service is implied if no extension is provided: https://bugs.freedesktop.org/show_bug.cgi?id=39386 About time :-) Awesome. And I want to take a moment to thank everyone for listening to these concerns. I'm optimistic that we can make this all work very nicely. Is this documented in the relevant man pages as well? Rahul -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On 10/09/2012 10:03 PM, Lennart Poettering wrote: On Tue, 09.10.12 17:25, Panu Matilainen (pmati...@laiskiainen.org) wrote: Can I pass this somehow to yum? Or do I have to creat a macro file for this? You can set it in yum.conf (tsflags=nodocs), but then rpm wont know about it (so if you install directly with rpm, it'll still install the docs). Putting it in the macro configuration ensures everything going through librpm honors it. This appears very much broken. I just tried to install a container with the following command line: yum -y \ --setopt=tsflags=nodocs \ --setopt=keepcache=0 \ --installroot=/home/lennart/minimal/install \ --nogpg \ --releasever=18 \ '--disablerepo=*' \ --enablerepo=fedora \ install systemd passwd openssh-server rpm And this fails when installing gawk with: ... Installing : shared-mime-info-1.0-5.fc18.x86_64 51/106 Installing : grep-2.14-1.fc18.x86_64 52/106 install-info: No such file or directory for /usr/share/info/grep.info.gz FWIW this is one of the problems with --nodocs (and _install_langs as well) - packages generally expect to be installed as whole and dont conditionalize stuff like install-info based on the file existence, so they end up spitting errors like this. Installing : gawk-4.0.1-2.fc18.x86_64 53/106 Error unpacking rpm package gawk-4.0.1-2.fc18.x86_64 error: unpacking of archive failed on file /usr/share/man/man1/gawk.1.gz;507472b1: cpio: Missing hard link(s) Installing : libidn-1.25-3.fc18.x86_64 54/106 error: gawk-4.0.1-2.fc18.x86_64: install failed Oh, that. Just found and fixed this upstream last week, but didn't realize it affected rpm 4.10 as well. This comes from an ancient piece of code that attempts to verify all hardlinks got created, its always been there but the result of this verification was never used for anything prior to rpm 4.10. The issue is simply that the verification fails to account for files (hardlinks) skipped on purpose, which is a relatively rare condition. Will fix shortly. - Panu - -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On 09/10/12 15:16, Lennart Poettering wrote: journalctl -D pathtothejournalfiles Lennart Can journalctl send the logs via logwatch? -- Regards, Frank Jack of all, fubars -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
Lennart Poettering wrote: On Tue, 09.10.12 09:09, Chris Adams (cmad...@hiwaay.net) wrote: How do you read this log when the system is not running (e.g. mounting filesystems of a drive on another system, running from a rescue image, etc.)? journalctl -D pathtothejournalfiles So the rescue system (which might not always be Fedora) must have journalctl installed. Is the file format stable, or can it break if the rescue system has a different version of journalctl? Is the format perchance even documented so that other tools for reading logs could be written? Björn Persson -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 09:50:43AM +0200, Björn Persson wrote: Lennart Poettering wrote: On Tue, 09.10.12 09:09, Chris Adams (cmad...@hiwaay.net) wrote: How do you read this log when the system is not running (e.g. mounting filesystems of a drive on another system, running from a rescue image, etc.)? journalctl -D pathtothejournalfiles So the rescue system (which might not always be Fedora) must have journalctl installed. Is the file format stable, or can it break if the rescue system has a different version of journalctl? Is the format perchance even documented so that other tools for reading logs could be written? This would be essential for libguestfs tools to parse logs out of guests (we do it now by reading /var/log/messages etc which has all of the properties you state). Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 09:54:28AM +0100, Richard W.M. Jones wrote: On Wed, Oct 10, 2012 at 09:50:43AM +0200, Björn Persson wrote: Lennart Poettering wrote: On Tue, 09.10.12 09:09, Chris Adams (cmad...@hiwaay.net) wrote: How do you read this log when the system is not running (e.g. mounting filesystems of a drive on another system, running from a rescue image, etc.)? journalctl -D pathtothejournalfiles So the rescue system (which might not always be Fedora) must have journalctl installed. Is the file format stable, or can it break if the rescue system has a different version of journalctl? Is the format perchance even documented so that other tools for reading logs could be written? This would be essential for libguestfs tools to parse logs out of guests (we do it now by reading /var/log/messages etc which has all of the properties you state). I checked out the code, and it does seem as if the format is intended to be backwards compatible. It uses a set of filesystem-like compatible and incompatible flags, so presumably a sufficiently recent journalctl would be able to read any previous version of the binary file format. It would be nice to have this confirmed, and indeed enshrined in the policy of the journal, because it is IMHO essential that the binary log files will always be readable. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-df lists disk usage of guests without needing to install any software inside the virtual machine. Supports Linux and Windows. http://et.redhat.com/~rjones/virt-df/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install set tuning [was Re: systemd requires HTTP server and serves QR codes]
On tis, 2012-10-09 at 12:34 -0400, Adam Jackson wrote: On 10/9/12 9:18 AM, Lennart Poettering wrote: From the list of packages this minimal set still installs, that I'd really like to see gone: chkconfig gamin info systemd-sysv chkconfig seems like it could have the 'alternatives' bit split off. I've not investigated this in detail. gamin is glib2's fault. Strictly it's an implementation detail, it's not like the glib headers expose gamin types. It's a little irritating that you end up with both libfam.so and libgamin-1.so installed, with literally identical APIs. Honestly, we should be building glib2 with --disable-fam, since glib will prefer the inotify notification module anyway (it has prio 20 and fam prio 10). -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
Richard W.M. Jones wrote: I checked out the code, and it does seem as if the format is intended to be backwards compatible. It uses a set of filesystem-like compatible and incompatible flags, so presumably a sufficiently recent journalctl would be able to read any previous version of the binary file format. So if my Fedora box won't boot, and I take the disk out and mount it in a CentOS box, I might not be able to read the log because journalctl in CentOS might be too old? Not fun. Björn Persson -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 12:00:41PM +0200, Björn Persson wrote: Richard W.M. Jones wrote: I checked out the code, and it does seem as if the format is intended to be backwards compatible. It uses a set of filesystem-like compatible and incompatible flags, so presumably a sufficiently recent journalctl would be able to read any previous version of the binary file format. So if my Fedora box won't boot, and I take the disk out and mount it in a CentOS box, I might not be able to read the log because journalctl in CentOS might be too old? Not fun. You can easily just boot the current Fedora Live CD on your other box instead of CentOS. Or boot the Fedora Live CD inside a KVM guest and mount the broken disk to your guest instead of the host. Daniel -- |: http://berrange.com -o-http://www.flickr.com/photos/dberrange/ :| |: http://libvirt.org -o- http://virt-manager.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :| -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On 10/10/2012 08:54 AM, Richard W.M. Jones wrote: This would be essential for libguestfs tools to parse logs out of guests (we do it now by reading /var/log/messages etc which has all of the properties you state). I'm not sure how you are doing this currently but for shutdown guest I assume you would mount then run something like journalctl -D /path/to/journal/files | the script you use to parse the logs And or use systemd-gateway for active guests as in https://jira.skyggnir.is/secure/admin/user/ViewUser.jspa?atl_token=B392-8G0S-SYQQ-21VP%7Ccbcceb6069c5f860c32629fc73971afb2f37df29%7Clinname=haf-sbj# systemctl start systemd-journal-gatewayd.service Then run # wget http://localhost:19531/entries To download the journal contents in a /var/log/messages compatible format Or if you want to download it in JASON compatible format # curl -HAccept: application/json http://localhost:19531/entries If you simply want to browse the log file of an running guest you would just visit the http://IP:19531/browse http://localhost:19531/browse in your favorite browser JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On 10/10/2012 07:54 AM, Frank Murphy wrote: On 09/10/12 15:16, Lennart Poettering wrote: journalctl -D pathtothejournalfiles Lennart Can journalctl send the logs via logwatch? As far as I know logwatch has not been patched to parse and use journal. Try filing an RFE against logwatch for that JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 10:11:03AM +, Jóhann B. Guðmundsson wrote: On 10/10/2012 08:54 AM, Richard W.M. Jones wrote: This would be essential for libguestfs tools to parse logs out of guests (we do it now by reading /var/log/messages etc which has all of the properties you state). I'm not sure how you are doing this currently but for shutdown guest I assume you would mount then run something like journalctl -D /path/to/journal/files | the script you use to parse the logs The question is whether this works with different versions of journal on the host and in the guest. A typical case we have to deal with is someone running a stable RHEL host, and Fedora guests (ie. host version guest version). For RHEL 6 I guess this will involve backporting. This is why stable, well-documented formats like plain text are better. That's not to say however that the journal isn't possible to handle -- the format looks like someone thought about this case to some extent, and we already have to deal with undocumented binary formats like the Windows registry and Windows event log. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Tue, Oct 09, 2012 at 09:45:09PM -0400, Matthew Miller wrote: On Tue, Oct 09, 2012 at 06:14:39PM -0700, Jesse Keating wrote: Anaconda isn't going to do that unless there is rpm support to re-docify yourself. To accomplish this right now, every package would have to split out a -docs subpackage with all the docs in it. Anaconda /might/ do what you want in the future, by way of kickstart commands, but that's not something we're going to expose in the UI. I was hoping that there might be a few packages we could target to split out the docs from to get us most of the way there, but it really just adds up. There are a few that might be quick but very small wins, though: libxml2 takes up 5.2M, of which 3.8M is docs It really should go in -devel, I agree ! I'll file bugs against these. Much less work than anything else I've seen here, and gets us something -- particularly libxml2, where it's mostly developer docs. no disagreement. Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ dan...@veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
Daniel P. Berrange wrote: On Wed, Oct 10, 2012 at 12:00:41PM +0200, Björn Persson wrote: So if my Fedora box won't boot, and I take the disk out and mount it in a CentOS box, I might not be able to read the log because journalctl in CentOS might be too old? Not fun. You can easily just boot the current Fedora Live CD on your other box instead of CentOS. Or boot the Fedora Live CD inside a KVM guest and mount the broken disk to your guest instead of the host. Downloading a CD image, setting up a virtual machine and figuring out how to mount a disk on it might not be a very hard problem, but it does take more time and effort than typing less /mnt/var/log/messages. Björn Persson -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Tue, 09.10.12 21:26, Matthew Miller (mat...@fedoraproject.org) wrote: On Tue, Oct 09, 2012 at 05:19:59PM -0700, J. Randall Owens wrote: Just on the naming, I'd rather steer clear of the actual concept, let me get this straight: You want a group called adm, presumably short for administrator, the point of which is that it can view system things, but not actually *administer* them? Why on Earth call it adm? The group is already there, so it's not a big stretch, but I agree the naming is confusing when used in this way. (wheel isn't exactly straightforward either, but at least it's Traditional.) As I already mentioned: adm has been around for along time, and has been used in this context in Debian since about forever. We just adopted the same logic in systemd that already made sense on Debian for a long time. In systemd we try to unify Linux a bit, part of that is to take influences and be inspired by the various distros around. In this case the Debian way made most sense to us, so we made it the default in systemd, too. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 09:50, Björn Persson (bjorn@rombobjörn.se) wrote: Lennart Poettering wrote: On Tue, 09.10.12 09:09, Chris Adams (cmad...@hiwaay.net) wrote: How do you read this log when the system is not running (e.g. mounting filesystems of a drive on another system, running from a rescue image, etc.)? journalctl -D pathtothejournalfiles So the rescue system (which might not always be Fedora) must have journalctl installed. Is the file format stable, or can it break if the rescue system has a different version of journalctl? Is the format perchance even documented so that other tools for reading logs could be written? Yes, they need journalctl installed. Yes, the format is stable, we haven't broken it since we first came up with it, and we are happy with it so it is unlikely that we will break it any time soon. The format is designed to be extensible while staying compatible and there are two bit flag fields in the header that encode feature flags that allow us to evolve the format as needed while still clarifying the level of compatibility. That means the newest journalctl should always be capable to read all old files, and to a lesser degree even old journalctls decode newer files. Since we are quite confident that the design of the file format is pretty OK I actually intend to document it in the systemd wiki soon. Maybe this will happen already by the time F18 is released, but most likely around F19 the latest. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Wed, Oct 10, 2012 at 12:58 PM, Lennart Poettering mzerq...@0pointer.de wrote: On Tue, 09.10.12 21:26, Matthew Miller (mat...@fedoraproject.org) wrote: On Tue, Oct 09, 2012 at 05:19:59PM -0700, J. Randall Owens wrote: Just on the naming, I'd rather steer clear of the actual concept, let me get this straight: You want a group called adm, presumably short for administrator, the point of which is that it can view system things, but not actually *administer* them? Why on Earth call it adm? The group is already there, so it's not a big stretch, but I agree the naming is confusing when used in this way. (wheel isn't exactly straightforward either, but at least it's Traditional.) As I already mentioned: adm has been around for along time, and has been used in this context in Debian since about forever. We just adopted the same logic in systemd that already made sense on Debian for a long time. In systemd we try to unify Linux a bit, part of that is to take influences and be inspired by the various distros around. In this case the Debian way made most sense to us, so we made it the default in systemd, too. Maybe you could take more influences and be inspired by debian to split some of the non core requirements of systemd off into some sub packages so people can remove http, qrencode and possibly even the journal if they so wished. Peter -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Wed, Oct 10, 2012 at 07:17:58PM +0800, Daniel Veillard wrote: libxml2 takes up 5.2M, of which 3.8M is docs It really should go in -devel, I agree ! Check it out -- we've accomplished something with this thread. :) -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 12:12:26PM +0530, Rahul Sundaram wrote: About time :-) Awesome. And I want to take a moment to thank everyone for listening to these concerns. I'm optimistic that we can make this all work very nicely. Is this documented in the relevant man pages as well? In fact, I think it's big enough that it should go in the release notes. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 12:49 PM, Richard W.M. Jones rjo...@redhat.com wrote: On Wed, Oct 10, 2012 at 10:11:03AM +, Jóhann B. Guðmundsson wrote: On 10/10/2012 08:54 AM, Richard W.M. Jones wrote: This would be essential for libguestfs tools to parse logs out of guests (we do it now by reading /var/log/messages etc which has all of the properties you state). I'm not sure how you are doing this currently but for shutdown guest I assume you would mount then run something like journalctl -D /path/to/journal/files | the script you use to parse the logs The question is whether this works with different versions of journal on the host and in the guest. A typical case we have to deal with is someone running a stable RHEL host, and Fedora guests (ie. host version guest version). Can't you run the journal from the guest? Or does this open another can of worms? -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Wed, Oct 10, 2012 at 01:58:58PM +0200, Lennart Poettering wrote: The group is already there, so it's not a big stretch, but I agree the naming is confusing when used in this way. (wheel isn't exactly straightforward either, but at least it's Traditional.) As I already mentioned: adm has been around for along time, and has been used in this context in Debian since about forever. We just adopted the same logic in systemd that already made sense on Debian for a long time. Documented here: http://www.debian.org/doc/manuals/securing-debian-howto/ch12.en.html adm: Group adm is used for system monitoring tasks. Members of this group can read many log files in /var/log, and can use xconsole. /Historically, var/log was /usr/adm (and later /var/adm), thus the /name of the group. And as I mentioned, I don't mind making adm able to read the system messages (assuming we address the authpriv issue and discuss and document the change). It just should also use the wheel group to make sense with Fedora. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 02:54:13PM +0200, drago01 wrote: On Wed, Oct 10, 2012 at 12:49 PM, Richard W.M. Jones rjo...@redhat.com wrote: On Wed, Oct 10, 2012 at 10:11:03AM +, Jóhann B. Guðmundsson wrote: On 10/10/2012 08:54 AM, Richard W.M. Jones wrote: This would be essential for libguestfs tools to parse logs out of guests (we do it now by reading /var/log/messages etc which has all of the properties you state). I'm not sure how you are doing this currently but for shutdown guest I assume you would mount then run something like journalctl -D /path/to/journal/files | the script you use to parse the logs The question is whether this works with different versions of journal on the host and in the guest. A typical case we have to deal with is someone running a stable RHEL host, and Fedora guests (ie. host version guest version). Can't you run the journal from the guest? Or does this open another can of worms? Security worms, yes. We try very much to avoid running code from the guest. cf. grub problems previously discussed on this list. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones New in Fedora 11: Fedora Windows cross-compiler. Compile Windows programs, test, and build Windows installers. Over 70 libraries supprt'd http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: minimal install set tuning [was Re: systemd requires HTTP server and serves QR codes]
Alexander Larsson wrote: Honestly, we should be building glib2 with --disable-fam, since glib will prefer the inotify notification module anyway (it has prio 20 and fam prio 10). It looks[1] like Matthias was watching this thread. Yay! [1] http://pkgs.fedoraproject.org/cgit/glib2.git/commit/?id=3d798c3fdcbab4255323c570fcfc0090bd2980df -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: systemd requires HTTP server and serves QR codes
On Wed, 2012-10-10 at 13:58 +0200, Lennart Poettering wrote: On Tue, 09.10.12 21:26, Matthew Miller (mat...@fedoraproject.org) wrote: On Tue, Oct 09, 2012 at 05:19:59PM -0700, J. Randall Owens wrote: Just on the naming, I'd rather steer clear of the actual concept, let me get this straight: You want a group called adm, presumably short for administrator, the point of which is that it can view system things, but not actually *administer* them? Why on Earth call it adm? The group is already there, so it's not a big stretch, but I agree the naming is confusing when used in this way. (wheel isn't exactly straightforward either, but at least it's Traditional.) As I already mentioned: adm has been around for along time, and has been used in this context in Debian since about forever. We just adopted the same logic in systemd that already made sense on Debian for a long time. It's very nice that debian uses this concept, but Fedora doesn't and had stricter policies. Can you explain the rationale for relaxing them (esp. wrt /var/log/secure aka authpriv.* messages) In systemd we try to unify Linux a bit, part of that is to take influences and be inspired by the various distros around. In this case the Debian way made most sense to us, so we made it the default in systemd, too. Except this is a regression in the security model IMHO. Note I am not saying it must not be done, but I want to understand if there is any value on it or you just picked it 'because Debian'. Simo. -- Simo Sorce * Red Hat, Inc * New York -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
I apologize, I'm ill and not generally up to providing detailed responses. So just some sourced facts to counter [1] untruths. For education on what current syslogs do, http://blog.gerhards.net/2012/10/main-advantages-of-rsyslog-v7-vs-v5.html is a possible start and http://www.rsyslog.com/doc/manual.html contains much more. On Tue, Oct 9, 2012 at 11:24 PM, Lennart Poettering mzerq...@0pointer.de wrote: I am not generally against adding time-based rotation, but really, this is much less of a necessity than other things the journal provides, which syslog does not: for example per-service rate limits, False. http://www.rsyslog.com/doc/imuxsock.html, There is input rate limiting available, currently enabled by default in Fedora. and unfakable meta-data for log messages. False: http://www.rsyslog.com/doc/imuxsock.html, trusted syslog properties are available (and in v7 they can be enabled in the Fedora configuration by default) On Wed, Oct 10, 2012 at 12:08 AM, Lennart Poettering mzerq...@0pointer.de wrote: I am not a security guy, but having logs where unprivileged users cannot insert undetectable fakes (Re: the implied claim that systemd provides that): For the unprivileged user part, see above. For the cryptographic protection, false. http://cgit.freedesktop.org/systemd/systemd/tree/man/journalctl.xml#n358 defaults to 15 minutes, which is an eternity. Mirek [1] An adjective belongs here. I can think of about 10 candidates, but I feel too ill and grumpy to trust myself to choose well. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 5:05 PM, Miloslav Trmač m...@volny.cz wrote: On Tue, Oct 9, 2012 at 11:24 PM, Lennart Poettering mzerq...@0pointer.de wrote: which syslog does not: for example per-service rate limits, False. http://www.rsyslog.com/doc/imuxsock.html, There is input rate limiting available, currently enabled by default in Fedora. Insufficient in rsyslog. And it's right what Lennart said. This really needs to be per service/user not per pid. Pids are almost entirely useless to key-off here. and unfakable meta-data for log messages. False: http://www.rsyslog.com/doc/imuxsock.html, trusted syslog properties are available (and in v7 they can be enabled in the Fedora configuration by default) It's well meant, but really, it sounds more like a joke. Adding garbage to the end of the human readable plain text is not comparable with the journal. On Wed, Oct 10, 2012 at 12:08 AM, Lennart Poettering mzerq...@0pointer.de wrote: I am not a security guy, but having logs where unprivileged users cannot insert undetectable fakes (Re: the implied claim that systemd provides that): It surely does provide it. Rsyslog can do something similar, but really, with pushing stuff into plain text files, mixing it into the human readable message it can't really get too far without creating a mess in the files. For the unprivileged user part, see above. For the cryptographic protection, false. It's not about tamper-proof log files, it was about unfakeable message source context. http://cgit.freedesktop.org/systemd/systemd/tree/man/journalctl.xml#n358 defaults to 15 minutes, which is an eternity. The sealing was not even mentioned, but it's still better than nothing. And 15 min are the current default, and this will change as soon as the details are hashed out to efficiently move the sealing forward in time. [1] An adjective belongs here. I can think of about 10 candidates, but I feel too ill and grumpy to trust myself to choose well. I'm sure you should wait until you are back to full speed. You comparision seem pretty bad researched. :) Thanks, Kay -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 6:13 PM, Kay Sievers k...@vrfy.org wrote: and unfakable meta-data for log messages. False: http://www.rsyslog.com/doc/imuxsock.html, trusted syslog properties are available (and in v7 they can be enabled in the Fedora configuration by default) It's well meant, but really, it sounds more like a joke. Adding garbage to the end of the human readable plain text is not comparable with the journal. That's where the v7 reference comes in - stored as a Lumberjack field. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Tue, 09.10.12 22:30, Simo Sorce (s...@redhat.com) wrote: logrotate has time based policies for very good reasons. Yeah, because Unix doesn't really allow much else... Oh come on, stop bashing unix, logrotate could certainly grow a size checking policy if people felt the need, unix is not holding you back, in fact you are building this stuff on a unix-like system. Ah, Unix cron can start things based on disk space changes? Interesting, I wasn't aware of that. I thought it only could start logrotate by time, not by disk space changes... Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10 Oct 2012, Lennart Poettering wrote: On Tue, 09.10.12 22:30, Simo Sorce (s...@redhat.com) wrote: logrotate has time based policies for very good reasons. Yeah, because Unix doesn't really allow much else... Oh come on, stop bashing unix, logrotate could certainly grow a size checking policy if people felt the need, unix is not holding you back, in fact you are building this stuff on a unix-like system. Ah, Unix cron can start things based on disk space changes? Interesting, I wasn't aware of that. I thought it only could start logrotate by time, not by disk space changes... yum info incron Description : This program is an inotify cron system. : It consists of a daemon and a table manipulator. : You can use it a similar way as the regular cron. : The difference is that the inotify cron handles : filesystem events rather than time periods. -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 17:05, Miloslav Trmač (m...@volny.cz) wrote: On Tue, Oct 9, 2012 at 11:24 PM, Lennart Poettering mzerq...@0pointer.de wrote: I am not generally against adding time-based rotation, but really, this is much less of a necessity than other things the journal provides, which syslog does not: for example per-service rate limits, False. http://www.rsyslog.com/doc/imuxsock.html, There is input rate limiting available, currently enabled by default in Fedora. I know, I asked Rainer to add that. But this is actually much less useful than what the journal does: it's per-pid, not per-service. and unfakable meta-data for log messages. False: http://www.rsyslog.com/doc/imuxsock.html, trusted syslog properties are available (and in v7 they can be enabled in the Fedora[M#}5 configuration by default) Yes, I know, I asked Rainer to add that. But it's not on, and there's no accepted syntax for syslog messages to carry this, and it's pretty incomplete. No selinux labels, no audit, and no service information. For the cryptographic protection, false. http://cgit.freedesktop.org/systemd/systemd/tree/man/journalctl.xml#n358 defaults to 15 minutes, which is an eternity. This is not what I talked of. I simply was pointing to the fact that messages end up in /var/log/messages that cannot be traced back to who actually sent them. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 14:16, Seth Vidal (skvi...@fedoraproject.org) wrote: On Tue, 09.10.12 22:30, Simo Sorce (s...@redhat.com) wrote: logrotate has time based policies for very good reasons. Yeah, because Unix doesn't really allow much else... Oh come on, stop bashing unix, logrotate could certainly grow a size checking policy if people felt the need, unix is not holding you back, in fact you are building this stuff on a unix-like system. Ah, Unix cron can start things based on disk space changes? Interesting, I wasn't aware of that. I thought it only could start logrotate by time, not by disk space changes... yum info incron Description : This program is an inotify cron system. : It consists of a daemon and a table manipulator. : You can use it a similar way as the regular cron. : The difference is that the inotify cron handles : filesystem events rather than time periods. And rsyslog pulls that in? I wasn't aware of that. I am learning new stuff every day... Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10 Oct 2012, Lennart Poettering wrote: On Wed, 10.10.12 14:16, Seth Vidal (skvi...@fedoraproject.org) wrote: On Tue, 09.10.12 22:30, Simo Sorce (s...@redhat.com) wrote: logrotate has time based policies for very good reasons. Yeah, because Unix doesn't really allow much else... Oh come on, stop bashing unix, logrotate could certainly grow a size checking policy if people felt the need, unix is not holding you back, in fact you are building this stuff on a unix-like system. Ah, Unix cron can start things based on disk space changes? Interesting, I wasn't aware of that. I thought it only could start logrotate by time, not by disk space changes... yum info incron Description : This program is an inotify cron system. : It consists of a daemon and a table manipulator. : You can use it a similar way as the regular cron. : The difference is that the inotify cron handles : filesystem events rather than time periods. And rsyslog pulls that in? I wasn't aware of that. I am learning new stuff every day... I never said anything like that. I said it existed. Please stop adding words where they are not. -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 08:54, Frank Murphy (frankl...@gmail.com) wrote: On 09/10/12 15:16, Lennart Poettering wrote: journalctl -D pathtothejournalfiles Lennart Can journalctl send the logs via logwatch? Not sure I can parse this, but IIUC you are wondering whether logwatch is compatible with the journal. Not to my knowledge, no. But adding this should be fairly easy as the output of journalctl is a pixel-perfect copy of the original format, so where it works on /var/log/messages it should simply work on the output of journalctl and all should be good. Note however that with the capabilities of the journal it might be interesting to add journal support to logwatch that goes beyond mere compatibility. For example, tests such as look for messages which are claimed to come from PID xyz but actually came from uvw and suchlike would be really interesting to have. That information is not available in the /var/log/messages format however... Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 09:54, Richard W.M. Jones (rjo...@redhat.com) wrote: On Wed, Oct 10, 2012 at 09:50:43AM +0200, Björn Persson wrote: Lennart Poettering wrote: On Tue, 09.10.12 09:09, Chris Adams (cmad...@hiwaay.net) wrote: How do you read this log when the system is not running (e.g. mounting filesystems of a drive on another system, running from a rescue image, etc.)? journalctl -D pathtothejournalfiles So the rescue system (which might not always be Fedora) must have journalctl installed. Is the file format stable, or can it break if the rescue system has a different version of journalctl? Is the format perchance even documented so that other tools for reading logs could be written? This would be essential for libguestfs tools to parse logs out of guests (we do it now by reading /var/log/messages etc which has all of the properties you state). I'd recommend simply using our C API for this. For details see: http://www.freedesktop.org/software/systemd/man/ Look for the various APIs with the sd_journal_ prefix. With those you get full access to the journal. Here you find an example how to do this: http://www.freedesktop.org/software/systemd/man/sd_journal_next.html Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10.10.12 10:12, Richard W.M. Jones (rjo...@redhat.com) wrote: On Wed, Oct 10, 2012 at 09:54:28AM +0100, Richard W.M. Jones wrote: On Wed, Oct 10, 2012 at 09:50:43AM +0200, Björn Persson wrote: Lennart Poettering wrote: On Tue, 09.10.12 09:09, Chris Adams (cmad...@hiwaay.net) wrote: How do you read this log when the system is not running (e.g. mounting filesystems of a drive on another system, running from a rescue image, etc.)? journalctl -D pathtothejournalfiles So the rescue system (which might not always be Fedora) must have journalctl installed. Is the file format stable, or can it break if the rescue system has a different version of journalctl? Is the format perchance even documented so that other tools for reading logs could be written? This would be essential for libguestfs tools to parse logs out of guests (we do it now by reading /var/log/messages etc which has all of the properties you state). I checked out the code, and it does seem as if the format is intended to be backwards compatible. It uses a set of filesystem-like compatible and incompatible flags, so presumably a sufficiently recent journalctl would be able to read any previous version of the binary file format. It would be nice to have this confirmed, and indeed enshrined in the policy of the journal, because it is IMHO essential that the binary log files will always be readable. Yes, the compatible and incompatible flag bit fields are precisely to provide good compatibility as the format evolves. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 2:32 PM, Lennart Poettering mzerq...@0pointer.de wrote: Can journalctl send the logs via logwatch? Not sure I can parse this, but IIUC you are wondering whether logwatch is compatible with the journal. Not to my knowledge, no. But adding this should be fairly easy as the output of journalctl is a pixel-perfect copy of the original format, so where it works on /var/log/messages it should simply work on the output of journalctl and all should be good. Note however that with the capabilities of the journal it might be interesting to add journal support to logwatch that goes beyond mere compatibility. For example, tests such as look for messages which are claimed to come from PID xyz but actually came from uvw and suchlike would be really interesting to have. That information is not available in the /var/log/messages format however... So, in other words, all our existing log analysis tools have to be modified if they are to be of any use in Fedora 18? Best, -- Konstantin Ryabitsev LinuxFoundation.org Montréal, Québec -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 8:37 PM, Konstantin Ryabitsev i...@fedoraproject.org wrote: On Wed, Oct 10, 2012 at 2:32 PM, Lennart Poettering mzerq...@0pointer.de wrote: Can journalctl send the logs via logwatch? Not sure I can parse this, but IIUC you are wondering whether logwatch is compatible with the journal. Not to my knowledge, no. But adding this should be fairly easy as the output of journalctl is a pixel-perfect copy of the original format, so where it works on /var/log/messages it should simply work on the output of journalctl and all should be good. Note however that with the capabilities of the journal it might be interesting to add journal support to logwatch that goes beyond mere compatibility. For example, tests such as look for messages which are claimed to come from PID xyz but actually came from uvw and suchlike would be really interesting to have. That information is not available in the /var/log/messages format however... So, in other words, all our existing log analysis tools have to be modified if they are to be of any use in Fedora 18? What part of Run the syslog daemon like you always did, if you need syslog files. did you not understand? Kay -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 02:37:05PM -0400, Konstantin Ryabitsev wrote: So, in other words, all our existing log analysis tools have to be modified if they are to be of any use in Fedora 18? No, not in the even slightest. I don't think that's even up for discussion. -- Matthew Miller ☁☁☁ Fedora Cloud Architect ☁☁☁ mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, 10 Oct 2012, Kay Sievers wrote: So, in other words, all our existing log analysis tools have to be modified if they are to be of any use in Fedora 18? What part of Run the syslog daemon like you always did, if you need syslog files. did you not understand? Kay, This is not an acceptable tone. There is no need for this sort of sarcasm or snark. Please amend this in the future. -sv -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 2:39 PM, Kay Sievers k...@vrfy.org wrote: So, in other words, all our existing log analysis tools have to be modified if they are to be of any use in Fedora 18? What part of Run the syslog daemon like you always did, if you need syslog files. did you not understand? Well, hang on, Kay. My understanding was that we're trying to make syslog an optional install in Fedora 18 (or is it 19?). If that is the case, then even if I require rsyslog for a package, that won't work unless rsyslog is started and running. So, sysadmin's experience changes: Was: Install logwatch. Becomes: Install logwatch. Make sure you install and enable rsyslog. I just want to make sure people are aware of the change. Best, -- Konstantin Ryabitsev LinuxFoundation.org Montréal, Québec -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: replacing rsyslogd in minimal with journald [was Re: systemd requires HTTP server and serves QR codes]
On Wed, Oct 10, 2012 at 8:44 PM, Konstantin Ryabitsev i...@fedoraproject.org wrote: On Wed, Oct 10, 2012 at 2:39 PM, Kay Sievers k...@vrfy.org wrote: So, in other words, all our existing log analysis tools have to be modified if they are to be of any use in Fedora 18? What part of Run the syslog daemon like you always did, if you need syslog files. did you not understand? Well, hang on, Kay. My understanding was that we're trying to make syslog an optional install in Fedora 18 (or is it 19?). Surely not f18, and there is not even a feature for f19 as of now. If that is the case, then even if I require rsyslog for a package, that won't work unless rsyslog is started and running. Services can pull-in service dependencies to start stuff they depend on, it's unreleated RPM dependencies. So, sysadmin's experience changes: Was: Install logwatch. Becomes: Install logwatch. Make sure you install and enable rsyslog. I just want to make sure people are aware of the change. Ah, sorry that I was just unable to translate: all our existing log analysis tools have to be modified if they are to be of any use in Fedora to just want to make sure ... you install and enable rsyslog. :) Kay -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
