Updating config option layout and scripts

[Don Zickus]

Hi Laura,

As per our conversation, here is my pull request for the config changes:
https://pagure.io/fedora-kernel-dzickus.git rh_sync


As part of an effort to foster better cross collaboration with internal Red
Hat kernels, align the configs layout to match that kernel.  This will allow
Red Hat engineers to provide easier guidance on how to set various config
options.

In addition, the scripts that process the config options will migrate to the
configs/ directory too.  Future config workflows will stage all work in the
configs/ area.

A simple diff between the kernels will easily expose which config options
are different.  Reading the comments in the file provides guidance to Fedora
to determine if that kernel should make a similar change or not.  While the
RH kernel stays internal, requested changes will be posted publicly for
review with said reason.

Rename debugconfig -> configs/base-debug
Rename baseconfig -> configs/base-generic
Rename configs/base-generic/arm/arm64 -> configs/base-generic/arm/aarch64

You can browse the changes here:
https://pagure.io/fedora-kernel-dzickus/branch/rh_sync

Note: Laura asked me not post the patches as the diffstat for the 'git mv'
is obnoxiously large.  Instead I am providing the changes on pagure.

Cheers,
Don
___
kernel mailing list -- kernel@lists.fedoraproject.org
To unsubscribe send an email to kernel-le...@lists.fedoraproject.org

Re: Reviving the hardware census

[Don Zickus]

On Fri, Nov 10, 2017 at 01:17:50PM -0500, Nathaniel McCallum wrote:
> The more I look at lshw, the more I'm ambivalent (I'm not against it,
> just not for it either). It certainly collects a lot of relevant
> information. However, I see the following problems.
> 
> 1. lshw tries to make things human readable. This is bad for
> databases. We want to record things like immutable numeric hardware
> IDs rather than the current contents of pci.ids. lshw does provide
> -numeric, but this just adds the numbers to the human readable
> strings.
> 
> 2. lshw collects a lot of data we may not be interested in. For
> example, it reports the four different kinds of floppy disk drives my
> BIOS supports. Also, L1, L2 and L3 cache. There is a bunch of stuff
> like this. Is it useful? Some of it definitely is. But transferring
> unwanted data over the wire is not being a good netizen. Also, some
> people pay for data per MB transferred. We should be respectful.
> 
> 3. lshw supports '-sanitize', but this merely replaces the values with
> '[REMOVED]'. See above for why we don't want to transmit this data.
> 
> 4. lshw reports bus configuration. I'm not sure if this is relevant or
> how we would want to map this data. For example, if Dell uses the same
> hardware in a bunch of laptops then we can deduplicate this to one
> "hardware profile." But if bus configuration is part of this profile,
> then we will have much higher cardinality. If this information is
> important to have, we can make it work. But if not, it would be better
> to try to save storage.
> 
> 5. lshw doesn't seem to have a way to separate "system hardware" from
> "transient hardware." To be fair, this may be impossible. But it would
> be nice to understand the difference between, say, my bluetooth radio
> and my YubiKey. I can't easily remove the former but I can the latter.
> This also goes to cardinality reduction.
> 
> 6. lshw mixes things that are transient with things that are
> permanent. We can remove the timestamps with -notime. But, for
> example, it reports the kernel module currently assigned to a piece of
> hardware. This is probably relevant information, but we will have to
> carefully separate the data based on its lifespan.
> 
> 7. lshw reports virtual NICs as physical ones. We probably don't care
> about this and I certainly don't want to bloat the database with
> everyone's docker subnets unnecessarily.

Again, I was just pointing out a tool we use internally for inventorying our
hardware that I thought might be useful.  If it doesn't work for you, feel
free to choose something else. :-)  I do appreciate the feedback you
provided.  I might work with folks on my team to address some of them as
they could be of benefit for our work too.

Cheers,
Don


> 
> 
> On Fri, Nov 10, 2017 at 12:38 PM, Jeremy Cline  wrote:
> > On 11/09/2017 09:12 AM, Don Zickus wrote:
> >> On Wed, Nov 08, 2017 at 05:02:05PM -0500, Nathaniel McCallum wrote:
> >>> It isn't documented in F27, but it does work. However, we probably
> >>> want at least this patch:
> >>> https://github.com/lyonel/lshw/commit/135a853c60582b14c5b67e5cd988a8062d9896f4
> >>
> >> And some beaker stuff looks interesting in
> >>
> >> https://github.com/lyonel/lshw/commit/f95aa917a84a8ee74ce79e9b4f9e198d21a2e4d9
> >>
> >> Regardless.  My overall point was the lshw tool seems to embody a lot of
> >> what you were looking for and thought it could be useful (with some more
> >> fixes) instead of re-inventing the wheel with new plugins. :-)
> >>
> >> Up to you guys.
> >
> > I have no desire to re-invent wheels so I think it makes sense for us to
> > use lshw. Knowing it's being used internally is also good.
> >
> >
> > --
> > Jeremy Cline
> > XMPP: jer...@jcline.org
> > IRC:  jcline
> >
> ___
> kernel mailing list -- kernel@lists.fedoraproject.org
> To unsubscribe send an email to kernel-le...@lists.fedoraproject.org
___
kernel mailing list -- kernel@lists.fedoraproject.org
To unsubscribe send an email to kernel-le...@lists.fedoraproject.org

Re: Reviving the hardware census

[Nathaniel McCallum]

The more I look at lshw, the more I'm ambivalent (I'm not against it,
just not for it either). It certainly collects a lot of relevant
information. However, I see the following problems.

1. lshw tries to make things human readable. This is bad for
databases. We want to record things like immutable numeric hardware
IDs rather than the current contents of pci.ids. lshw does provide
-numeric, but this just adds the numbers to the human readable
strings.

2. lshw collects a lot of data we may not be interested in. For
example, it reports the four different kinds of floppy disk drives my
BIOS supports. Also, L1, L2 and L3 cache. There is a bunch of stuff
like this. Is it useful? Some of it definitely is. But transferring
unwanted data over the wire is not being a good netizen. Also, some
people pay for data per MB transferred. We should be respectful.

3. lshw supports '-sanitize', but this merely replaces the values with
'[REMOVED]'. See above for why we don't want to transmit this data.

4. lshw reports bus configuration. I'm not sure if this is relevant or
how we would want to map this data. For example, if Dell uses the same
hardware in a bunch of laptops then we can deduplicate this to one
"hardware profile." But if bus configuration is part of this profile,
then we will have much higher cardinality. If this information is
important to have, we can make it work. But if not, it would be better
to try to save storage.

5. lshw doesn't seem to have a way to separate "system hardware" from
"transient hardware." To be fair, this may be impossible. But it would
be nice to understand the difference between, say, my bluetooth radio
and my YubiKey. I can't easily remove the former but I can the latter.
This also goes to cardinality reduction.

6. lshw mixes things that are transient with things that are
permanent. We can remove the timestamps with -notime. But, for
example, it reports the kernel module currently assigned to a piece of
hardware. This is probably relevant information, but we will have to
carefully separate the data based on its lifespan.

7. lshw reports virtual NICs as physical ones. We probably don't care
about this and I certainly don't want to bloat the database with
everyone's docker subnets unnecessarily.


On Fri, Nov 10, 2017 at 12:38 PM, Jeremy Cline  wrote:
> On 11/09/2017 09:12 AM, Don Zickus wrote:
>> On Wed, Nov 08, 2017 at 05:02:05PM -0500, Nathaniel McCallum wrote:
>>> It isn't documented in F27, but it does work. However, we probably
>>> want at least this patch:
>>> https://github.com/lyonel/lshw/commit/135a853c60582b14c5b67e5cd988a8062d9896f4
>>
>> And some beaker stuff looks interesting in
>>
>> https://github.com/lyonel/lshw/commit/f95aa917a84a8ee74ce79e9b4f9e198d21a2e4d9
>>
>> Regardless.  My overall point was the lshw tool seems to embody a lot of
>> what you were looking for and thought it could be useful (with some more
>> fixes) instead of re-inventing the wheel with new plugins. :-)
>>
>> Up to you guys.
>
> I have no desire to re-invent wheels so I think it makes sense for us to
> use lshw. Knowing it's being used internally is also good.
>
>
> --
> Jeremy Cline
> XMPP: jer...@jcline.org
> IRC:  jcline
>
___
kernel mailing list -- kernel@lists.fedoraproject.org
To unsubscribe send an email to kernel-le...@lists.fedoraproject.org

Re: Reviving the hardware census

[Jeremy Cline]

On 11/09/2017 09:12 AM, Don Zickus wrote:
> On Wed, Nov 08, 2017 at 05:02:05PM -0500, Nathaniel McCallum wrote:
>> It isn't documented in F27, but it does work. However, we probably
>> want at least this patch:
>> https://github.com/lyonel/lshw/commit/135a853c60582b14c5b67e5cd988a8062d9896f4
> 
> And some beaker stuff looks interesting in
> 
> https://github.com/lyonel/lshw/commit/f95aa917a84a8ee74ce79e9b4f9e198d21a2e4d9
> 
> Regardless.  My overall point was the lshw tool seems to embody a lot of
> what you were looking for and thought it could be useful (with some more
> fixes) instead of re-inventing the wheel with new plugins. :-)
> 
> Up to you guys.

I have no desire to re-invent wheels so I think it makes sense for us to
use lshw. Knowing it's being used internally is also good.


-- 
Jeremy Cline
XMPP: jer...@jcline.org
IRC:  jcline

___
kernel mailing list -- kernel@lists.fedoraproject.org
To unsubscribe send an email to kernel-le...@lists.fedoraproject.org