Re: [leaf-devel] usefullness of kernel config cdiff files
14.03.2016 14:45, Erich Titl пишет:
> Am 14.03.2016 um 13:32 schrieb Andrew:
>> 14.03.2016 12:28, Erich Titl пишет:
>>> Hi Andre
>>>
>>> Am 12.03.2016 um 19:40 schrieb Andrew:
Hi.
>>> ...>>
> Please let us know how you do a major kernel upgrade. If you write it
> down, you might see that there are unnecessaty steps or you may
> convince me that the diff files are valuable.
>
1. Upgrade kernel version, copy configs,
>>> Will you have to generate the config files here using the diff files?
>> No, Just copy/rename config + cdiffs in repo in this step
> Well, in the repository there are no config files, just the default and
> cdiffs.
Right, configs are generated by 'buildtool.pl source linux' and then
'make build' into the dir
>
>>> generate default (=i486) config
>>>
>>> How do you generate this one? According to one of your last mails it
>>> should already exist as Bering_KRELEASE_.config.
by generic procedure ('make oldconfig' is called), then - break process
on next arch.
>>> Note somewhere drivers (= kernel options) that should be
enabled in some specific arches (for ex., PCI-E cards drivers) if they
exists; usually - max 3-5 devices, or even none
>>> This is the manual intervention
>> Yes, of course. In other case we'll receive a lot of trash like
>> proximity/light sensors and a lot of missed useful drivers.
> OK, so you manually select/deselect new drivers in the default config
> only, and the hope is that the diff files can take care of that. My only
> concern here is, that the diff files do not appear to play nicely with
> git rebase/merge, as those functions apply diffs to diffs.
>
> ..>
cdiff files changes specified in them options.
If there is a trouble with diff syntax - we may use different syntax for
them (for ex., "-"/"+" replace to "<"/">"
>>> If we really could just work on the default config and the diff
>>> mechanism would allow us to upgrade all existing archs automatically
>>> then I would be all for it. But to me right now, this assumption appears
>>> to be wrong.
>> You may change 'make oldconfig' into 'make olddefconfig' in makefile,
>> and got same useless semi-generic configs for all archs/subarchs like in
>> your example.
> True, it applies the default values to all new options, it does not need
> manual intervention though. If we really can use cdiffs _without_
> _touching_ them _after_ an upgrade, that would be beneficial.
Yes, cdiff just changes options that are specified into it.
>
> cheers
>
> ET
>
> --
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785231=/4140
>
> ___
> leaf-devel mailing list
> leaf-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/leaf-devel
--
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231=/4140
___
leaf-devel mailing list
leaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] usefullness of kernel config cdiff files
Am 14.03.2016 um 13:32 schrieb Andrew:
> 14.03.2016 12:28, Erich Titl пишет:
>> Hi Andre
>>
>> Am 12.03.2016 um 19:40 schrieb Andrew:
>>> Hi.
>> ...>>
Please let us know how you do a major kernel upgrade. If you write it
down, you might see that there are unnecessaty steps or you may
convince me that the diff files are valuable.
>>> 1. Upgrade kernel version, copy configs,
>> Will you have to generate the config files here using the diff files?
> No, Just copy/rename config + cdiffs in repo in this step
Well, in the repository there are no config files, just the default and
cdiffs.
>>
>> generate default (=i486) config
>>
>> How do you generate this one? According to one of your last mails it
>> should already exist as Bering_KRELEASE_.config.
>
>>> by generic procedure ('make oldconfig' is called), then - break process
>>> on next arch.
>> Note somewhere drivers (= kernel options) that should be
>>> enabled in some specific arches (for ex., PCI-E cards drivers) if they
>>> exists; usually - max 3-5 devices, or even none
>> This is the manual intervention
> Yes, of course. In other case we'll receive a lot of trash like
> proximity/light sensors and a lot of missed useful drivers.
OK, so you manually select/deselect new drivers in the default config
only, and the hope is that the diff files can take care of that. My only
concern here is, that the diff files do not appear to play nicely with
git rebase/merge, as those functions apply diffs to diffs.
..>
>> If we really could just work on the default config and the diff
>> mechanism would allow us to upgrade all existing archs automatically
>> then I would be all for it. But to me right now, this assumption appears
>> to be wrong.
> You may change 'make oldconfig' into 'make olddefconfig' in makefile,
> and got same useless semi-generic configs for all archs/subarchs like in
> your example.
True, it applies the default values to all new options, it does not need
manual intervention though. If we really can use cdiffs _without_
_touching_ them _after_ an upgrade, that would be beneficial.
cheers
ET
--
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231=/4140
___
leaf-devel mailing list
leaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] usefullness of kernel config cdiff files
Hi Andre
Am 12.03.2016 um 19:40 schrieb Andrew:
> Hi.
...>>
>> Please let us know how you do a major kernel upgrade. If you write it
>> down, you might see that there are unnecessaty steps or you may
>> convince me that the diff files are valuable.
>>
> 1. Upgrade kernel version, copy configs,
Will you have to generate the config files here using the diff files?
generate default (=i486) config
How do you generate this one? According to one of your last mails it
should already exist as Bering_KRELEASE_.config.
> by generic procedure ('make oldconfig' is called), then - break process
> on next arch.
Note somewhere drivers (= kernel options) that should be
> enabled in some specific arches (for ex., PCI-E cards drivers) if they
> exists; usually - max 3-5 devices, or even none
This is the manual intervention
> 2. Copy default config to repo
> 3. Clean generated configs, run 'make build' again
> 4. If I have some specific new drivers that must be enabled on some arch
> - then I run 'make menuconfig' or edit config file manually (with 'make
> oldconfig' to ensure that no additional options hapened), and then -
> re-create cdiff.
Another manual intervention on all of the arch files which will need to
be generated from the diff files.
> 5. Try to make kernel configs for other archs (to ensure that no
> arch-specific options added), if needed - enable some drivers manually,
> and if config is changed - re-create cdiff
You see here, you generate configs and recreate diffs, I don't think
this step is necessary, but in the end the results should be the same,
IHMO just too much work and could be avoidedd.
If we really could just work on the default config and the diff
mechanism would allow us to upgrade all existing archs automatically
then I would be all for it. But to me right now, this assumption appears
to be wrong.
>
> I receive kernel config that 1) have predictible set of changes and 2)
> arch differences are only in drivers/platform options, networking
> options are same
>
> As you see, it isn't too hard.
I never said it was hard, just some steps might not be necessary
cheers
ET
--
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231=/4140
___
leaf-devel mailing list
leaf-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] usefullness of kernel config cdiff files
Hi.
12.03.2016 13:02, Erich Titl пишет:
> Hi Andrew
>
> Am 11.03.2016 um 18:19 schrieb Andrew:
>> 11.03.2016 16:24, Erich Titl пишет:
>>> Hi Andrew
>>>
>>> Am 11.03.2016 um 12:50 schrieb Andrew:
11.03.2016 08:48, Erich Titl пишет:
> Hi Andrew
>
> Am 10.03.2016 um 21:19 schrieb Andrew:
>> 10.03.2016 22:07, Erich Titl пишет:
>>> Am 10.03.2016 um 19:39 schrieb Andrew:
10.03.2016 20:03, Erich Titl пишет:
> ...
>
> But in the end you have to do it for every arch, not only
> common config.
> Humans are notoriously weak when looking at changed/missed lines.
Not always.
> Always, humans are just not good at that.
If no options added into new kernel patch (this is 99% cases) - you
don't need to touch kernel configs at all.
>>> Good, but how do you handle the new options. just leaving them away and
>>> sticking with an old config does not really cut it for me.
>> If new option is added - kernel building stucks on 'make defconfig'.
>
> Why do you always refer to defconfig? I did not mention it. Look at
> olddefconfig.
>
Sorry, I mean 'make oldconfig' here.
>
>>
When there is usual kernel config - at kernel update there's
usual tens
or even hundreds new options that should be choosed. And using
.cdiff
with generic kernel you should only enable new platform-specific
drivers; generic options like network stack are applied
automatically.
>>> The same applies if you use make olddefconfig.
>> And you'll lack, for ex., new iptables features from new kernel,
>> and got
>> a lot of unneeded drivers.
> Well, I am not that sure we are using the best of all
> configurations and
> typically the defaults are quite OK.
What you mean as 'best configuration'? Default configuration that
have a
lot of stuff disabled (for ex., network drivers, a lot of iptables
modules and so on) and a log of useless stuff enabled (for ex., video
drivers are completely useless for us - we don't use graphic modes)
isn't a best configuration for our case.
>>> Without knowing what new modules are provided I am not sure. Someone
>>> may
>>> use video drivers for whatnot or audio drivers for hardware I don't
>>> have. We are providing a lot of packages which I have no clue what they
>>> could be useful for.
>> Why you think that new modules are unknown? Use 'make defconfig' - and
>> you'll always know what is added to kernel, and what of them you
>> enabled.
>
> Well defconfig will use the defaults and you might not want them.
> Whatever. it is not merely to discuss the way to do kernel upgrade but
> the usefulness of the diff files. I personally doubt they are any
> good, but if someone thinks they are, well I could not care less. I
> can live with the current set-up, I don't fancy it but that is my
> opinion.
>
Sorry, I again missed... I mean 'make oldconfig' here too.
And diff files are mostly used for easing kernel upgrade process and
make it's result more predictable. There's no difference how configs are
stored till we want to change config parameters/upgrade kernel.
>
>
> And you still have
> to generate the full config for every arch. But then time is
> not the
> only parameter to look at. We should analyze if there were non
> necessary
> steps involved in this process.
In any case, you should look at configs difference - so
generating some
kind of diffs is necessary.
>>> Why? The kernel developers gave us pretty good tools to manage
>>> kernel
>>> upgrades and they did _not_ provide diff files. I believed in
>>> the past
>>> this was a wekness, now I am convinced they knew why.
>> To be sure that you didn't missed something.
> You typically miss stuff when you do it manually
Ok, so you prefer disable all potentially useful stuff (like network
drivers, some iptables modules) because these drivers disabled in
default kernel config? Or in what way we may be sure that, for ex.,
iptables modules set is identical across all archs?
>>> If you want to assure this, then indeed you may need a diff file, but
>>> just to compare two full config files, not to generate one from the
>>> other.
>> IMHO update in this case will be harder + there is more chances to make
>> error in configs.
>
> Please let us know how you do a major kernel upgrade. If you write it
> down, you might see that there are unnecessaty steps or you may
> convince me that the diff files are valuable.
>
1. Upgrade kernel version, copy configs, generate default (=i486) config
by generic procedure ('make oldconfig' is called), then - break process
on next arch. Note somewhere drivers (= kernel options) that should be
enabled in some specific arches (for ex., PCI-E cards drivers) if they
exists; usually - max 3-5 devices, or even none
Re: [leaf-devel] usefullness of kernel config cdiff files
Hi Andrew Am 11.03.2016 um 18:19 schrieb Andrew: 11.03.2016 16:24, Erich Titl пишет: Hi Andrew Am 11.03.2016 um 12:50 schrieb Andrew: 11.03.2016 08:48, Erich Titl пишет: Hi Andrew Am 10.03.2016 um 21:19 schrieb Andrew: 10.03.2016 22:07, Erich Titl пишет: Am 10.03.2016 um 19:39 schrieb Andrew: 10.03.2016 20:03, Erich Titl пишет: ... But in the end you have to do it for every arch, not only common config. Humans are notoriously weak when looking at changed/missed lines. Not always. Always, humans are just not good at that. If no options added into new kernel patch (this is 99% cases) - you don't need to touch kernel configs at all. Good, but how do you handle the new options. just leaving them away and sticking with an old config does not really cut it for me. If new option is added - kernel building stucks on 'make defconfig'. Why do you always refer to defconfig? I did not mention it. Look at olddefconfig. When there is usual kernel config - at kernel update there's usual tens or even hundreds new options that should be choosed. And using .cdiff with generic kernel you should only enable new platform-specific drivers; generic options like network stack are applied automatically. The same applies if you use make olddefconfig. And you'll lack, for ex., new iptables features from new kernel, and got a lot of unneeded drivers. Well, I am not that sure we are using the best of all configurations and typically the defaults are quite OK. What you mean as 'best configuration'? Default configuration that have a lot of stuff disabled (for ex., network drivers, a lot of iptables modules and so on) and a log of useless stuff enabled (for ex., video drivers are completely useless for us - we don't use graphic modes) isn't a best configuration for our case. Without knowing what new modules are provided I am not sure. Someone may use video drivers for whatnot or audio drivers for hardware I don't have. We are providing a lot of packages which I have no clue what they could be useful for. Why you think that new modules are unknown? Use 'make defconfig' - and you'll always know what is added to kernel, and what of them you enabled. Well defconfig will use the defaults and you might not want them. Whatever. it is not merely to discuss the way to do kernel upgrade but the usefulness of the diff files. I personally doubt they are any good, but if someone thinks they are, well I could not care less. I can live with the current set-up, I don't fancy it but that is my opinion. And you still have to generate the full config for every arch. But then time is not the only parameter to look at. We should analyze if there were non necessary steps involved in this process. In any case, you should look at configs difference - so generating some kind of diffs is necessary. Why? The kernel developers gave us pretty good tools to manage kernel upgrades and they did _not_ provide diff files. I believed in the past this was a wekness, now I am convinced they knew why. To be sure that you didn't missed something. You typically miss stuff when you do it manually Ok, so you prefer disable all potentially useful stuff (like network drivers, some iptables modules) because these drivers disabled in default kernel config? Or in what way we may be sure that, for ex., iptables modules set is identical across all archs? If you want to assure this, then indeed you may need a diff file, but just to compare two full config files, not to generate one from the other. IMHO update in this case will be harder + there is more chances to make error in configs. Please let us know how you do a major kernel upgrade. If you write it down, you might see that there are unnecessaty steps or you may convince me that the diff files are valuable. cheers ET smime.p7s Description: S/MIME Cryptographic Signature -- Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] usefullness of kernel config cdiff files
11.03.2016 16:24, Erich Titl пишет: > Hi Andrew > > Am 11.03.2016 um 12:50 schrieb Andrew: >> 11.03.2016 08:48, Erich Titl пишет: >>> Hi Andrew >>> >>> Am 10.03.2016 um 21:19 schrieb Andrew: 10.03.2016 22:07, Erich Titl пишет: > Am 10.03.2016 um 19:39 schrieb Andrew: >> 10.03.2016 20:03, Erich Titl пишет: >>> ... >>> >>> But in the end you have to do it for every arch, not only common config. >>> Humans are notoriously weak when looking at changed/missed lines. >> Not always. >>> Always, humans are just not good at that. >> If no options added into new kernel patch (this is 99% cases) - you >> don't need to touch kernel configs at all. > Good, but how do you handle the new options. just leaving them away and > sticking with an old config does not really cut it for me. If new option is added - kernel building stucks on 'make defconfig'. >> When there is usual kernel config - at kernel update there's usual tens >> or even hundreds new options that should be choosed. And using .cdiff >> with generic kernel you should only enable new platform-specific >> drivers; generic options like network stack are applied automatically. > The same applies if you use make olddefconfig. And you'll lack, for ex., new iptables features from new kernel, and got a lot of unneeded drivers. >>> Well, I am not that sure we are using the best of all configurations and >>> typically the defaults are quite OK. >> What you mean as 'best configuration'? Default configuration that have a >> lot of stuff disabled (for ex., network drivers, a lot of iptables >> modules and so on) and a log of useless stuff enabled (for ex., video >> drivers are completely useless for us - we don't use graphic modes) >> isn't a best configuration for our case. > Without knowing what new modules are provided I am not sure. Someone may > use video drivers for whatnot or audio drivers for hardware I don't > have. We are providing a lot of packages which I have no clue what they > could be useful for. Why you think that new modules are unknown? Use 'make defconfig' - and you'll always know what is added to kernel, and what of them you enabled. >>> And you still have >>> to generate the full config for every arch. But then time is not the >>> only parameter to look at. We should analyze if there were non necessary >>> steps involved in this process. >> In any case, you should look at configs difference - so generating some >> kind of diffs is necessary. > Why? The kernel developers gave us pretty good tools to manage kernel > upgrades and they did _not_ provide diff files. I believed in the past > this was a wekness, now I am convinced they knew why. To be sure that you didn't missed something. >>> You typically miss stuff when you do it manually >> Ok, so you prefer disable all potentially useful stuff (like network >> drivers, some iptables modules) because these drivers disabled in >> default kernel config? Or in what way we may be sure that, for ex., >> iptables modules set is identical across all archs? > If you want to assure this, then indeed you may need a diff file, but > just to compare two full config files, not to generate one from the other. IMHO update in this case will be harder + there is more chances to make error in configs. > And then AFAIK olddefconfig just sets the default values to _new_ > parameters, and I agree they may be wrong. Speaking for me, I often > don't know what those new parameters are and I believe it is the same > with most others, so the default appears logical. Any kernel parameter has built-in help. > AFAIK olddefconfig does _not_ use defaults for options already set, just > for new options, so we get a new clean config file without having to > manually accept stuff. And with a lot of disabled useful drivers or kernel modules, + a lot of enabled useless... > >> >>> I looked at my routine to get master in sync with new-initrd. >> It seems like you re-generated kernel config from scratch. With a lot >> of completely unneeded stuff. >>> Nope, I did not. I applied olddefconfig >> So something went completely wrong in your case. > Not necessarily, just rebasing yielded a lot of differences for the diff > files. Maybe my originals already had a number of addidional stuff which > either existed in master at the time I applied it or were somehow added > later. So there werde quite some differences in the diff files and > rebase did not like it. I was not surprised by this as I did not even > want to imagine how a diff of a diff could ever be cleanly resolved. > > .. We may change cdiff syntax (for ex., replace '+' and '-' to '<' and '>'). > >> As I said, generic config isn't a best choice. For ex, you don't need a >> SMP on i486/Geode. You don't need a lot of RAID drivers for Geode/i486 >> (physically they are missed on these platforms). You don't need SATA >> drivers here. And of course you don't need PCI-E cards
Re: [leaf-devel] usefullness of kernel config cdiff files
Hi Andrew Am 11.03.2016 um 12:50 schrieb Andrew: > 11.03.2016 08:48, Erich Titl пишет: >> Hi Andrew >> >> Am 10.03.2016 um 21:19 schrieb Andrew: >>> 10.03.2016 22:07, Erich Titl пишет: Am 10.03.2016 um 19:39 schrieb Andrew: > 10.03.2016 20:03, Erich Titl пишет: >> ... >> >> But in the end you have to do it for every arch, not only common config. >> Humans are notoriously weak when looking at changed/missed lines. > Not always. >> Always, humans are just not good at that. > If no options added into new kernel patch (this is 99% cases) - you > don't need to touch kernel configs at all. Good, but how do you handle the new options. just leaving them away and sticking with an old config does not really cut it for me. > >> > When there is usual kernel config - at kernel update there's usual tens > or even hundreds new options that should be choosed. And using .cdiff > with generic kernel you should only enable new platform-specific > drivers; generic options like network stack are applied automatically. The same applies if you use make olddefconfig. >>> And you'll lack, for ex., new iptables features from new kernel, and got >>> a lot of unneeded drivers. >> Well, I am not that sure we are using the best of all configurations and >> typically the defaults are quite OK. > What you mean as 'best configuration'? Default configuration that have a > lot of stuff disabled (for ex., network drivers, a lot of iptables > modules and so on) and a log of useless stuff enabled (for ex., video > drivers are completely useless for us - we don't use graphic modes) > isn't a best configuration for our case. Without knowing what new modules are provided I am not sure. Someone may use video drivers for whatnot or audio drivers for hardware I don't have. We are providing a lot of packages which I have no clue what they could be useful for. > ... > >> And you still have >> to generate the full config for every arch. But then time is not the >> only parameter to look at. We should analyze if there were non necessary >> steps involved in this process. > In any case, you should look at configs difference - so generating some > kind of diffs is necessary. Why? The kernel developers gave us pretty good tools to manage kernel upgrades and they did _not_ provide diff files. I believed in the past this was a wekness, now I am convinced they knew why. >>> To be sure that you didn't missed something. >> You typically miss stuff when you do it manually > Ok, so you prefer disable all potentially useful stuff (like network > drivers, some iptables modules) because these drivers disabled in > default kernel config? Or in what way we may be sure that, for ex., > iptables modules set is identical across all archs? If you want to assure this, then indeed you may need a diff file, but just to compare two full config files, not to generate one from the other. And then AFAIK olddefconfig just sets the default values to _new_ parameters, and I agree they may be wrong. Speaking for me, I often don't know what those new parameters are and I believe it is the same with most others, so the default appears logical. AFAIK olddefconfig does _not_ use defaults for options already set, just for new options, so we get a new clean config file without having to manually accept stuff. > > >> I looked at my routine to get master in sync with new-initrd. > It seems like you re-generated kernel config from scratch. With a lot > of completely unneeded stuff. >> Nope, I did not. I applied olddefconfig > So something went completely wrong in your case. Not necessarily, just rebasing yielded a lot of differences for the diff files. Maybe my originals already had a number of addidional stuff which either existed in master at the time I applied it or were somehow added later. So there werde quite some differences in the diff files and rebase did not like it. I was not surprised by this as I did not even want to imagine how a diff of a diff could ever be cleanly resolved. .. > > As I said, generic config isn't a best choice. For ex, you don't need a > SMP on i486/Geode. You don't need a lot of RAID drivers for Geode/i486 > (physically they are missed on these platforms). You don't need SATA > drivers here. And of course you don't need PCI-E cards drivers for that > platforms. Right, and it might be nice to have such a config. We also want only storage drivers compilled intt the kernel. With the advent of modules.sqfs I don't care that much about modules anymore. If there are a few too many, so be it if it makes somebody happy. > > From other side, for i686/x86_64 you need all PCI-E network cards > drivers (a lot of them are disabled - for ex., Mellanox, Emulex 10GbE > and so on). Yes, and if someone needs this driver we can always add it to modules. But we will never be able to define the correct superset of drivers needed. > > And you also
Re: [leaf-devel] usefullness of kernel config cdiff files
11.03.2016 08:48, Erich Titl пишет: > Hi Andrew > > Am 10.03.2016 um 21:19 schrieb Andrew: >> 10.03.2016 22:07, Erich Titl пишет: >>> Am 10.03.2016 um 19:39 schrieb Andrew: 10.03.2016 20:03, Erich Titl пишет: > ... > > But in the end you have to do it for every arch, not only common config. > Humans are notoriously weak when looking at changed/missed lines. Not always. > Always, humans are just not good at that. If no options added into new kernel patch (this is 99% cases) - you don't need to touch kernel configs at all. > When there is usual kernel config - at kernel update there's usual tens or even hundreds new options that should be choosed. And using .cdiff with generic kernel you should only enable new platform-specific drivers; generic options like network stack are applied automatically. >>> The same applies if you use make olddefconfig. >> And you'll lack, for ex., new iptables features from new kernel, and got >> a lot of unneeded drivers. > Well, I am not that sure we are using the best of all configurations and > typically the defaults are quite OK. What you mean as 'best configuration'? Default configuration that have a lot of stuff disabled (for ex., network drivers, a lot of iptables modules and so on) and a log of useless stuff enabled (for ex., video drivers are completely useless for us - we don't use graphic modes) isn't a best configuration for our case. > release. >>> Potentially with each new kernel release and at out speed that is every >>> two weeks. >> There's mostly > Mostly is not always :-) > > no config difference between kernel patch releases. It >> just requires to replace kernel patch... > And then it is easy to just use oldconfig. There's no need to touch kernel configs at all. > And you still have > to generate the full config for every arch. But then time is not the > only parameter to look at. We should analyze if there were non necessary > steps involved in this process. In any case, you should look at configs difference - so generating some kind of diffs is necessary. >>> Why? The kernel developers gave us pretty good tools to manage kernel >>> upgrades and they did _not_ provide diff files. I believed in the past >>> this was a wekness, now I am convinced they knew why. >> To be sure that you didn't missed something. > You typically miss stuff when you do it manually Ok, so you prefer disable all potentially useful stuff (like network drivers, some iptables modules) because these drivers disabled in default kernel config? Or in what way we may be sure that, for ex., iptables modules set is identical across all archs? > I looked at my routine to get master in sync with new-initrd. It seems like you re-generated kernel config from scratch. With a lot of completely unneeded stuff. > Nope, I did not. I applied olddefconfig So something went completely wrong in your case. > and yes, if you are sure that > the suggested defaults from the kernel people are wrong then I had more > stuff compiled in than necessary. But if you don't trust those folks > then you better write your own kernel :-) As I said, generic config isn't a best choice. For ex, you don't need a SMP on i486/Geode. You don't need a lot of RAID drivers for Geode/i486 (physically they are missed on these platforms). You don't need SATA drivers here. And of course you don't need PCI-E cards drivers for that platforms. From other side, for i686/x86_64 you need all PCI-E network cards drivers (a lot of them are disabled - for ex., Mellanox, Emulex 10GbE and so on). And you also don't need drivers for power controllers, proximity/light sensors and a lot of other stuff targetted for mobile devices/notebooks. That's why defconfig is a bad idea. > I just applied your changes in your old branch to new kernel configs. They lays perfectly. >>> But you need the full config files to do this and then you just convert >>> them back to diffs :-( >> No. > So you just apply the diffs to a completely new kernel config? Yes. Because diffs are trivial (storage drivers are compiled as built-in instead of modules), + minor manual edit of .cdiff (for one new SATA driver that was added in 4.4 kernel) > IMHO this > is utterly dangerous. And yes, then you need to inspect the files and > you will miss stuff and after some time it will be detected and fixed. > > Still don't believe it. > > cheers > > ET > > > -- > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140 > > ___ > leaf-devel mailing list > leaf-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] usefullness of kernel config cdiff files
Hi Andrew Am 10.03.2016 um 21:19 schrieb Andrew: > 10.03.2016 22:07, Erich Titl пишет: >> Am 10.03.2016 um 19:39 schrieb Andrew: >>> 10.03.2016 20:03, Erich Titl пишет: ... But in the end you have to do it for every arch, not only common config. Humans are notoriously weak when looking at changed/missed lines. >>> Not always. Always, humans are just not good at that. >>> >>> When there is usual kernel config - at kernel update there's usual tens >>> or even hundreds new options that should be choosed. And using .cdiff >>> with generic kernel you should only enable new platform-specific >>> drivers; generic options like network stack are applied automatically. >> The same applies if you use make olddefconfig. > And you'll lack, for ex., new iptables features from new kernel, and got > a lot of unneeded drivers. Well, I am not that sure we are using the best of all configurations and typically the defaults are quite OK. > >>> release. >> Potentially with each new kernel release and at out speed that is every >> two weeks. > There's mostly Mostly is not always :-) no config difference between kernel patch releases. It > just requires to replace kernel patch... And then it is easy to just use oldconfig. And you still have to generate the full config for every arch. But then time is not the only parameter to look at. We should analyze if there were non necessary steps involved in this process. >>> In any case, you should look at configs difference - so generating some >>> kind of diffs is necessary. >> Why? The kernel developers gave us pretty good tools to manage kernel >> upgrades and they did _not_ provide diff files. I believed in the past >> this was a wekness, now I am convinced they knew why. > To be sure that you didn't missed something. You typically miss stuff when you do it manually > I looked at my routine to get master in sync with new-initrd. >>> It seems like you re-generated kernel config from scratch. With a lot >>> of completely unneeded stuff. Nope, I did not. I applied olddefconfig and yes, if you are sure that the suggested defaults from the kernel people are wrong then I had more stuff compiled in than necessary. But if you don't trust those folks then you better write your own kernel :-) >>> >>> I just applied your changes in your old branch to new kernel configs. >>> They lays perfectly. >> But you need the full config files to do this and then you just convert >> them back to diffs :-( > No. So you just apply the diffs to a completely new kernel config? IMHO this is utterly dangerous. And yes, then you need to inspect the files and you will miss stuff and after some time it will be detected and fixed. Still don't believe it. cheers ET -- Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] usefullness of kernel config cdiff files
10.03.2016 22:07, Erich Titl пишет: > Am 10.03.2016 um 19:39 schrieb Andrew: >> 10.03.2016 20:03, Erich Titl пишет: >>> Hi Andrew >>> >>> Am 10.03.2016 um 18:53 schrieb Andrew: Hi. For common config case you can easily check what real differences are between configs. + kernel upgrade is enough easy: just update common config (be make oldconfig) and try to generate other configs (looking on cdiff output - for changed/missed lines); and then look on result of 'make oldconfig' + maybe correct some subarch-specific settings (for ex., enable new drivers for i686/x86_64). >>> But in the end you have to do it for every arch, not only common config. >>> Humans are notoriously weak when looking at changed/missed lines. >> Not always. >> >> When there is usual kernel config - at kernel update there's usual tens >> or even hundreds new options that should be choosed. And using .cdiff >> with generic kernel you should only enable new platform-specific >> drivers; generic options like network stack are applied automatically. > The same applies if you use make olddefconfig. And you'll lack, for ex., new iptables features from new kernel, and got a lot of unneeded drivers. > For me, migration to new kernel (when I experimented with different versions due to crashes with PPPoE on 4.1) takes max 10-15 minutes - even when I switched from 4.1 to old 3.2. >>> Which I consider very long for just a kernel upgrade. >> Not too long - it's done just once (ok, maybe - twice as in 5.2 case, >> when we switched from 5.1's 3.10 to 3.14 and then to 4.1) per minor >> release. > Potentially with each new kernel release and at out speed that is every > two weeks. There's mostly no config difference between kernel patch releases. It just requires to replace kernel patch... >>> And you still have >>> to generate the full config for every arch. But then time is not the >>> only parameter to look at. We should analyze if there were non necessary >>> steps involved in this process. >> In any case, you should look at configs difference - so generating some >> kind of diffs is necessary. > Why? The kernel developers gave us pretty good tools to manage kernel > upgrades and they did _not_ provide diff files. I believed in the past > this was a wekness, now I am convinced they knew why. To be sure that you didn't missed something. >>> I looked at my routine to get master in sync with new-initrd. >> It seems like you re-generated kernel config from scratch. With a lot >> of completely unneeded stuff. >> >> I just applied your changes in your old branch to new kernel configs. >> They lays perfectly. > But you need the full config files to do this and then you just convert > them back to diffs :-( No. > Whatever, I _believe_ _now_ that keeping the full config files would > make transition easier, mostly because of the format of the config > files, which _mark_ the absence of an option instead of just leaving it > out. If the config format would hold less of this non_information it > would be better suited to automatic upgrades. > > cheers > > ET > > > > -- > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140 > > ___ > leaf-devel mailing list > leaf-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/leaf-devel -- Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] usefullness of kernel config cdiff files
Am 10.03.2016 um 19:39 schrieb Andrew: > 10.03.2016 20:03, Erich Titl пишет: >> Hi Andrew >> >> Am 10.03.2016 um 18:53 schrieb Andrew: >>> Hi. >>> >>> For common config case you can easily check what real differences are >>> between configs. >>> >>> + kernel upgrade is enough easy: just update common config (be make >>> oldconfig) and try to generate other configs (looking on cdiff output - >>> for changed/missed lines); and then look on result of 'make oldconfig' + >>> maybe correct some subarch-specific settings (for ex., enable new >>> drivers for i686/x86_64). >> But in the end you have to do it for every arch, not only common config. >> Humans are notoriously weak when looking at changed/missed lines. > Not always. > > When there is usual kernel config - at kernel update there's usual tens > or even hundreds new options that should be choosed. And using .cdiff > with generic kernel you should only enable new platform-specific > drivers; generic options like network stack are applied automatically. The same applies if you use make olddefconfig. > >> >>> For me, migration to new kernel (when I experimented with different >>> versions due to crashes with PPPoE on 4.1) takes max 10-15 minutes - >>> even when I switched from 4.1 to old 3.2. >> Which I consider very long for just a kernel upgrade. > Not too long - it's done just once (ok, maybe - twice as in 5.2 case, > when we switched from 5.1's 3.10 to 3.14 and then to 4.1) per minor > release. Potentially with each new kernel release and at out speed that is every two weeks. > >> And you still have >> to generate the full config for every arch. But then time is not the >> only parameter to look at. We should analyze if there were non necessary >> steps involved in this process. > In any case, you should look at configs difference - so generating some > kind of diffs is necessary. Why? The kernel developers gave us pretty good tools to manage kernel upgrades and they did _not_ provide diff files. I believed in the past this was a wekness, now I am convinced they knew why. > >> >> I looked at my routine to get master in sync with new-initrd. > It seems like you re-generated kernel config from scratch. With a lot > of completely unneeded stuff. > > I just applied your changes in your old branch to new kernel configs. > They lays perfectly. But you need the full config files to do this and then you just convert them back to diffs :-( Whatever, I _believe_ _now_ that keeping the full config files would make transition easier, mostly because of the format of the config files, which _mark_ the absence of an option instead of just leaving it out. If the config format would hold less of this non_information it would be better suited to automatic upgrades. cheers ET -- Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
Re: [leaf-devel] usefullness of kernel config cdiff files
Hi. For common config case you can easily check what real differences are between configs. + kernel upgrade is enough easy: just update common config (be make oldconfig) and try to generate other configs (looking on cdiff output - for changed/missed lines); and then look on result of 'make oldconfig' + maybe correct some subarch-specific settings (for ex., enable new drivers for i686/x86_64). For me, migration to new kernel (when I experimented with different versions due to crashes with PPPoE on 4.1) takes max 10-15 minutes - even when I switched from 4.1 to old 3.2. 10.03.2016 12:11, Erich Titl пишет: > Hi Folks > > Some time ago I suggested to not keep complete kernel config files, but > base them on a common basic configuration and diff files, so common > settings would be kept in a single file only. This has been implemented > in the config.cdiff files. I was falsely convinced, that common config > settings could be handled easier that way. I _believe_ now I was wrong. > > Working extensively on a few config settings lately I was forced all the > time to make a detour by generating full kernel config files from the > cdiff files just to upgrade to new kernel releases and modify small > settings. This appears orthogonal to the original idea of standardizing > kernel configs. It also appears that git does not handle the cdiff files > nicely and therefore it appears not to be obvious to merge or rebase the > branches. > > It also appears that upgrading kernel releases always requires to > generate a full config to be copied to the new kernel and running make > oldconfig or olddefconfig, just to be forced to generate a new diff file > afterwards. So this also defeats to a certain degree the usefulness of > the kernel diff files. > > I would like to discuss here if it would not be better to drop this > feature as at least for me it does not show any potential to make the > administration of the kernel config files any easier. I suggest to move > back to keep complete kernel config files. > > cheers > > ET > > > -- > Transform Data into Opportunity. > Accelerate data analysis in your applications with > Intel Data Analytics Acceleration Library. > Click to learn more. > http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140 > > ___ > leaf-devel mailing list > leaf-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/leaf-devel -- Transform Data into Opportunity. Accelerate data analysis in your applications with Intel Data Analytics Acceleration Library. Click to learn more. http://pubads.g.doubleclick.net/gampad/clk?id=278785111=/4140 ___ leaf-devel mailing list leaf-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/leaf-devel
