Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Lamar Owen]

On 04/24/2017 11:52 AM, Warren Young wrote:

On Apr 24, 2017, at 7:53 AM, Lamar Owen  wrote:

James' point isn't the hardware cost, it's the people cost for retraining.

Unless you’ve hired monkeys so that you must train them to do their tasks by 
rote, that is a soft cost, not a hard cost.


Dollars are dollars.  An hour spent in training as one hour less to 'do 
work.'  (I'm intentionally playing devil's advocate here; I personally 
don't have a problem with the changes other than I now have to remember 
to check the OS type and version every time I log in to a server prior 
to issuing commands).

Note also that Byrne’s solution was to move to an entirely different OS, but we 
don’t hear about the “retraining cost” involved with that.  Surely it was a 
larger jump from C6 or C7 to FreeBSD 10 than from C6 to C7?
Guaranteed that it was a much larger jump.  Although I am tangentially 
reminded of Apollo Domain/OS 10 where the SysV/BSD/Aegis behavior was 
settable by changing an environment variable.



It’ll be interesting to see how much change FreeBSD gets in the next 7 years.


What is interesting to me, having just worked on a 20-year-old server 
stack last week, is how much hasn't changed as well as how much of what 
gets used a lot has changed (remember life before yum? How about early 
yum that needed to download individual headers?). But 90% of what I 
learned 30 years ago on Xenix System 3 for the Tandy 6000 still works 
(mainly because I still use vi :-)  ).


That depends on the organization and its goals. 


Very much true.  My IT department that I run has a bit of a reputation; 
our 'stock' answer to any IT question is rumored to be 'it depends.'  
YMMV, etc.



...dual-socket Opteron LS20 blades (10+ years old)...CentOS 7, once installed, 
works great...

That doesn’t really contradict my point.

First, I said “most” hardware, but you’ve gone and cherry-picked uncommonly 
durable hardware here; you’re probably out in +3 sigma territory.


Hey, I just picked what I have here, that's all.  I could also talk 
about our 2007, 2009, and 2010-vintage donated EMC Clariion hardware.  
We have gotten many Dell PowerEdge servers and Optiplex/Precision 
desktops donated to us; got 19 Dell PE1950's donated in a lot three 
years ago, and those are some of our best servers.  The last servers we 
actually bought were a pair of Dell PE6950's in 2007; a grant funded two 
of them plus VMware VI3 and a couple of EMC Clariion CX3-10c SANs.  (All 
of those are still running and still doing their jobs.)


I'd rather have a five-year-old Precision than a 2017-model generic 
desktop.  A bit slower, but it's going to last a whole lot longer. For 
my own personal use I never buy new; I'll take the same money that would 
buy a low-end current-year marvel and buy a three to five year-old 
Precision that will run faster and much longer.  My current laptop is a 
Precision M6700 with a Core i7-3740QM.  It was $600 and will run rings 
around anything built today at that price point (and even twice or 
thrice that price point I dare say!).


But we're talking servers here, and the LS20 blade for the BladeCenter 
is middle-of-the-road as far as server hardware is concerned.  The 
PE1950 is on the lower side of MOR.



A lot of commodity PC-grade SOHO “server” hardware won’t even last the 3 years 
between major CentOS upgrades before dying of something.  There was a period 
where I’d budget 1-2 years for a Netgear switch, for example.  (They appear to 
be lasting longer now.)


I haven't looked at the lower end of the server hardware scale in a long 
time, although we did get some older low-end Dell PE SC1425's donated to 
us a while back.  They run C7 quite well, too.  I'd rather buy a used 
higher-end box than a new low-end box, which is going to both cost more 
and wear out sooner.


But that's just SOP for a non-profit.


Second, the application of my quoted opinion to your situation is that you 
should run that hardware with CentOS 7 through the EOL of the hardware or 
software, whichever comes first.  That is, I’m advising the change-adverse 
members of the audience to opt into the second group above, taking OS changes 
in big lumps when it’s time to move to new hardware anyway.
There is no easy solution.  The sysadmin's work and continuing education 
is never done.  I don't mind learning new things nor is my budgeted time 
so tight that I can't spend company time getting familiar with newer 
admin paradigms.  I understand that everyone is not like me (which is 
probably a good thing).


The sysadmin 'political landscape' is not too different from the 
'regular' political landscape, really.  You have conservatives, and you 
have progressives.  They both think they're right, and they both tend to 
demonize those who disagree.  And both are growing more extremist with 
time.  Is there no middle ground to be had (in the sysadmin world, at 
least)?


I certainly understand and sympathize with James' 

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Valeri Galtsev]

On Mon, April 24, 2017 10:52 am, Warren Young wrote:
> On Apr 24, 2017, at 7:53 AM, Lamar Owen  wrote:
>> James' point isn't the hardware cost, it's the people cost for
>> retraining.
>
> Unless you’ve hired monkeys so that you must train them to do their
tasks by rote, that is a soft cost, not a hard cost.  If you’ve hired
competent IT staff, they will indeed need some time to work out the
differences, but they will do that on their own if only given that time.

I've been through that, I agree almost on all counts with James Byrne, so
I can give some comments from my chair here. Yes, I do consider myself a
notch more intelligent sysadmin than a monkey, and it does cost me time to
adjust to the differences, and it is annoying, and most annoying is to
adjust to some changes in philosophy (whoever considers the last
non-existent is allowed to re-qualify me back to the level of monkey ;-)

>
> Note also that Byrne’s solution was to move to an entirely different
OS,
> but we don’t hear about the “retraining cost” involved with that. 
Surely it was a larger jump from C6 or C7 to FreeBSD 10 than from C6 to
C7?

Yes and no. Maybe it is just my case, as I stared migrating servers to
FreeBSD even before C7 was released. FreeBSD feels closer to C5, whereas
difference between C5 and C7 is more dramatic (in my by no means objective
feeling). So, everyone who maintained C5 after quick "jump start" may feel
at hone with FreeBSD. My case may be even simpler as as many older
sysadmins I maintained a few UNIXes in the past, including FreeBSD.

>
> He also seems to be sweeping aside the fact that FreeBSD major releases
generally stay in support for about half the span of RHEL and its
derivatives.

True, but keeping your system incrementing in smaller steps that happen
more often is not a big deal. But this is a question of taste: both long
support life like RHEL and CentOS and shorter but smoother changes like
FreeBSD or some Linuxes (Debian and its clone Ubuntu come to mind) - they
both have their advantages and their place where they shine.

>  If he wants to stay on a supported OS the whole time that C7
> remains in support, he’s probably looking at 2 major OS version upgrades.

I've been through several FreeBSD major version upgrades on servers I
migrated to FreeBSD earliest, and they went smoothly, requiring just 3
reboots in the process. They all had a bunch of jails that were upgraded
as well. Not a single major issue that I had to resolve in a process (call
me lucky... knocking on wood ;-)

>
> It’ll be interesting to see how much change FreeBSD gets in the next 7
years.

It really is. Unless my usual luck in choosing what I expect to be in a
future fails me, not much change will happen to FreeBSD. I was thanking my
luck big time for choosing RedHat (and continuing to Fedora, then CentOS)
instead of Debian once when big flop in Debian (and all clones) was
discovered that was sitting there for over two years (search for Debian
predictable keys). My Debian friend was re-creating all his certificates,
re-generating ssh keys, rebuilding systems from scratch (as you don't know
who might have had root access to your box). And I was repeating myself,
that RedHat never had such a big flop. So I hope, I will be the same lucky
with my choice of FreeBSD as I was with my choice of RedHat (and clones)
back then.

And while we are here: My big thanks to RedHat, and big thanks to CentOS
team for the great job you guys are doing!! I wish I could help you more
than just maintaining CentOS and centosvault public mirrors.

Valeri

>
>> In many ways the Fedora treadmill is easier, being that there are many
more smaller jumps than the huge leap from C6 to C7.
>
> That depends on the organization and its goals.
>
> If you have a true IT staff that exists just to keep servers up to date
and working properly, then yes, you’re right, smaller upgrades every
3-6
> months are often easier to handle than trying to choke down 2-10 years
of
> changes all at once, depending on the LTS release strategy and how many
major upgrades you skip.
>
> If you’re trying to treat the OS as a base atop which you do something
else, and you just need something that will keep working for 2-10 years
despite being continually patched, then choking that big ball of changes
down every 2-10 years might be preferable.
>
> My main point is that if you’re going to take the second path, don’t
cry about how much change there is to choke down when you’re finally
forced to move forward.  You choose to put off dealing with it for many
years; the chickens have come back home to roost, so there will of
course
> be a lot of work to do.
>
>> ...dual-socket Opteron LS20 blades (10+ years old)...CentOS 7, once
installed, works great...
>
> That doesn’t really contradict my point.
>
> First, I said “most” hardware, but you’ve gone and cherry-picked
uncommonly durable hardware here; you’re probably out in +3 sigma
territory.  A lot 

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Warren Young]

On Apr 24, 2017, at 7:53 AM, Lamar Owen  wrote:
> 
> James' point isn't the hardware cost, it's the people cost for retraining.

Unless you’ve hired monkeys so that you must train them to do their tasks by 
rote, that is a soft cost, not a hard cost.  If you’ve hired competent IT 
staff, they will indeed need some time to work out the differences, but they 
will do that on their own if only given that time.

Note also that Byrne’s solution was to move to an entirely different OS, but we 
don’t hear about the “retraining cost” involved with that.  Surely it was a 
larger jump from C6 or C7 to FreeBSD 10 than from C6 to C7?

He also seems to be sweeping aside the fact that FreeBSD major releases 
generally stay in support for about half the span of RHEL and its derivatives.  
If he wants to stay on a supported OS the whole time that C7 remains in 
support, he’s probably looking at 2 major OS version upgrades.

It’ll be interesting to see how much change FreeBSD gets in the next 7 years.

> In many ways the Fedora treadmill is easier, being that there are many more 
> smaller jumps than the huge leap from C6 to C7.

That depends on the organization and its goals.

If you have a true IT staff that exists just to keep servers up to date and 
working properly, then yes, you’re right, smaller upgrades every 3-6 months are 
often easier to handle than trying to choke down 2-10 years of changes all at 
once, depending on the LTS release strategy and how many major upgrades you 
skip.

If you’re trying to treat the OS as a base atop which you do something else, 
and you just need something that will keep working for 2-10 years despite being 
continually patched, then choking that big ball of changes down every 2-10 
years might be preferable.

My main point is that if you’re going to take the second path, don’t cry about 
how much change there is to choke down when you’re finally forced to move 
forward.  You choose to put off dealing with it for many years; the chickens 
have come back home to roost, so there will of course be a lot of work to do.

> ...dual-socket Opteron LS20 blades (10+ years old)...CentOS 7, once 
> installed, works great...

That doesn’t really contradict my point.

First, I said “most” hardware, but you’ve gone and cherry-picked uncommonly 
durable hardware here; you’re probably out in +3 sigma territory.  A lot of 
commodity PC-grade SOHO “server” hardware won’t even last the 3 years between 
major CentOS upgrades before dying of something.  There was a period where I’d 
budget 1-2 years for a Netgear switch, for example.  (They appear to be lasting 
longer now.)

Second, the application of my quoted opinion to your situation is that you 
should run that hardware with CentOS 7 through the EOL of the hardware or 
software, whichever comes first.  That is, I’m advising the change-adverse 
members of the audience to opt into the second group above, taking OS changes 
in big lumps when it’s time to move to new hardware anyway.
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Lamar Owen]

On 04/20/2017 05:55 PM, Warren Young wrote:
... I find that most hardware is ready to fall over by the time the 
CentOS that was installed on it drops out of support anyway.

...


James' point isn't the hardware cost, it's the people cost for 
retraining.  In many ways the Fedora treadmill is easier, being that 
there are many more smaller jumps than the huge leap from C6 to C7. For 
the most part, however, I agree with most of your post.  I strongly 
disagree with the paragraph above, though.


I have worked for non-profits for most of my career thus far, which 
spans almost 30 years.  Non-profits by their very nature live on the 
slimmest of margins, and donations of hardware by individuals and 
companies have been in my experience the bread and butter for obtaining 
server-quality hardware.  The typical donation will be at least one or 
two generations old before the non-profit gets it; my current employer 
is just putting in production some IBM BladeCenters with the dual-socket 
Opteron LS20 blades (10+ years old).  Given the spiky workload, these 
blades are suitable for the targeted use, and the electrical 
requirements aren't a problem (I've done the math; it would take ten 
years or more to justify the purchase price of a new blade based on 
power savings alone, and our power is quite inexpensive here).  At least 
I can use very recent blades, and the eBay prices for 5-year-old blades 
are pretty good, so when I need that much more power I can get it.


Oh, and the LS20 blades are built like tanks.  We have a couple hundred 
of them that were donated, and we're going to use them.


For what it's worth, CentOS 7, once installed, works great as long as 
the lack of a GUI console isn't a problem (something with the 
BladeCenter's KVM switch and C7's kernel keeps the keyboard from working 
properly).


And don't even get me started on networking equipment, where I still 
have Catalyst 5500-series hardware in production.  (going on 20 years 
old and still trucking!)


And having said that, I just pulled out of service a server for another 
non-profit that had a power supply fan seize.  I posted about moving its 
application Friday.  It is an AMD K6-2/400 with a Western Digital 6GB 
boot drive and a Maxtor 30GB data drive, running Red Hat Linux 5.2.  The 
Antec power supply was put into service in 1999.  It stopped working 
Friday, and could have probably been put back into operation with a new 
power supply without a huge amount of work, but I decided it was time.  
Heh, it was time ten years ago!


The 6GB WD drive was only 19 years old; while I honestly wanted to see 
it turn 20, it was time (power supply glitches caused by overheating of 
the power supply; worst-case for hard disk death in my experience).  
Yeah, 24x7 operation for 19 years with minimal downtime.  I'm going to 
personally put it back into service for hysterical raisins, since the 
VA-503+ board doesn't need re-cap and it runs very well for what it is.  
I'm not sure what I'm going to run on it yet.  (It will be in service 
for the same reasons I'm going to put a Reh CPU280 running UZI280 into 
service.).




And that’s why I use *all* the major OSes and several weird ones besides.  None 
of it is perfect, yet it all has its place.

I couldn't agree more.

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Warren Young]

On Apr 20, 2017, at 7:33 AM, James B. Byrne  wrote:
> 
> When a vendor ... fundamentally changes the way the administration
> of an operating system is presented

I’ve gotten the sense from this other part of the thread that the answer to my 
question, “What are you moving to?” is FreeBSD.

If you think FreeBSD system administration hasn’t changed over the past 10 
years, you must not have been using it that long.  What makes you think it 
won’t change again in the next 10 years, possibly in very large breaking ways?

> vanishingly few firms in my
> experience (i.e.NONE) have ever had operational programming staff
> write or even modify a device driver.

My company is very small.  I’ve modified device drivers to make them work 
properly on Linux, purely in a “scratch my own itch” kind of way.

I assure, you, many larger organizations also do this or something similar.  
Netflix is famous for using FreeBSD on their streaming servers and for tuning 
the FreeBSD kernel heavily for that purpose.

> A business is in existence to
> make money for its owners not dick around with esoteric computer
> theory and practice.

I’m not glorifying change for its own sake.  I’m just saying it happens, and 
however inessential it may be to your business’ operations is really not 
on-point.  The fact is that it happens everywhere in this industry, so your 
only choice is in which bag of changes you want to deal with, not whether you 
get a bag of changes.

> The idea that one has to rebuild from scratch entire host systems and
> then laboriously port over data and customised portions to a new host
> simply to upgrade the underlying OS is absolutely ludicrous.

I find that most hardware is ready to fall over by the time the CentOS that was 
installed on it drops out of support anyway.

That is to say, I think the right way to use CentOS is to install one major 
version on the hardware when it’s built, and then ride it for the 7-10 years 
until that OS version drops out of support.  (7 being the worst case, when you 
install a new system jst before the next major OS version comes out.)

Then there’s all the change that is outside the OS proper.  For example, 
there’s all the current changes in the way encryption is handled, which would 
require operational changes anyway.  You can’t keep running BIND 4 on your 
public-facing DNS servers, for example, even if all the security problems were 
somehow fixed without changing any user interface.

Ditto mail, HTTP, and many other critical services, since old versions often 
don’t even speak today’s required protocols.  (TLS 1.1 minimum, DMARC, DKIM, 
SPF, etc.)

FreeBSD, this supposed bastion of stability, now actively discourages you from 
using BIND in the first place, for example.  Now they want you to migrate to 
NSD + Unbound.  Oh noes, more change!

> Consider
> the tremendous labour costs regularly incurred in accomplishing what
> amounts to maintaining the status quo.

If you only wanted the status quo ante, why upgrade at all?

Obvious answer: because you actually do want *some* change.

> We just upgraded a FreeBSD host from 10.3 to 11.0 in situ without
> problem

Lucky you.  I’ve had such upgrades take a system out for a day, working around 
all the breakages.

Upgrading FreeBSD is historically one of the most painful things about it.  
It’s getting better, but only by changing how everything about packaging was 
done.  Holy ChangeLogs, Batman!

Don’t get the wrong idea that I don’t like FreeBSD, by the way.  I know these 
things about it because I use it regularly.  This is one of those “bags of 
changes” I referred to above.  Sometimes I want the Linux bag, and sometimes I 
want the FreeBSD bag, and I know going into the decision that each bag implies 
a future bag of changes I’ll have to deal with.

> It was the OS running the metal for multiple BHyve virtual machines

Ah, more change.  Bhyve only goes back to FreeBSD 10, so if you were using 
FreeBSD prior to that, you’d have had to either drag forward whatever VM 
manager you were using or migrate to bhyve.

> given we use ZFS in FreeBSD, and that we snapshot regularly, getting
> back to 10.3 would have been, and still could be, nearly
> instantaneous.

That’s a great reason to pick FreeBSD.  Just don’t fool yourself that by 
switching that you’ve somehow gotten off the upgrade treadmill.  You’ve only 
switched bags.

> Systemd is not the problem.  It
> is a symptom of a deeper malaise, indifference.

systemd offers benefits to certain classes of end users which could not have 
been achieved without *some* kind of change.

We can argue about how well systemd did its job — I share many of the negative 
opinions about it — but I think you’ll have a very tough time convincing me 
that we could have gotten all the benefits without changing the user interface.

Again it comes back to the bag of features: if you didn’t want any of the 
features systemd brought, then you may be right to abandon 

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Warren Young]

On Apr 19, 2017, at 2:22 PM, Chris Murphy  wrote:
> 
> On Wed, Apr 19, 2017 at 5:21 AM, James B. Byrne  wrote:
>> 
>> On Mon, April 17, 2017 17:13, Warren Young wrote:
>> 
>>> Also, I’ll remind the list that one of the *prior* times the systemd
>>> topic came up, I was the one reminding people that most of our jobs
>>> summarize as “Cope with change.â€
>> 
>> At some point 'coping with change' is discovered to consume a
>> disproportionate amount of resources for the benefits obtained…Linux
>> does not meet our business needs.
> 
> Apple has had massively disruptive changes on OS X and iOS. Windows
> has had a fairly disruptive set of changes in Windows 10.

…And FreeBSD finally just got through the whole binary-package-everything 
phase, meaning installation and upgrade practices have changed almost entirely 
in the past few years.  And before that, there was “move everything to ZFS,” 
which totally changed file system management, the boot system, backups, at-rest 
data encryption, file sharing services, and more.

The other BSDs haven’t had as much change, but that’s both bad and good.

Solaris has had mighty shakeups in the past 10 so years, much before the Oracle 
buyout and much more after.

So what is Harte & Lyne Limited moving *to*?

> the Linux community appears to have a
> change-for-change-sake fetish.

Are you seriously saying that none of the change in the Linux world in the past 
10 years has had any practical benefit?

The only such charge that immediately comes to mind for me is this move in the 
past 5 years or so to “flat” user interfaces, led by the mobile world but 
infecting desktop OSes as well…but not Linux.  If Linux were doing change for 
change’s sake, wouldn’t Linux have flattened its UIs like Google, Apple, and 
Microsoft have?

I wonder if what you’d actually prefer is magical levels of foresight, so that 
we could have made all the change required 30, 40 years ago, and now just be 
riding on top of perfect stability?

Or is is that you think the *ix world already had perfection and lost it?  What 
was the perfect OS, what version, and does it still run your apps today?  Will 
it still run them 10 years from now?
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Jonathan Billings]

On Thu, Apr 20, 2017 at 09:33:30AM -0400, James B. Byrne wrote:
> Red Hat, again in my sole opinion, increasingly appears to me to be
> emulating another company notorious for shuffling the user interface
> to little evident purpose other than profit.  That is good business
> for them.  It is not good for us.

>From my perspective as a Red Hat customer who supports hundreds of
RHEL7 Workstation systems, Red Hat really doesn't seem to care or test
their Workstation product.  Their support doesn't seem to have much
training when it comes to problems with the GUI.  Since GNOME itself
moves along at a much faster pace than RHEL, I always end up looking
for archives of documentation, and trawling through GNOME's bugzilla.

Red Hat makes its business on the Server side.  They don't really care
about graphical user interfaces apart from the installer.

-- 
Jonathan Billings 
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Pete Biggs]

> 
> Think about what that would take in terms of man hours to accomplish
> moving from EL6 to 7.  And moving from 5 to 6 was not much better. 
> This is just too expensive to repeat every three years.

So why do it? There is absolutely nothing wrong with sticking with EL6
for a long time, certainly for the lifetime of the hardware - EL5 has
only just gone EoL, and if you pay RH you can still have it on support.
 Just because EL7 exists, it doesn't mean that you have to upgrade to
it. I've only just started to seriously roll out CentOS7, and then
mostly only on new machines.

P.
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[James B. Byrne]

On Wed, April 19, 2017 16:22, Chris Murphy wrote:

>
> Apple has had massively disruptive changes on OS X and iOS. Windows
> has had a fairly disruptive set of changes in Windows 10. About the
> only things that don't change are industrial OS's.
>

I have no idea how this reference applies to my earlier post.  We do
not use Apple or Windows servers and the desktop environment is
stabilised at Win7pro.  There will be be no Windows 10 here ever.  OSX
/ iOS employment is limited to personal devices, none of which are
permitted on premise in any case.

> When it comes to breaking user space, there's explicit rules against
> that in Linux kernel development. And internally consistent API/ABI
> stability is something you're getting in CentOS/RHEL kernels, it's one
> of the points the distributions exist. But the idea that Windows and
> OS X have better overall API stability I think is untrue, having
> spoken to a very wide assortment of developers who build primarily
> user space apps.

This may be true.  It is likely important to software developers.  It
is also totally irrelevant to a business.

Businesses, other than software development houses and consultants,
are software users.  When a vendor massively rearranges things in
their software, deprecates scripting syntax that has existed for years
if not decades, and fundamentally changes the way the administration
of an operating system is presented it really matter not a wit to a
business that the internal kernel level api remains unchanged.  It is
the accumulated administrative experience that is lost in consequence
that concerns a business given that replacing that loss will cost
either directly in retraining or indirectly in error and resultant
disruption; or both.

>
> What does happen, in kernel ABI changes can break your driver, as
> there's no upstream promise for ABI compatibility within the kernel
> itself. The effect of this is very real on say, Android, and might be
> one of the reasons for Google's Fuscia project which puts most of the
> drivers, including video drivers, into user space. And Microsoft also
> rarely changes things in their kernel, so again drivers tend to not
> break.
>
>

And this illustrates the point that I attempting to make.  A business
owner assumes that whatever OS is used it will deal with the various
devices that make up its hardware environment. For if it does not then
they seek an OS that does.  However, vanishingly few firms in my
experience (i.e.NONE) have ever had operational programming staff
write or even modify a device driver.  A business is in existence to
make money for its owners not dick around with esoteric computer
theory and practice.

Red Hat, again in my sole opinion, increasingly appears to me to be
emulating another company notorious for shuffling the user interface
to little evident purpose other than profit.  That is good business
for them.  It is not good for us.

Bear in mind that we have been RedHat/Whitebox/CA-OS/CentOS users
since 1998 so it is not like we are moving away from Linux with
anything like enthusiasm. But this upgrade treadmill that has
developed within RH is simply too costly for us to bear any longer. 
The idea that one has to rebuild from scratch entire host systems and
then laboriously port over data and customised portions to a new host
simply to upgrade the underlying OS is absolutely ludicrous. Consider
the tremendous labour costs regularly incurred in accomplishing what
amounts to maintaining the status quo.

We just upgraded a FreeBSD host from 10.3 to 11.0 in situ without
problem; and with very little downtime (three reboots in the space of
30 minutes).  This was no standalone device either.  It was the OS
running the metal for multiple BHyve virtual machines, themselves
running various operating systems (but mainly FreeBSD-11).  One of
said vms being our Samba-4 AD-DC.  And, had it all gone south then,
given we use ZFS in FreeBSD, and that we snapshot regularly, getting
back to 10.3 would have been, and still could be, nearly
instantaneous.

Think about what that would take in terms of man hours to accomplish
moving from EL6 to 7.  And moving from 5 to 6 was not much better. 
This is just too expensive to repeat every three years.

And allow me to forestall any claims that the chimera that is 'cloud
computing' is the answer. All that does is make creating the requisite
new platforms marginally less tedious. And that small advantage is
purchased at the cost of handling over control of all your data to
entities who are thoroughly discredited with respect to security and
privacy.

I am not anti or pro systemd, upstart, or etc/rc (or any other
software although I admit to holding a generally dim view of things
from Redmond).  I do not really care what is used so long as it works
and that introducing it does not greatly diminish the value of
existing user skills and knowledge.  However, I am past the point of
patience with gratuitous changes that offer no appreciable benefit 

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Chris Murphy]

On Wed, Apr 19, 2017 at 5:21 AM, James B. Byrne  wrote:
>
> On Mon, April 17, 2017 17:13, Warren Young wrote:
>
>>
>> Also, I’ll remind the list that one of the *prior* times the systemd
>> topic came up, I was the one reminding people that most of our jobs
>> summarize as “Cope with change.â€
>>
>
> At some point 'coping with change' is discovered to consume a
> disproportionate amount of resources for the benefits obtained.  In my
> sole opinion the Linux community appears to have a
> change-for-change-sake fetish. This is entirely appropriate for an
> experimental project.  The mistake that I made many years ago was
> inferring that Linux was nonetheless suitable for business.
>
> To experimenters a ten year product cycle may seem an eternity. To
> many organisations ten years is barely time to work out all the kinks
> and adapt internal processes to automated equivalents.  And the
> smaller the business the more applicable that statement becomes.
>
> I do not have any strong opinion about systemd as I have virtually no
> experience with it.  But the regular infliction of massively
> disruptive changes to fundamental software has convinced us that Linux
> does not meet our business needs. Systemd and Upstart are not the
> cause of that.  They are symptoms of a fundamental difference of focus
> between what our firm needs and what the Linux community wants.

Apple has had massively disruptive changes on OS X and iOS. Windows
has had a fairly disruptive set of changes in Windows 10. About the
only things that don't change are industrial OS's.

When it comes to breaking user space, there's explicit rules against
that in Linux kernel development. And internally consistent API/ABI
stability is something you're getting in CentOS/RHEL kernels, it's one
of the points the distributions exist. But the idea that Windows and
OS X have better overall API stability I think is untrue, having
spoken to a very wide assortment of developers who build primarily
user space apps.

What does happen, in kernel ABI changes can break your driver, as
there's no upstream promise for ABI compatibility within the kernel
itself. The effect of this is very real on say, Android, and might be
one of the reasons for Google's Fuscia project which puts most of the
drivers, including video drivers, into user space. And Microsoft also
rarely changes things in their kernel, so again drivers tend to not
break.


-- 
Chris Murphy
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[James B. Byrne]

On Mon, April 17, 2017 17:13, Warren Young wrote:

>
> Also, I’ll remind the list that one of the *prior* times the systemd
> topic came up, I was the one reminding people that most of our jobs
> summarize as “Cope with change.”
>

At some point 'coping with change' is discovered to consume a
disproportionate amount of resources for the benefits obtained.  In my
sole opinion the Linux community appears to have a
change-for-change-sake fetish. This is entirely appropriate for an
experimental project.  The mistake that I made many years ago was
inferring that Linux was nonetheless suitable for business.

To experimenters a ten year product cycle may seem an eternity. To
many organisations ten years is barely time to work out all the kinks
and adapt internal processes to automated equivalents.  And the
smaller the business the more applicable that statement becomes.

I do not have any strong opinion about systemd as I have virtually no
experience with it.  But the regular infliction of massively
disruptive changes to fundamental software has convinced us that Linux
does not meet our business needs. Systemd and Upstart are not the
cause of that.  They are symptoms of a fundamental difference of focus
between what our firm needs and what the Linux community wants.

-- 
***  e-Mail is NOT a SECURE channel  ***
Do NOT transmit sensitive data via e-Mail
 Do NOT open attachments nor follow links sent by e-Mail

James B. Byrnemailto:byrn...@harte-lyne.ca
Harte & Lyne Limited  http://www.harte-lyne.ca
9 Brockley Drive  vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada  L8E 3C3

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Warren Young]

On Apr 15, 2017, at 12:19 AM, Anthony K  wrote:
> 
> Also, there's a lot of people moving to FreeBSD - but it appears that the 
> grass isn't greener there either as they are now trialling OpenRC.

You appear to have misunderstood my post.

First, TrueOS is not FreeBSD.  TrueOS is to FreeBSD as Ubuntu is to Debian, 
kinda-sorta.  Some of the things the TrueOS people do make their way back into 
FreeBSD, but TrueOS largely exists for those who want an easier desktop 
experience than stock FreeBSD or want a semi-supported bleeding-edge 
distribution of FreeBSD.

Now that TrueOS is based on the CURRENT (i.e. bleeding-edge) branch of FreeBSD 
development, TrueOS also serves a pioneer role for FreeBSD: those being the 
guys with all the arrows in their backs.

Because of that, TrueOS’s adoption of OpenRC doesn’t mean FreeBSD will follow 
suit.  Maybe they will, maybe they won’t.

Second, it’s not a “trial”.  It was announced, and then suddenly between two 
versions BSD rc was switched to OpenRC.  No “are you sure,” no “here are the 
consequences,” no “sorry, what you’re doing here is incompatible.”  Just boom, 
best-effort automatic conversion; if it breaks, you get to keep both pieces.

(Kinda makes you smile when you remember all the threads from those who found 
out that RHEL family OSes can’t self-upgrade between major versions.  Suddenly 
it’s looking like a feature.  Imagine if the EL6 to EL7 transition happened the 
same way.)

FreeBSD proper splits the difference between these two upgrade methods.  You 
have to explicitly opt into minor version upgrades, and automatic major version 
upgrades are possible but always offered with plenty of warnings and migration 
advice.

If you want a FreeBSD-specific lesson from this, it would be “don't run 
12.0-CURRENT on critical servers.”

Also, I’ll remind the list that one of the *prior* times the systemd topic came 
up, I was the one reminding people that most of our jobs summarize as “Cope 
with change.”
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Always Learning]

On Sun, 2017-04-16 at 18:25 +0100, Pete Biggs wrote:


> Yes. And despite what people think, those agencies don't have super
> powers. They have tools to help them, and lots of resources, but
> nothing out of the ordinary.

Untrue. They are in advance of mainstream developments. Spying has
existed for thousands, of years *and* it is their job to discover and
then discretely monitor what is going-on.

It is never one team doing everything but many highly specialist teams
dedicated to particular aspects of intelligence gathering which they do
expertly, and impressively, well.

All countries monitor, by all available means, what is happening in
their own territory and around the world. Just because, for example, the
USA and Russia are not officially loving buddies it never ever prevents
their intelligence agencies covertly sharing intelligence of mutual
interest. It is a incestuous world with an international web of contacts
doing favours and often disregarding their own government's official
political pronouncements.

>  There is nothing that the NSA can do that can't be done by other
>  agencies or even individuals (or enough individuals working together).

Mmmm, you forgot physical access to targets :-) That is one of their
advantages together with links into national infrastructures and
seemingly endless money. They are much more audacious than "normal"
people.

> There is no doubt that every single security agency in the world has a
> team working on discovering exploitable code in all operating systems.
> It's what they do. Any exploit they find that has been reported is
> probably because some other agency has found it as well so they want to
> stop them using it.

Not only software but hardware too. Most hardware has backdoors which
may not be routinely disclosed to purchasers. The question then arises
if the "official" backdoor is the only one. Difficult to detect if the
logic is coded on a chip.

> The only truly secure machine is one that is at the bottom of a mine
> shaft, turned off and dismantled. :-)

Nope, just protected from public networks like the Internet and from
radio transmissions of all types. Faraday-cage types and 'high-security
rooms' don't have to be buried at the bottom of mines; they exist
everywhere.



-- 
Regards,

Paul.
England, EU.  England's place is in the European Union.

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Gordon Messmer]

On 04/16/2017 03:53 AM, ken wrote:
And, yes, the exploits also include more than a few against linux.  Go 
to their site and look under vault7.  Or search for "linux" or 
"redhat"... you'll get hundreds of hits.  Here's just one: 
https://wikileaks.org/spyfiles4/documents/FinSpy-3.10-User-Manual.docx 
(If you have only a few seconds to look at it, see page 34.) 



That document appears to describe a remote control application, not an 
exploit.  It's only useful once you have administrative access to the 
system in question.


I won't say that I don't think exploits against Linux systems exist, 
that would be naive.  But, I haven't yet seen any CVEs for GNU/Linux 
systems resulting from the Vault7 leaks.


___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Pete Biggs]

> Indeed. I think the assertion "OSS is somehow safer because of community
> audit" is a logical fallacy. How would one go about "auditing" in the first
> place?

There are tools to audit source code for problems - OSS is safer
*because* the source is available and can be audited. 

>  Even if the various Intelligence agencies are not injecting
> vulnerabilities then they would certainly be in a strong position to
> discover some of the holes already existing some time before they become
> public.

Yes. And despite what people think, those agencies don't have super
powers. They have tools to help them, and lots of resources, but
nothing out of the ordinary. There is nothing that the NSA can do that
can't be done by other agencies or even individuals (or enough
individuals working together).

There is no doubt that every single security agency in the world has a
team working on discovering exploitable code in all operating systems.
It's what they do. Any exploit they find that has been reported is
probably because some other agency has found it as well so they want to
stop them using it.

> 
> Unless you're operating an air gap network you can be damn sure that 'they'
> can get into your systems if they really want to.

The only truly secure machine is one that is at the bottom of a mine
shaft, turned off and dismantled. :-)

P.

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Pete Biggs]

On Sun, 2017-04-16 at 06:53 -0400, ken wrote:
> On 04/15/2017 04:46 AM, Pete Biggs wrote:
> > Not wishing to extend this thread further, but ...
> > 
> > > There are conspiracy theories out there that the NSA is involved with
> > > bringing systemd to Linux so they can have easy access to *"unknown"*
> > > bugs - aka backdoors - to all Linux installations using systemd *[1]*.
> > 
> > They're conspiracy theories, and that's it.
> 
> Hmm.  That's not quite it.  Wikileaks recently posted a trove of docs on 
> CIA exploits.  It was big news.  I'm surprised you missed that.  And, 
> yes, the exploits also include more than a few against linux.

That's not what I said - I said that the security agencies writing
backdoors into systemd was a conspiracy theory. I said later that they
have exploits as part of their toolkit. I'm surprised you missed that
part when you replied to it ...


> Years ago it was revealed that one of the linux developers inserted an 
> exploit into the gcc code which, when the login code was compiled, would 
> give him access to any system running it, effectively every linux 
> system.  This exploit was in the linux code for a long time and was 
> never discovered.  It was revealed only by the developer himself, and 
> only because he was retiring.  Point is: Code is often complex, 
> especially that written in C (or C++ and others), so much so that an 
> exploit can be written into it and not discovered for a long time, or 
> ever.  This is yet another argument against systemd: it would be much 
> easier to hide an exploit in it than in a handful of bash scripts.

Perhaps bash is exploitable - designed to hide the malicious code put
into the init.d scripts by the NSA.

P.

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Alice Wonder]

On 04/16/2017 06:51 AM, Andrew Holway wrote:


There is no doubt that most security agencies have a long list of zero-

day exploits in their toolbox - I would hazard to suggest that they
wouldn't be doing their job if they didn't! But I seriously doubt they
would commission exploitable code in something that is openly
auditable.

P.



P., I used to think that too... indeed, I was thoroughly convinced of it.
But reality changed my mind.



Indeed. I think the assertion "OSS is somehow safer because of community
audit" is a logical fallacy. How would one go about "auditing" in the first
place? Even if the various Intelligence agencies are not injecting
vulnerabilities then they would certainly be in a strong position to
discover some of the holes already existing some time before they become
public.


I'm more worried about cloud services and the large number of root 
certificates that software trusts by default.


That's where a lot of the hacks are going to happen, and AFAIK the only 
defense against it is DNSSEC + DANE which very few zones actually utilize.


___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Andrew Holway]

>
> There is no doubt that most security agencies have a long list of zero-
>> day exploits in their toolbox - I would hazard to suggest that they
>> wouldn't be doing their job if they didn't! But I seriously doubt they
>> would commission exploitable code in something that is openly
>> auditable.
>>
>> P.
>>
>
> P., I used to think that too... indeed, I was thoroughly convinced of it.
> But reality changed my mind.


Indeed. I think the assertion "OSS is somehow safer because of community
audit" is a logical fallacy. How would one go about "auditing" in the first
place? Even if the various Intelligence agencies are not injecting
vulnerabilities then they would certainly be in a strong position to
discover some of the holes already existing some time before they become
public.

Unless you're operating an air gap network you can be damn sure that 'they'
can get into your systems if they really want to.
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Jonathan Billings]

On Apr 16, 2017, at 6:53 AM, ken  wrote:
> Years ago it was revealed that one of the linux developers inserted an 
> exploit into the gcc code which, when the login code was compiled, would give 
> him access to any system running it, effectively every linux system.  This 
> exploit was in the linux code for a long time and was never discovered.  It 
> was revealed only by the developer himself, and only because he was retiring. 
>  Point is: Code is often complex, especially that written in C (or C++ and 
> others), so much so that an exploit can be written into it and not discovered 
> for a long time, or ever. This is yet another argument against systemd: it 
> would be much easier to hide an exploit in it than in a handful of bash 
> scripts.


When you say “one of the linux developers”, you mean Ken Thompson?

http://wiki.c2.com/?TheKenThompsonHack 

This story predates Linux, and describes a problem with any potential software. 
 

You realize ‘bash’ could be just as malicious as systemd in this scenario?  Are 
you meticulously going through *it’s* source code in your version of the world? 
 Note:  bash is not written in bash.

--
Jonathan Billings 


___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[ken]

On 04/15/2017 04:46 AM, Pete Biggs wrote:

Not wishing to extend this thread further, but ...


There are conspiracy theories out there that the NSA is involved with
bringing systemd to Linux so they can have easy access to *"unknown"*
bugs - aka backdoors - to all Linux installations using systemd *[1]*.

They're conspiracy theories, and that's it.


Hmm.  That's not quite it.  Wikileaks recently posted a trove of docs on 
CIA exploits.  It was big news.  I'm surprised you missed that.  And, 
yes, the exploits also include more than a few against linux.  Go to 
their site and look under vault7.  Or search for "linux" or "redhat"... 
you'll get hundreds of hits.  Here's just one: 
https://wikileaks.org/spyfiles4/documents/FinSpy-3.10-User-Manual.docx 
(If you have only a few seconds to look at it, see page 34.)




The bottom line is that in
general people don't like not understanding things and when they come
across something they don't understand they create a mythology around
those things to rationalise their non-understanding.


True, but that "mansplanation" can point in a lot of ways, including at 
Pollyanna.





Systemd is complex; it's implementation was badly handled on a social
level. Nevertheless it is open source. It is highly unlikely that the
NSA, or any other agency, would risk putting in backdoors to code that
could be audited by Joe "random hacker" Blogs, let alone that might be
discovered by hostile agencies.


Years ago it was revealed that one of the linux developers inserted an 
exploit into the gcc code which, when the login code was compiled, would 
give him access to any system running it, effectively every linux 
system.  This exploit was in the linux code for a long time and was 
never discovered.  It was revealed only by the developer himself, and 
only because he was retiring.  Point is: Code is often complex, 
especially that written in C (or C++ and others), so much so that an 
exploit can be written into it and not discovered for a long time, or 
ever.  This is yet another argument against systemd: it would be much 
easier to hide an exploit in it than in a handful of bash scripts.



There is no doubt that most security agencies have a long list of zero-
day exploits in their toolbox - I would hazard to suggest that they
wouldn't be doing their job if they didn't! But I seriously doubt they
would commission exploitable code in something that is openly
auditable.

P.


P., I used to think that too... indeed, I was thoroughly convinced of 
it.  But reality changed my mind.

___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Pete Biggs]

Not wishing to extend this thread further, but ...

> There are conspiracy theories out there that the NSA is involved with 
> bringing systemd to Linux so they can have easy access to *"unknown"* 
> bugs - aka backdoors - to all Linux installations using systemd *[1]*. 

They're conspiracy theories, and that's it. The bottom line is that in
general people don't like not understanding things and when they come
across something they don't understand they create a mythology around
those things to rationalise their non-understanding. Factor in to that
the general mindset of Linux hackers/admins that they must know and
understand every part of their system and you create the perfect
environment for such theories to grow and blossom.

Systemd is complex; it's implementation was badly handled on a social
level. Nevertheless it is open source. It is highly unlikely that the
NSA, or any other agency, would risk putting in backdoors to code that
could be audited by Joe "random hacker" Blogs, let alone that might be
discovered by hostile agencies.

There is no doubt that most security agencies have a long list of zero-
day exploits in their toolbox - I would hazard to suggest that they
wouldn't be doing their job if they didn't! But I seriously doubt they
would commission exploitable code in something that is openly
auditable.

P.


___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos

Re: [CentOS] OT: systemd Poll - So Long, and Thanks for All the fish.

[Anthony K]

On 09/04/17 14:39, Anthony K wrote:


So, at which stage are you in w/ regards to adopting systemd?  Are you 
still ridiculing it, violently opposed to it, or have you mellowed to it? 
Thanks for all those that responded. systemd still appears to be a sore 
topic.


systemd is still coping a whole lot of ridicule but not so violent 
opposition.  Can't say I understand why, but you can't please all of the 
people all of the time!


Quick comments to some issues identified in the conversation:
=

There are several responses siting poor documentation but I can't fault 
the documentation; there's plenty of it and the man pages are quite well 
structured - man -k --names-only systemd


Also, there's a lot of people moving to FreeBSD - but it appears that 
the grass isn't greener there either as they are now trialling OpenRC.


One issue I resolved quickly after installing CentOS 7 was to revert to 
ethx for interface names and to install iptables and remove firewalld.  
The other occassional issue I have is where restarting services takes a 
seriously long time and I've discovered that restarting 
`systemd-logind.service, dbus.service, and polkit.service resolves this, 
albeit for a short period before it crops up again *[0]*.



In closing:
###
There are conspiracy theories out there that the NSA is involved with 
bringing systemd to Linux so they can have easy access to *"unknown"* 
bugs - aka backdoors - to all Linux installations using systemd *[1]*.  
I guess anything goes now that Edward Snowden has educated us all - for 
better or worse.



Thanks again to all respondents - I quite enjoyed the read - I did read 
all responses.



Regards,
ak.

*[0]* - https://github.com/systemd/systemd/issues/2925
*[1]* - 
https://www.google.com.au/search?complete=0=en=webhp=hp=nsa+and+systemd


___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos