Date: Tue, 30 Mar 2021 21:23:50 -0400 (EDT)
From: David P. Reed <[email protected]>
To: Theodore Ts'o <[email protected]>
Cc: Make-Wifi-fast <[email protected]>,
Cake List <[email protected]>,
cerowrt-devel <[email protected]>,
bloat <[email protected]>
Subject: Re: [Make-wifi-fast] [Cerowrt-devel] wireguard almost takes a bullet
Theodore -
I appreciate you showing the LF executive salary numbers are not quite as high
as I noted. My numbers may have been inflated, but I've definitely seen a
$900,000 package for at least one executive reported in the press (an executive
who was transferred in from a F100 company which is close to the LF).
On the other hand, they are pretty damn high salaries for a non-profit. Are
they appropriate? Depends. There are no stockholders and no profits, just a
pretty substantial net worth.
Regarding the organizaton of "Linux, Inc." as a hierachical control structure - I'll
just point out that hierarchical control of the development of Linux suggests that it is not at all
a "community project" (if it ever was). It's a product development organization with
multiple levels of management.
Yet the developers are employees of a small number of major corporations. In this sense,
it is like a "joint venture" among those companies.
To the extent that those companies gain (partial) control of the Linux kernel, as appears
to be the case, I think Linux misrepresents itself as a "community project",
and in particular, the actual users of the software may have little say in the direction
development takes going forwards.
There's little safeguard, for example, against "senior management" biases in
support of certain vendors, if other vendors are excluded from effective participation by
one of many techniques. In other words, there's no way it can be a level playing field
for innovation.
In that sense, the Linux kernel community has reached a point very much like
Microsoft Windows development reached in 1990 or so. I note that date because
at that point, Microsoft was challenged with a variety of anti-trust actions
based on the fact that it used its Windows monopoly status to put competitors
in the application space, and competitors producing innovative operating
systems out of business (GO Computer Corporation being one example of many).
This troubles me. It may not trouble the developers who are in the Linux
community and paid by the cartel of companies that control its direction.
I have no complaint about the technical competence of individual developers - the quality
is pretty high, at least as good as those who worked on Windows and macOS. But it's
becoming clear that their is a narrowing of control of an OS that has a lot of influence
in a few hands. That those few hands don't work for one company doesn't eliminate its
tendency to become a cartel. (one that is not transparent at all about functioning as
such - preferring to give the impression that the kernel is developed by part-time
voluntary "contributions").
The contrast with other open source communities is quite sharp now. There is
little eleemosynary intent that can be detected any more. I think that is too
bad, but things change.
This is just the personal opinion of someone who has been developing systems
for 50+ years now. I'm kind of disappointed, but my opinion does not really
matter much.
David
On Monday, March 29, 2021 9:52pm, "Theodore Ts'o" <[email protected]> said:
On Mon, Mar 29, 2021 at 04:28:11PM -0400, David P. Reed wrote:
>
>
> What tends to shape Linux and FreeBSD, etc. are the money sources
> that flow into the communities. Of course Linux is quite
> independently wealthy now. The senior executives of the Linux
> Foundation are paid nearly a million dollars a year, each. Which
> just indicates that major corporations are seriously interested in
> controlling the evolution of Linux (not the Gnu part, the part that
> has Linus Torvalds at its center).
First of all, I don't believe your salary numbers are correct.
https://nonprofitlight.com/ca/san-francisco/linux-foundation
Secondly, the "senior executives" of the Linux Foundation don't have
any control over "the evolution of Linux". The exception to that are
the "Fellows" (e.g., Linus Torvalds, Greg K-H, etc.) and I can assure
you that they don't take orders from Jim Zemlin, the executive
director, or any one else at the Linux Foundation.
The senior developers of Linux do tend to work for the big
corporations, but culturally, we do try to keep our "corporate hats"
and our "community" hats quite separate, and identify when we our
company hats on. Many senior developers have transitioned between
multiple companies, and over time, it's been understood that their
primarily allegiance is to Linux, and not to the company. In fact,
the primary job of maintainers is to say "no" to companies when they
try to push crap code into the kernel. And that's because it's the
maintainer's responsibility to clean up the mess if they say yes to
code that's Just Not Ready, since they have a long-term responsbility
towards their subsystem, unlike engineers or contractors that only
have a short-term goal to get the code upstream.
This is where having a hierarchial ownership model IMHO works better
than a "core team" model where there can be a diffusion of
responsibility, where anyone with a commit bit can commit anywhere in
the OS. In contrast, David Miller "owns" the networking area, and so
someone who might be, say, the ext4 or xfs maintainer does not have
the right (read: Linus will reject a pull request from me if I try to
change code in the networking stack with out DaveM's signoff) to
change code outside of their subsystem.
So you're right that Linus probably doesn't know or care about
bufferbloat. He's delegated pretty much all networking issues to
David Miller as the networking czar, and within networking, David
Miller has his submaintainers with different specialities. This does
get complicated when there are changes which cross subsystems. For
example, before Wireguard could land in the kernel, there were changes
needed in both the crypto and networking layers, and Jason had to
negotiate with multiple senior developers in those subsystems, and the
code was subject to quite a lot of review before it could land. (It
took months, and we didn't try to rush things before a major
release....)
> I just spent 9 months trying to get a very tiny fix to the Linux
> kernel into the mainline kernel. I actually gave up, because it
> seemed utterly pointless, even though it was clearly a design error
> that I was fixing, and I was trying to meet all the constraints on
> patches. No one was fighting me, no one said it was wrong.
It sounds like the real problem is no one was paying attention to you.
There is a *huge* number of changes going into the Linux kernel, and
so the the challenge is getting review bandwidth by the relevant
maintainers. Blindly posting to the linux-kernel mailing list will
generally not get you very far.
The Linux development process is not really optimized for "drive by
patching". Knowng where (and to whom) a patch needs to be reviewed is
not necessarily easy for a novice, and while there are tools such as
./scripts/get_maintainer.pl that try to make it a bit easier, I can
see how someone who Just Wants To Get A Single Patch accepted, can see
it as "bureaucracy".
Cheers,
- Ted
