On 08/05/2014 09:03 AM, Thierry Carrez wrote:
Hi everyone,
With the incredible growth of OpenStack, our development community is
facing complex challenges. How we handle those might determine the
ultimate success or failure of OpenStack.
With this cycle we hit new limits in our processes, tools and cultural
setup. This resulted in new limiting factors on our overall velocity,
which is frustrating for developers. This resulted in the burnout of key
firefighting resources. This resulted in tension between people who try
to get specific work done and people who try to keep a handle on the big
picture.
It all boils down to an imbalance between strategic and tactical
contributions. At the beginning of this project, we had a strong inner
group of people dedicated to fixing all loose ends. Then a lot of
companies got interested in OpenStack and there was a surge in tactical,
short-term contributions. We put on a call for more resources to be
dedicated to strategic contributions like critical bugfixing,
vulnerability management, QA, infrastructure... and that call was
answered by a lot of companies that are now key members of the OpenStack
Foundation, and all was fine again. But OpenStack contributors kept on
growing, and we grew the narrowly-focused population way faster than the
cross-project population.
At the same time, we kept on adding new projects to incubation and to
the integrated release, which is great... but the new developers you get
on board with this are much more likely to be tactical than strategic
contributors. This also contributed to the imbalance. The penalty for
that imbalance is twofold: we don't have enough resources available to
solve old, known OpenStack-wide issues; but we also don't have enough
resources to identify and fix new issues.
We have several efforts under way, like calling for new strategic
contributors, driving towards in-project functional testing, making
solving rare issues a more attractive endeavor, or hiring resources
directly at the Foundation level to help address those. But there is a
topic we haven't raised yet: should we concentrate on fixing what is
currently in the integrated release rather than adding new projects ?
We seem to be unable to address some key issues in the software we
produce, and part of it is due to strategic contributors (and core
reviewers) being overwhelmed just trying to stay afloat of what's
happening. For such projects, is it time for a pause ? Is it time to
define key cycle goals and defer everything else ?
On the integrated release side, "more projects" means stretching our
limited strategic resources more. Is it time for the Technical Committee
to more aggressively define what is "in" and what is "out" ? If we go
through such a redefinition, shall we push currently-integrated projects
that fail to match that definition out of the "integrated release" inner
circle ?
The TC discussion on what the integrated release should or should not
include has always been informally going on. Some people would like to
strictly limit to end-user-facing projects. Some others suggest that
"OpenStack" should just be about integrating/exposing/scaling smart
functionality that lives in specialized external projects, rather than
trying to outsmart those by writing our own implementation. Some others
are advocates of carefully moving up the stack, and to resist from
further addressing IaaS+ services until we "complete" the pure IaaS
space in a satisfactory manner. Some others would like to build a
roadmap based on AWS services. Some others would just add anything that
fits the incubation/integration requirements.
On one side this is a long-term discussion, but on the other we also
need to make quick decisions. With 4 incubated projects, and 2 new ones
currently being proposed, there are a lot of people knocking at the door.
Thanks for reading this braindump this far. I hope this will trigger the
open discussions we need to have, as an open source project, to reach
the next level.
Yes.
Additionally, and I think we've been getting better at this in the 2
cycles that we've had an all-elected TC, I think we need to learn how to
say no on technical merit - and we need to learn how to say "thank you
for your effort, but this isn't working out" Breaking up with someone is
hard to do, but sometimes it's best for everyone involved.
I'm wary of explicit answers in the form of policy at this point - I
think we've spent far too long using policy as a shield from hard
questions. The questions of "Is OpenStack IaaS, or should it also be
PaaS" I think are useless questions that exist only to further the lives
of analysts, journalists and bloggers. Instead, I'd love to focus more
on "what is software that solves problems" and "what is software that
solves problems that we're in a good position to solve" So if Savanna
solves problems for users well in a way that makes sense, I'd hate to
exclude it because it didn't meet a policy's pre-conceived notion that
Hadoop is not "IaaS" or even "IaaS+" You know what I NEVER do when I'm
working on deploying services? I never say "Hey Jim, we should go
operate our IaaS resources" Literally, it has never happened.
Finally, as it pertains to technical merits, and at the risk of
magicking myself into a corner of defining a policy after saying I don't
like them ... I'd really like to think a bit more about the nature of
what we do and what we excel at. I'll be the first to admit I've been
involved in transgressions of this - but I believe it's becoming clear
to me that where we as a general community and as a software project
excel are the places where we're orchestrating and/or gluing things
together that are existing technologies. Nova orchestrates hypervisors,
cinder provides iscsi, keystone has long resisted being a user
management system and instead is a glue layer between openstack services
and existing user management systems. Trove orchestrates databases - it
did not attempt to write an RDBMS from scratch in python. Simply turning
that into a rule of thumb is perhaps overly simplistic and could lead
into the dangerous land of adhering to policy in the face of sanity. But
for myself, I'm going to be thinking about all of our projects through
that lens over the coming months. Please don't be surprised or take it
personally if I start suggesting that we've been too liberal in
suggesting projects that sit on the other side of that line, and that
perhaps its time to bring that chapter in our evolution to a close.
Monty
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
