[this post and accompanying images are available online at 
https://s.apache.org/i1tM ]

EDITOR'S NOTE: I came across the author's original post, "An Introduction to 
Apache Software — What you need to know", dated 3 February 2017, and was 
interested in finding away to share with the greater Apache community. The 
author's enthusiasm was palpable, and earnestly intended to help educate 
others. With the ASF celebrating its 20th Anniversary this year, it's easy for 
many of us to simply rely on tribal knowledge, not realizing that navigation to 
definitive guides aren't intuitive to newcomers. Those of us who have been here 
for a while "just know", partially because we were creating it as we went 
along. Below is an updated version of the original post, amended through the 
guidance of three long-standing ASF Members. And that's the point of it all at 
the end of the day: at Apache, we help each other as it contributes to our 
collective success, and this writeup will help others find their Success at 

by Maximilian Michels

Before you started reading this post, you have already been using Apache 
software. The Apache web server (Apache HTTP Server) serves about every second 
web page on the WWW, including this website. One could say, Apache software 
runs the WWW. But it doesnt stop there. Apache is more than a web server. 
Apache software also runs on mobile devices. Apache software is part of 
enterprise and banking software. Apache software is literally everywhere in 
today's software world.

Apache has become a powerful brand and a philosophy of software development 
which remains unmatched in the world of open-source. Although the Apache® 
trademark is a known term even among the less tech-savvy people, many people 
struggle to define what Apache software really is about, and what role it plays 
for today's software development and businesses.

In the last years I've learned a lot about Apache through my work on Apache 
Flink and Apache Beam with dataArtisans, as a freelancer/consultant, and as a 
volunteer. In this post I want to give an overview of the Apache Software 
Foundation and its history. Moreover, I want to show how the "Apache way" of 
software development has shaped the open-source software development as it is 

The History of the Foundation
The Apache Software Foundation (ASF) was founded in 1999 by a group of 
open-source enthusiasts who saw the need to create a legal entity to 
institutionalize their work. Among the first projects was the famous web server 
called Apache HTTP, which is also simply referred to as "Apache web server". At 
that time, the Apache web server was already quite mature. In fact, not only 
did the Apache web server give the foundation its name but it became the role 
model for the "Apache way" of open and collaborative software development. To 
see how that took place, we have to go back a bit further in time.

A Web Server goes a long way
As early as 1994, Rob McCool at the National Center for Supercomputing 
Applications (NCSA) in Illinois created a simple web server which served pages 
using one of the early versions of today's HTTP protocol. Web servers were not 
ubiquitous like they are today. In these days, the Web was still in its early 
days and there was only one web browser developed at CERN where the WWW was 
invented only shortly before. Rob's web server was adopted quite fruitfully 
throughout the web due to its extensible nature. When its source code spread, 
web page administrators around the world developed extensions for the web 
server and helped to fix errors. When Rob left the NCSA in late 1994, he also 
left a void because there was nobody left to maintain the web server along with 
its extensions. Quickly it became apparent that the group of existing users and 
developers needed to join forces to be able to maintain NCSA HTTP.

At the beginning of 1995, the Apache Group was formed to coordinate the 
development of the NCSA HTTP web server. This led to the first release of the 
Apache web server in April 1995. During the same time, development at NCSA 
started picking off again and the two teams were in vivid exchange about future 
ideas to improve the web server. However, the Apache Group was able to develop 
its version of the web server much faster because of their structure which 
encouraged worldwide collaboration. At the end of the year, the Apache server 
had its architecture redone to be modular and it executed much faster.

One year later, at the beginning of 1996, the Apache web server already 
succeeded the popularity of the NCSA HTTP which had been the most popular web 
server on the Internet until then. Apache 1.0 finally was released on Dec 1, 
1995. The web server continued to thrive and is still the most widely used web 
browser as of this writing.

The Rise of the Foundation

The team effort that led to the development and adoption of the Apache web 
server was a huge success. The Apache project kept receiving feedback and code 
changes (also called patches) from people all over the world. Could this be the 
development model for future software? More and more projects started to 
organize their groups similarly to the Apache group. As the number of project 
grew, financial interests arose and potential legal issues threatened the 
existence of the Apache group. Out of this need, the Apache Software Foundation 
(ASF) was incorporated as a US 501(c)(3) non-profit organization in June 1999. 
In the US, the 501(c)(3) is a legal entity specifically designed for non-profit 
charitable organizations. This is in contrast to other for-profit open-source 
software organizations or even US 501(c)(6) non-profit organizations which do 
not require to be charitable.

After the ASF was incorporated, new projects could easily leverage the 
foundation's services. Over the next year, every few months a new project 
entered the ASF. The first projects after Apache HTTP Server were Apache 
mod_perl (March 2000), Apache tcl (July 2000), and Apache Portable Runtime 
(December 2000). After a short break in 2001 which was used to come up with a 
programmatic approach to onboard new projects via an incubator, the ASF has 
seen very consistent growth of up to 12 projects (2012) each year.

The ASF became a framework for open-source software development which, in its 
entirety, remains unmatched by other forms of open-source software development. 
The secret of ASF's success is its unique approach to scaling its operations, 
in which the foundation does not try to exercise control over its projects. 
Instead, it focuses on providing volunteers with the infrastructure and a 
minimal set of rules to manage their projects. The projects itself remain 
almost autonomous.

Apache Governance - How does the foundation work?
There are about 200 independent projects running under the Apache umbrella. The 
question may arise, how does the foundation govern its projects? First of all, 
the ASF is an organization that is run almost entirely by volunteers. In the 
early days, many of the volunteers were developers which did not like to spend 
much time with administrative things (who does?), so the organization is 
structured in a way that requires little central control but favors autonomy of 
the projects which run under its umbrella.

Per-Project Entities
For every project (e.g. Apache HTTP, Apache Hadoop, Apache Commons, Apache 
Flink, Apache Beam, etc.), there are a Project Management Committee (PMC), 
Committers, Contributors, and Users.

Project Management Committee (PMC)
The PMC manages a project's community and decides over its development 
direction. Its most rudimentary and traditional role is to approve releases for 
a project. In that sense it has a similar function as the original Apache Group 
which led the development of Apache HTTP Server. When a new project graduates 
from the Incubator (covered later), the foundation's central instance, the 
Board, approves the initial PMC which is selected from the PPMC (Podling PMC) 
formed during incubation. Each PMC elects one PMC member as the PMC Chair which 
represents the project and writes quarterly reports to the ASF Board. The Chair 
needs to be approved by the Board.

Through a project's lifetime new PMC members can be elected by the existing 
PMC. Note that each new PMC member needs to be approved by the Board but this 
approval is merely formal and there are few instances that a new PMC member is 
not approved. PMC members do not need the formal permission of the foundation 
to elect new Committers. PMC members themselves are also Committers. Let's 
learn about Committers next.

Committers can modify the code base of the project but they can't make 
decisions regarding the governance of the project. They are trusted by the PMC 
to work in the interest of the project. When they contribute changes, they 
commit (thus, the name) these changes to the project. Committers don't only 
change code but they can also update documentation, write blog posts on the 
project's website, or give talks at conferences. Committers are selected from 
the users of the project; more about this process in the Meritocracy section.

Users and Contributors
Users are as important as the developers because they try out the project’s 
software, report bugs, and request new features. The term is a slightly 
confusing because, in the Apache world, most users tend to be developers 
themselves. They are users in the sense that they are using an Apache project 
for their own work; usually they are not actively developing the Apache 
software they are using. However, they may also provide patches to the 
Committers. Users who contribute to a project are called Contributors. 
Contributors may eventually become Committers.

In the image, the per-project entities are represented as circles. They exist 
for every project. Note that the user group circle is not depicted in full size 
because big projects tend to have much more Users than Committers and PMC 

Foundation-Wide Entities
The ASF does not work without some central services. Here are the most 
important entities:

Apache Members
Apache members represent the heart of the foundation. They have been referred 
to as the "shareholders of the ASF" because they are deeply invested in the ASF 
(not in the financial sense). A prerequisite to becoming a member is to be 
active in at least one project. To become a member, you have to show interest 
in the foundation and try to promote its values. The ASF holds membership 
meetings which are usually held annually. At membership meetings new members 
can be proposed and subsequently elected. Elected members receive an invitation 
which they can choose to accept within 30 days. Becoming a member it not merely 
a recognition for supporting the ASF, but it also grants the right to elect the 

The Board of Directors (Board)
The Board takes care of the overall government of the foundation. In 
particular, it is concerned with legal and financial matters like brand and 
licensing issues, fundraising, and financial planning. The board is elected by 
the Apache members annually and is also composed of Apache members. The current 
board can be viewed here. Note that there is only one central Board for the 
entire foundation but Board members can be PMC members in different projects.

Officers of the corporation
The Officers of the corporation are the executive part of the administration. 
They execute the decisions of the board and take care of everyday business. 
Most of the officers are implicitly officers by being the PMC chair of a 
project. Additionally, there are dedicated officers for central work of the 
foundation, e.g. fundraising, marketing, accounting, data privacy, etc.

Infrastructure (INFRA)
The support and administration team (INFRA) is the team that runs the Apache 
infrastructure and provides tools and support for developers. INFRA is the only 
team at Apache which consists of contractors which are paid for their work. 
Their work includes running the apache.org web site and the mailing lists which 
are Apache’s main way of communication. Over time, various other tools and 
services were created to assist the projects. The main tools available which 
are used by almost all projects are:

Web space for the project's websites.
Mailing lists, for discussing the roadmap of the project, exchanging ideas, or 
reporting bugs (unwanted software behavior). 

 - Typically the mailing lists are divided into a developer and a user mailing 
 - Bug trackers, which help developers to keep track of new features or bugs.
 - Version control, which helps developers to keep track of the code changes.
 - Build servers, which help to integrate/test new code or changes to existing 

The Incubator
Founded in 2002, the Incubator is a project at the ASF dedicated to forming 
(bootstrapping) new Apache projects. The process is the following: People 
(volunteers, enthusiasts, or company employees) make a proposal to the 
Incubator. The proposal contains the project name, the list of initial PPMC 
(Podling PMC) members, and the motivation and goals for the new project. Once 
the IPMC (Incubator PMC) has discussed the proposal, it holds a vote to decide 
if the project enters the incubation phase. In the incubation phase, projects 
carry "incubating" in their names, e.g. "Apache Flink (incubating)"; this is 
dropped only once they graduate. To graduate, a project has to show that it is 
mature enough. The Community Development project at the ASF has created a 
catalogue of criteria called the Maturity Model. It requires having an active 
community, quality of code, and being legally compliant. Formally, the project 
needs to prove it fulfils the criteria to the Incubator IPMC which is comprised 
of Apache members. All existing work donated in the course of entering the 
incubator and all future work inside the project has to be licensed to the ASF 
under the Apache License. This ensures that development remains in the 
open-source according to the Apache philosophy. More about incubation on the 
official website.

Meritocracy - How are decisions made?
The Apache Software Foundation uses the term "meritocracy" to describe how it 
governs itself. Going back to the ancient Greeks, meritocracy was a political 
system to put those into power which proved that they were motivated, put 
effort into their work, and were able to help a project. The core of this 
philosophy can be found throughout history from ancient China to medieval 
Europe and is still present in many of today’s cultures in the sense that 
effort, increased responsibility, and service to a part of society ought to pay 
off in terms of power of decision, social status, or money.

Meritocracy in the Apache Software Foundation denotes that people who either 
work in the interest of the foundation or a project get promoted. Users who 
submit patches may be offered Committer status. Comitters who are drive a 
project, may gain PMC status. PMC members active across projects and taking 
part in the foundation's work may earn the Member status.

Decision-making within the foundation and projects are typically performed 
using Consensus. Consensus can be "lazy" which implies that even a few people 
can drive a discussion and make decisions for the entire community as long as 
nobody objects. The discussions have to be held in public on the mailing list. 
For instance, if a Committer decides to introduce a new feature X, she may do 
so by proposing the feature on the mailing list. If nobody objects, she can go 
ahead and develop the feature. If lazy consensus does not work because an 
argument cannot be settled, a majority based vote can be started.

Meritocracy and "lazy" Consensus are the core principles for governance within 
the Apache Software Foundation. Meritocracy ensures that new people can join 
those already in power. "Lazy" Consensus creates the opportunity to split up 
decision-making among the group such that it doesn't always require the action 
of all members of the community.

The Apache License - A license for the world of open-source

With the incorporation of the foundation in 1999, a license had to be created 
to prevent conflicts with the intellectual property contributed by others to 
the ASF. Originally, the license was meant to be used exclusively by the ASF 
but it quickly became one of the most widely used software licenses for all 
kinds of open-source software development.
The Apache license is very permissive in the sense that source code 
modifications are not required to be open-sourced (made publicly available) 
even when the source code is distributed or sold to other entities. This is in 
contrast to “Copyleft” licenses like the GNU Public License (GPL) which, upon 
redistribution, requires public attribution and publication of changes made to 
the original source code. The Apache license was first derived from the BSD 
license which is similarly permissive. The reason for this was that the Apache 
HTTP Server was originally licensed under the BSD license.

The current version of the Apache License is 2.0, released in January 2004. The 
changes made since the initial release are only minor but they set the 
prerequisite for its prevalence. At first, the license was only available to 
Apache projects. Due to the success of the Apache model, people also wanted to 
use the license outside the foundation. This was made possible in version 2.0. 
Also, the new version made it possible to combine GPL code with Apache licensed 
code. In this case, the resulting product would have to be licensed under the 
GPL to be compatible with the GPL license. Another change for version 2.0 was 
to make inclusion of the license in non Apache licensed projects easier and 
require explicit patent grants for patent-relevant parts.

Apache Today

The ASF today is not the small group that it used to be back in 1999. At the 
time of this writing, the Apache Software Foundation hosts 51 podlings in the 
Incubator and 199 top-level committees (PMCs). This amounts to almost 300 
projects (latest statistics). Note that, a PMC may decide to host multiple 
projects if necessary. For instance, the Apache Commons PMC has split up the 
different parts of the Apache Commons library into separate projects (e.g. CLI, 
Email, Daemon, etc.). 50 of the 300 projects have been retired and are now part 
of Apache Attic, the project which hosts all retired projects. The above graph 
is taken from https://projects.apache.org.

Apache Conferences

The Apache Software Foundation regularly organizes conferences around the world 
called ApacheCon. These conferences are dedicated to the Apache community or 
certain topics like Big Data or IoT. It is a place to meet community members 
and learn about the latest ideas and trends within the global Apache community. 
Apart from the official conferences, there are conferences on Apache software 
organized by companies or external organization, e.g. Strata, FlinkForward, 
Kafka Summit, Spark Summit.

Here's a list of some projects that I came across in the past. I grouped them 
into categories for a better overview. I realize you might not know a lot of 
the projects but maybe this list can be the starting point to discover more 
about these Apache projects :)

Big Data
 - Hadoop
 - Flink
 - Spark
 - Beam
 - Samza
 - Storm
 - NiFi
 - Kafka
 - Flume
 - Tez
 - Zeppelin

 - Mesos
 - CloudStack
 - Libcloud

Machine Learning
 - Mahout

 - OpenOffice

 - CouchDB
 - HBase
 - Zookeeper
 - Derby
 - Cassandra

Query Tools / APIs
 - Hive
 - Pig
 - Drill
 - Crunch
 - Ignite
 - Solr
 - Lucene

Programming Languages
 - Groovy

 - Bigtop
 - Ambari
 - Libraries

 - Avro
 - Thrift
 - ActiveMQ
 - Parquet

Developer Tools
 - Ant
 - Maven
 - Ivy
 - Subversion

Web Servers
 - HTTP (the one!)
 - Tomcat

Web Frameworks
 - Cocoon
 - Struts
 - Sling

Apache - A Successful Open-Source Development Model
My first attempt to learn more about Apache goes back several years. I was 
using the Apache License while working on Scalaris at Zuse Institute Berlin. I 
realized that the license was somehow connected to the Apache Software 
Foundation but I didn't really understand the depth of this relationship until 
I started working on Apache Flink with dataArtisans. Besides the official 
homepage of the foundation, relatively little information was available on the 
Internet about the foundation and its projects. In hindsight, the best source 
of information was to read the email archives, get to know other people at the 
ASF, and become a volunteer myself :)

When I originally wrote this post I couldn’t find an introductory guide to the 
ASF. So I decided to do a bit of research myself and tried to write down what I 
had learned working on Apache projects. I hope that I could provide an overview 
of the ASF and show you how significant the foundation has been for the 
open-source software development.

Thank you
Thank you for reading this article. Feel free to write me an email if I got 
something wrong or you would like to comment on anything.

Thank you Roman Shaposhnik, Shane Curcuru, Dave Fisher, and Sally Khudairi for 
your comments which were very helpful to revise this post for the 20th 
anniversary of the ASF.

 - Web Server usage
 - Apache Software Foundation
 - Apache History
 - About Apache
 - Robert McCool
 - CERN httpd
 - Apache HTTP
 - Initial press release
 - Apache License
 - Apache License Version 2.0
 - How it works
 - Apache Dev
 - Apache Incubator
 - Apache projects
# # #

"Success at Apache" is a monthly blog series that focuses on the people and 
processes behind why the ASF "just works". 

= = =

NOTE: you are receiving this message because you are subscribed to the 
announce@apache.org distribution list. To unsubscribe, send email from the 
recipient account to announce-unsubscr...@apache.org with the word 
"Unsubscribe" in the subject line.

Reply via email to