Thank you for your input.
Items like P_ I_ and other general code clean up I was hoping to do in a
different phase. many of us could easy make a massive diff with all most of the
issues corrected. However this diff would make most of our heads explode. I
think that it is very important to make each phase small and directed to help
allow movement in a good direction. I know personally there are a number of
item in the code itself I would like to address as well as fixing up the
testing system. I believe that getting the relayout down first will help all
other efforts as this will allow for clearer and more understandable diffs when
we address issues such as fixing up the P_ I_ stuff or adding new tests, etc.
Personally I would like to address:
1) provide an alternative build system to the current auto-tools2) clean up the
testing. This is not small amount of work, and I would hope that I would be
able to provide a better subsystem that we all can use. One that is more
accessible and useful than what we have at the moment.3) some stuff in the code
like the memory allocator and threaded event system, which i have a background
in doing
But before this can be done correctly I really believe we have to clean up the
source layout. I want to get this done first and as quick as possible to reduce
the pain caused by the source moves that would upset existing ( and future)
pull requests.
Jason
From: Phil Sorber <[email protected]>
To: [email protected]; Jason Kenny <[email protected]>
Sent: Tuesday, January 26, 2016 12:32 PM
Subject: Re: Proposal for how to update source code layout.
This seems generally good to me. I welcome a more modern structure. I also
prefer the first layout you mentioned, but it may be necessary to do the second
layout and then move to the first. That might make others more comfortable. I
personally see a lot of value in what you describe, but I know others may still
need more convincing and specific details.
Also, you didn't mention some things specifically, but I assume we are doing
away with things like the P_ and I_ in the header files? We should also make it
so that the docs don't get needlessly built. I'm sure there are a bunch of
other things, but those are the ones that come to mind.
Another thing I think we need to be careful with is people who have long lived
feature branches. They really need to be rebasing against master a lot or
perhaps just pushing their changes upstream quicker.
Be prepared for a lot of bike shedding. This is likely to be a long slow
process as we address the merits of each change and how it fits into the bigger
picture and what are the pros and cons of each.
On Mon, Jan 25, 2016 at 3:43 PM Jason Kenny <[email protected]>
wrote:
Hi everyone,
Second try as the first attempt seems to be a blob of unformated text.
I want to see about moving forward on the source clean up I believe we all
agreed to the last ATS summate. What I would like to get general agreement on
what I plan to do to the source layout so it is clear why and where everything
will go before I start making lots of small pull requests pulling moving source
and cleaning up the build.
First what are the goals?
Cleaning up the source to will proved:
1) Better clarity on where to put new code.
a. Making it easier to refactor code
b. Making it easier for others to change code
2) simplify the build code
a. making it easier to use other build system if desired
b. making it easier for others to add or remove code
3) untangle the dependency mess of everything needing everything else
4) untangle what modules are being built
5) Make it easier for everyone to understand the Architecture and
design
6) Make it easier to improve/cleanup the testing systems
7) Allow a better setup to hopefully avoid any major source layout
refactors for a long time
I want to clarify that this was generally agreed on by everyone at the ATS
summate. We want to avoid this unless needed. I believe people feel overall
this would be a good thing, but I want to make sure that I define the
envisioned layout we will move to so we all understand the where we are going
before doing this. Ideally we will want to do the major part of this work as
fast as we can to minimize the pain as much as possible.
Before I go into the layout, I want to talk about process. I plan to break this
up into phases.
Phase 1 – move source to new location and any header cleanup that might be
needed. Once this phase is done we will still have dependency issues to deal
with, however it should be clearer on what is wrong, and some of the dependency
issues should just go away
Phase 2 – refactor source to deal with dependency issues, to clean up mega
source files and start some cleanup of the tests. This will make it easier to
find objects and make changes in a way to that should not make compiling more
complex. It should also make it easier to understand and learn the code.
The hope I have is that will help minimize the size of any given diff. Will
have more diff overall, but these will all be small and to the point.
Now let talk about layout (finally :-) )
The current layout has something like this at the top layer
/build
/ci
/cmd
/contrib
/doc
/example
/iocore
/lib
/mgmt
/plugins
/proxy
/rc
/tools
Source needed to build Traffics Server exits in /proxy, /mgmt, /lib, /cmd and
/iocore. These directories also have tests in them as well. In some case the
tests and the source is mixed. I would like to change the top level to look
like this instead:
/build
/ci
/contrib
/doc
/plugins
/rc
/src
/tests
/tools
We move the code in /examples, /proxy, /mgmt, /lib, /cmd and /iocore. The
addition of a src and test directory will help clarify what is the source to
than traffic server application vs that of being a test or something else, such
as an example or an optional plug-in. The code for /examples moves under
/plugins, everything else under /src
In the /src directory we would have a layout like this:
src/
api/
c/
cpp/
config/
cmd/
…
core/
…
Infra/
…
Details:
api – contains all the API code for a given language. Any new language API
would be added here under a directory correctly names for the language.
config- Contains the default config file to run traffic server.
cmd – Contains the code for each program in a name directory.
core – Contains the main modules to build different programs under the cmd/
directory. Modules under here depend on modules in Infra, or other modules in
Core itself. Code here should be more Traffic server specific modules.
infra – Contains the support modules. Modules in here are leaf modules, may be
reusable. If the module is not a leaf module it can only depend on code in
Infra and should be generally reusable.
Dependencies:
Modules should not depend on other modules that depend on them. The only
expectations would be header only dependencies that don’t require link time
circular dependencies.
Modules in CMD can depend on modules in API, Core, Infra
Modules in API can depend on modules in Core, Infra
Modules in Core can depend on modules in Core, Infra
Modules in Infra can depend on modules in Infra
Header Patterns:
The general include pattern will be changed to #include <module/header.h> for
modules depending in on other headers. Files depending on headers in the same
module will be in the form of #include “header.h”
An alternative to this layout would be
Src/
api/
c/
cpp/
config/
mgmt/
proxy/
…
iocore/
…
lib/
…
The main difference is here is that it look like it did before. However much of
the code in proxy or mgmt would be moved to lib as it is a leaf component, or
the code would be moved in to a directory based on module it would be used to
generate, vs having lots of modules in a given directory. Because of this I
felt it might be easier to use the first layout as it contains a simpler set of
logic on where code goes.
Given that we agree with the layout. The process will be to move components
that are on the leaf first and slowly move up the chain. Each commit will be on
component at a time, with changes to the source files #include and any make
file changes to allow the code to build as expected. After a component is moved
that changes that break up source can happen, where the commit will include the
makefile changes and the split source any header file changes need to support
the split.
I think that is it. I will deal with “tests” after main source changes.
Any question or concerns?
Jason
