I will reply to both Kiran and Alex mail here.
Kiran :
"a small correction it is only master-slave replication atm" :
Hmm, too bad. We need MMR. I overlooked that then. The question is : can
we do the MMR between RC1 and RC2 ?
What
----
(FTR, The last release we did, Apache Directory Server 1.5.7 was issued on
april, 24th, 2010)
Even if we release a 2.0-RC1 in the very near future say a couple weeks from
now, I'd recommend releasing a 1.5.8 so the features and changes made since
1.5.7 can be tested by users. There have been some big changes especially on
the Kerberos side.
A release would take less than a weeks time (vote and all) and should not
hold back the work being done to progress towards 2.0.
I have three concerns with approach :
- first, we agreed to let down the 1.5.8 last year. That's not a blocker
though.
- second, even if it takes one week to get the release done, it's still
one week. Assuming we are close to 2.0 (ie, 4 to 8 weeks), then I don't
think it worths losingthis week in an intermediate and short live release
- most important, if we release a 1.5.8, that will send a signal to
users that they have to migrate to 1.5.8. Those who will do, and will
see a 2.0 release a few weeks (months?) later may be stuck for a long
time in such a release.
I'm not extracting this from thin air, we actually have one user, and
not the least one (IBM Rational Directory Server) who is *stuck* in
1.5.5, and they won't migrate to 1.5.8. They probably will jump to 2.0
when out.
Hopefully, we have JIRA which stores the issues, tasks and wishes, and it's
the first input we will use to determinate the 'what'. Let's move back to
the past a bit. In may, 26th, 2010, a mail about the version scheme was sent
and discussed on the directory ML [1][2]. What was proposed back and agreed
then was :
- skip ADS-1.5.8 and target a 2.0.0-RC1 instead
- skip shared-0.9.20 and target for a 1.0.0-RC1 instead
- Keep the RC in order to give us the opportunity to stabilize the code
*and* the documentation
This might have been presumed thinking 2.0 was not far away. Now we're 8
months out since the last release. So we might want to re-evaluate this, and
you made some points about not releasing shared 1.0.0-RC1 to avoid having to
solidify those API's for 2.0 because this might slow 2.0 down.
yes, because the API will have a *massive* impact on users. This is
critical, and I share your vision about the stability requirement here.
I don't think that it's the same story for the server.
What was also raised at this time is that once we will jump for a
2.0(server) and 1.0(api), we will be stuck with the exposed interfaces for a
very long time, so we'd better be careful. This still stand.
Yeah this is my main worry. Let me though put this discussion out into
another thread or two for better clarity and tracking.
We must distinguish between the LdapAPI (ie, merge of shared and Client
API) and the server API (ie core-api plus the base classes in shared :
entry, etc).
I'm less worried about the Server API than I'm for the LdapAPI.
Also I'd like to quote you on that aspect, because I find your post
sensible
(http://www.mail-archive.com/dev@directory.apache.org/msg28128.html) :
"Also I remember a great email conversation almost 7 years ago on
another project's mailing list. It was agreed that releasing fast and
releasing often was more important than making sure the code was
absolutely stable. Firstly because you can never really be absolutely
stable and second because only life in the wild will show if the code is
fit enough: fixing sometimes what you think is the big issue does not
really fix the big problems in the wild. *Third and most important was
the idea that stability is not that important. Yeah you heard me right.*
Just get the feature out and let the user community report the problems
so you can fix it fast but fix it fast and release fast. This keeps the
user community more involved with the project and communicating with us.
It's actually the best way to keep a vibrant community and spend less
time testing/improving something which you really cannot tell is broken.
You only know in the wild. I'm not saying write and release buggy
software and nor were these guys saying this. I'm saying let's fear not
providing stability on the first major release. If we blindly stick to
the release early release often strategy and communicate with our users,
then eventually we will converge towards stability with the least
testing effort and the greatest gain in community. So let's make sure we
get the releases out fast to fix the issues that arise for our users
when they contact us with problems. This way we can stimulate a more
vibrant user community while allowing us to focus more time in the right
places developing rather than testing."
That does not mean we should disregard the Server API stability. We must
do it, even if it postpone the 2.0 release for a few weeks.
We all agree that the "release often" mantra is a good one, but I don't
think it applies well with the version scheme we are using. However, we
decided to follow this versioning scheme, even if it sounds like a dead
body we are carrying, and I think it's better delaying a long expected
release for a few more weeks rather than delivering an intermediary
version and to release a major version a few weeks later. It sounds not
consistent to me.
We then started to move forward, cleaning some parts that needed it,
following the classic #bug report :
- installers were totally rewrote [8]
- configuration has been completely re-factored, it's now stored in DIT
- all the 1.5.8/1.5.9 issues have been moved to 2.0.0-RC1 in JIRA
- performances of each operations were greatly improved (by a factor 2 to
4) [5]
- some discussions about the JDBM backend occured, we agreed we should have
a MVCC backend sooner or later, but no clear decision was even stated about
when we should do that. However, recollecting my memory, it seems that it's
out of question for 2.0.
- Multi-Master replication has been added [2]
- we moved documentation from Confluence to use Dockbook instead [6]
- PasswordPolicy have been added
- the ACI subsystem has been reviewed [3]
- some refactoring of shared has been discussed and started [4]
- the various LDAP messages data structure have been merged (a long
expected task)
- some discussion started about the shared/API merge [7][9][10]
- the kerberos codec has been completely rewritten [11]
- AdministrativeModel implementation refactoring has been started (support
of IAP is at stake)
Sounds to me like we should get a 1.5.8 release out to the community fast
while continuing with 2.0. There's a lot of things that could use real work
tire kicking here. It will not necessarily slow us down and it will get us a
bit more tidy for 2.0. We're still going to have to tidy up as usual for
2.0-RC1.
See upper.
All those parts have been heavily discussed, as discussion about a 2.0
release started back in january 2009 ! [12] This effort has been pursued and
in may 2010, we started a bug parade [13] (reports are available on [14],
[15], [16], [17], [18]). Then we hit some serious issues in the code that
needed more than a few lines of code, bt some serious refactoring (namely,
config in DIT, ACI subsystem, Kerberos refactoring, AdministrativeModel).
Those huge refactoring stopped the release effort until they are done.
Right now, we are almost ready to start again to run the bug parade, once
the Administrative Model is finished (still working on that).
Release early release often. If we get a 1.5.8 out and let peeps kick the
tires but tell them, look 2.0 is coming soon so don't get hooked on this
WDYT?
See upper.
Sooo... time to review what's missing, and what's need some love now. This
is the 'What' part we all are looking at, and the best point to start it
will be to look at opened JIRAes. Otherwise, we still have to discuss some
serious re-factoring to be done in Shared.
Back in december, Alex started to review Shared, we discussed about it, and
we agreed that the work started on september was not enough. The rationals
are clear :
- shared and API *are* the same thing. We want to merge the client-API and
shared to provide a LdapAPI (and we even started a new project for that).
- many classes are useless, and need to be removed. Alex started to do that
in a shared branch
Yeah I was doing some experiments in my branch not certain one way is better
than another but after seeing some nice impact I can start solidifying a
direction and express my findings here.
We are all waiting for some feedbacks. Those modifications are a direct
follow up of the first round of modifications we have started back in
september. We didn't finished, you picked up the torch, and that's good.
There may be some other elements, but nothing is yet decided, so it's fine
to add there what is missing.
Otherwise, we have to remember that we have 3 different 'release' at stake
here :
- ADS 2.0
- Studio 2.0
- LdapAPI 1.0 (ex-shared)
LdapAPI is absolutely *critical*. Once it's out, there is no way for us to
change it, because it will be heavily sued, as it's meant to be a
replacement for JNDI, and many of the outdated LDAP API. So we have to be
extraordinary cautious in this area.
Exactomundo!!! This is my massive fear. There was immense work done here and
it's awesome and can save the world from JNDI hell when dealing with LDAP.
So we need to do it right and adhere to API maintenance conventions.
Unfortunately though the LDAP Client API is not only restricted to the ldap
client api module, the API's surface area spans across several shared
modules: really almost all of them. So these API's don't have clear
boundaries. I'll discuss this in another thread.
Sure. let's not hijack this long thread...
However, and that's the good point, we can spend as much time as needed to
get LdapAPI 1.0 out, as it won't be a show-stopper for ADS and Studio. It
doesn't matter if we continue with a shared-0.9.20 in ADS 2.0 or in Studio
2.0, because ADS will be used way more often as a standalone server, or even
if it's embedded, the exposed interface will be the core-API part, which is
not part of LdapAPI (except a few classes which will need some careful
review).
I wanted to save this for a separate email but just as a hint. The surface
area of the server's API includes shared libraries, since the LDAP access
and operation model are defined within shared modules. So there's no clear
API boundary to the core, it slurps in shared and it becomes one big API. So
trying to lock in core-api is futile if you don't lock in shared libraries
to follow the same change conventions.
If we thik about subentries, for instance, it's not an issue if LdapAPI
(which should contain those classes) is not locked down when ADS 2.0 is
out of the door, simply because a Server user won't see those classes.
It's for internal use only, and there is no reason for a user to
manipulate them.
But anyway, we have to do a map of what is used and what is not, and we
will have a better view.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
