My two cents...
I think the committers do a great job of managing the product. I
feel the single biggest failure when it comes to producing quality
software is lack of vision, and/or enforcement of this vision.
If every "wisher" or "submitter" had their code committed - even if
it is "good code" - the product would quickly become unwieldy to
maintain and/or learn (for new users), lessening its usefulness to
everyone.
The only problem I have with Lucene's current focus is that I feel
the Lucene folks should work on standardizing the API, focusing on
interfaces and/or abstract classes with proper protected level access.
By doing this, people are much freer to develop their own
enhancements, and can quickly apply them to later Lucene releases
just by applying a patch (at worst), or just a link (at best !).
Similar to how the JDK works. We have rarely if ever needed to change
our code between JDK releases.
I realize this is a dream right now, because of the bad shape (sorry)
of the structure of much of Lucene, but if the committers spent more
time on issues like this, I think they would hear far less complaints
from the community.
As an example of the above - being able to access the underlying
readers in a multi-reader (I know there is a current bug for this).
There is no harm to Lucene folks to expose this, and it is very
helpful in many cases. If some developer uses this information in the
wrong way, that is their fault, not Lucene's.... Making something
protected is very different than making it public.
Robert Engels
On Dec 3, 2008, at 11:36 PM, John Wang wrote:
Grant:
I am sorry that I disagree with some points:
1) "I think it's a sign that Lucene is pretty stable." - While
lucene is a great project, especially with 2.x releases, great
improvements are made, but do we really have a clear picture on how
lucene is being used and deployed. While lucene works great running
as a vanilla search library, when pushed to limits, one needs to
"hack" into lucene to make certain things work. If 90% of the user
base use it to build small indexes and using the vanilla api, and
the other 10% is really stressing both on the scalability and api
side and are running into issues, would you still say: "running
well for 90% of the users, therefore it is stable or extensible"? I
think it is unfair to the project itself to be measured by the
vanilla use-case. I have done couple of large deployments, e.g. >30
million documents indexed and searched in realtime., and I really
had to do some tweaking.
2) "You want stuff committed, keep it up to date, make it
manageable to review, document it, respond to questions/concerns
with answers as best you can. " - To some degree I would hope it
depends on what the issue is, e.g. enforcing such process on a one-
line null check seems to be an overkill. I agree with the process
itself, what would make it better is some transparency on how
patches/issues are evaluated to be committed. At least seemed from
the outside, it is purely being decided on by the committers, and
since my understanding is that an open source project belongs to
the public, the public user base should have some say.
3) which brings me to this point: "I personally, would love to work
on Lucene all day every day as I have a lot of things I'd love to
engage the community on, but the fact is I'm not paid to do that,
so I give what I can when I can. I know most of the other
committers are that way too." - Is this really true? Isn't a large
part of the committer base also a part of the for-profit,
consulting business, e.g. Lucid? Would groups/companies that pay
for consulting service get their patches/requirements committed
with higher priority? If so, seems to me to be a conflict of
interest there.
4) "Lather, rinse, repeat. Next thing you know, you'll be on the
receiving end as a committer." - While I agree that being a
committer is a great honor and many committers are awesome, but
assuming everyone would want to be a committer is a little
presumptuous.
In conclusion, I hope I didn't unleash any wrath from the
committers for expressing candor.
-John
On Wed, Dec 3, 2008 at 2:52 PM, Grant Ingersoll
<[EMAIL PROTECTED]> wrote:
On Dec 3, 2008, at 2:27 PM, Jason Rutherglen (JIRA) wrote:
Hoss wrote: "sort of mythical "Lucene powerhouse"
Lucene seems to run itself quite differently than other open source
Java projects. Perhaps it would be good to spell out the reasons
for the reluctance to move ahead with features that developers work
on, that work, but do not go in. The developer contributions seem
to be quite low right now, especially compared to neighbor projects
such as Hadoop. Is this because fewer people are using Lucene? Or
is it due to the reluctance to work with the developer community?
Unfortunately the perception in the eyes of some people who work on
search related projects it is the latter.
Or, could it be that Hadoop is relatively new and in vogue at the
moment, very malleable and buggy(?) and has a HUGE corporate
sponsor who dedicates lots of resources to it on a full time basis,
whilst Lucene has been around in the ASF for 7+ years (and 12+
years total) and has a really large install base and thus must move
more deliberately and basically has 1 person who gets to work on it
full time while the rest of us pretty much volunteer? That's not
an excuse, it's just the way it is. I personally, would love to
work on Lucene all day every day as I have a lot of things I'd love
to engage the community on, but the fact is I'm not paid to do
that, so I give what I can when I can. I know most of the other
committers are that way too.
Thus, I don't think any one of us has a reluctance to move ahead
with features or bug fixes. Looking at CHANGES.txt, I see a lot
of contributors. Looking at java-dev and JIRA, I see lots of
engagement with the community. Is it near the historical high for
traffic, no it's not, but that isn't necessarily a bad thing. I
think it's a sign that Lucene is pretty stable.
What we do have a reluctance for are patches that don't have tests
(i.e. this one), patches that massively change Lucene APIs in non-
trivial ways or break back compatibility or are not kept up to
date. Are we perfect? Of course not. I, personally, would love
for there to be a way that helps us process a larger volume of
patches (note, I didn't say commit a larger volume). Hadoop's
automated patch tester would be a huge start in that, but at the
end of the day, Lucene still works the way all ASF projects do: via
meritocracy and volunteerism. You want stuff committed, keep it
up to date, make it manageable to review, document it, respond to
questions/concerns with answers as best you can. To that end, a
real simple question can go a long way and getting something
committed, and it simply is: "Hey Lucener's, what else can I do
to help you review and commit LUCENE-XXXX?" Lather, rinse,
repeat. Next thing you know, you'll be on the receiving end as a
committer.
-Grant
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
