John,
sorry I have to comment, but I feel here some substantial missconceptions abot
Open Source
1)
"e.g. >30 million documents indexed and searched in realtime., and I really had
to do some tweaking."
So what? What I or anyone else has to do with it? "some tweaking" is definitely
better than making everything from the scratch or going to commercial
vendors... no?
2)
"what would make it better is some transparency on how patches/issues
are evaluated to be committed. At least seemed from the outside, it is
purely being decided on by the committers, and since my understanding
is that an open source project belongs to the public, the public user
base should have some say."
Transparency, Jira + this mailing list. Everybody is allowed to express an
opinion, *even committers* , weather you like it or not is just another
question. If you put up convincing arguments, be assured even committers can
change opinions.
Imo, it does not go much more transparent than that.
Sure it belongs to public, you do not have to pay for it, read ASF Licence. If
you have better proposal on how to organize Open Source projects, speak-up. I
do not know how we could ever avoid committers having final say on things
without provoking haos?
3) "Would groups/companies that pay for consulting service get their
patches/requirements committed with higher priority?"
Sure, of course, *even commmercial users are parts of the comunity* and we
schould be greatful that they contribute and commit ther resouces so that
others can benefit from it. Think again about it, there is absolutly nothing
bad behind it, no conspiracy.
Just one example on micro scale. I had an itch and had to do some "tweaking",
my customer(comercial) had nothing against contributing back to Lucene, so I
did it. I get my money and I give something back to the comunity. End result, I
am happy, Lucene gets better and everybody profits a bit from it.
Should I have problems with my consciones? I do not think so.
Conflict of interests, no, that is rather evolution. What do you think why
commiters work on Lucene, do you honestly beleive they have no families to feed
and just sit and wait someone feeds them with proposals for nice features?
Commiters as well as everybody else here have their own, private agendas,
goals, ideas, needs ... and all these things get somehow conflated into Lucene.
Back to my example, I was lucky that a few commiters shared my opinion about
usfulness and the priority of this patch, it could have been different. If all
commiters were busy with private agenda and had higher priorities at that
moment, well, that would habe been bad luck for me. No hard feelings even in
that case, why should I expect someone puts my itch as their priority.
Cheers, eks
________________________________
From: John Wang <[EMAIL PROTECTED]>
To: java-dev@lucene.apache.org
Sent: Thursday, 4 December, 2008 6:36:28
Subject: Re: [jira] Commented: (LUCENE-1473) Implement Externalizable in main
top level searcher classes
Grant:
I am sorry that I disagree with some points:
1) "I think it's a sign that Lucene is pretty stable." - While lucene is a
great project, especially with 2.x releases, great improvements are made, but
do we really have a clear picture on how lucene is being used and deployed.
While lucene works great running as a vanilla search library, when pushed to
limits, one needs to "hack" into lucene to make certain things work. If 90% of
the user base use it to build small indexes and using the vanilla api, and the
other 10% is really stressing both on the scalability and api side and are
running into issues, would you still say: "running well for 90% of the users,
therefore it is stable or extensible"? I think it is unfair to the project
itself to be measured by the vanilla use-case. I have done couple of large
deployments, e.g. >30 million documents indexed and searched in realtime., and
I really had to do some tweaking.
2) "You want stuff committed, keep it up to date, make it manageable to review,
document it, respond to questions/concerns with answers as best you can. " - To
some degree I would hope it depends on what the issue is, e.g. enforcing such
process on a one-line null check seems to be an overkill. I agree with the
process itself, what would make it better is some transparency on how
patches/issues are evaluated to be committed. At least seemed from the outside,
it is purely being decided on by the committers, and since my understanding is
that an open source project belongs to the public, the public user base should
have some say.
3) which brings me to this point: "I personally, would love to work on Lucene
all day every day as I have a lot of things I'd love to engage the community
on, but the fact is I'm not paid to do that, so I give what I can when I can.
I know most of the other committers are that way too." - Is this really true?
Isn't a large part of the committer base also a part of the for-profit,
consulting business, e.g. Lucid? Would groups/companies that pay for consulting
service get their patches/requirements committed with higher priority? If so,
seems to me to be a conflict of interest there.
4) "Lather, rinse, repeat. Next thing you know, you'll be on the receiving
end as a committer." - While I agree that being a committer is a great honor
and many committers are awesome, but assuming everyone would want to be a
committer is a little presumptuous.
In conclusion, I hope I didn't unleash any wrath from the committers for
expressing candor.
-John
On Wed, Dec 3, 2008 at 2:52 PM, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
On Dec 3, 2008, at 2:27 PM, Jason Rutherglen (JIRA) wrote:
Hoss wrote: "sort of mythical "Lucene powerhouse"
Lucene seems to run itself quite differently than other open source Java
projects. Perhaps it would be good to spell out the reasons for the reluctance
to move ahead with features that developers work on, that work, but do not go
in. The developer contributions seem to be quite low right now, especially
compared to neighbor projects such as Hadoop. Is this because fewer people are
using Lucene? Or is it due to the reluctance to work with the developer
community? Unfortunately the perception in the eyes of some people who work on
search related projects it is the latter.
Or, could it be that Hadoop is relatively new and in vogue at the moment, very
malleable and buggy(?) and has a HUGE corporate sponsor who dedicates lots of
resources to it on a full time basis, whilst Lucene has been around in the ASF
for 7+ years (and 12+ years total) and has a really large install base and thus
must move more deliberately and basically has 1 person who gets to work on it
full time while the rest of us pretty much volunteer? That's not an excuse,
it's just the way it is. I personally, would love to work on Lucene all day
every day as I have a lot of things I'd love to engage the community on, but
the fact is I'm not paid to do that, so I give what I can when I can. I know
most of the other committers are that way too.
Thus, I don't think any one of us has a reluctance to move ahead with features
or bug fixes. Looking at CHANGES.txt, I see a lot of contributors. Looking
at java-dev and JIRA, I see lots of engagement with the community. Is it near
the historical high for traffic, no it's not, but that isn't necessarily a bad
thing. I think it's a sign that Lucene is pretty stable.
What we do have a reluctance for are patches that don't have tests (i.e. this
one), patches that massively change Lucene APIs in non-trivial ways or break
back compatibility or are not kept up to date. Are we perfect? Of course not.
I, personally, would love for there to be a way that helps us process a larger
volume of patches (note, I didn't say commit a larger volume). Hadoop's
automated patch tester would be a huge start in that, but at the end of the
day, Lucene still works the way all ASF projects do: via meritocracy and
volunteerism. You want stuff committed, keep it up to date, make it
manageable to review, document it, respond to questions/concerns with answers
as best you can. To that end, a real simple question can go a long way and
getting something committed, and it simply is: "Hey Lucener's, what else can
I do to help you review and commit LUCENE-XXXX?" Lather, rinse, repeat. Next
thing you know, you'll be on the receiving end
as a committer.
-Grant
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
