Successful, sufficiently sophisticated open source projects share one thing in common. Financial backing by commercial interests in terms of putting developers on the project or hiring top contributors and assigning the open source project to them as part of their work responsibilities. We have seen this over and over again. With Lucene being a perfect example.
1. The Microsoft .NET platform needs a search solution as part of its ecosystem of tools and technologies if it wants to compete successfully with Java and other open source technologies. 2. There are many companies that are benefiting from Lucene.net. Some of these companies are large enough to contribute developer resources to this project. Some of these companies also benefit sufficiently from the health of Lucene.net that it is in their own best interest to do so. 3. Open source has never been a significant part of the Microsoft ecosystem's DNA. However, in the recent years, Microsoft has contributed to some (while small) number of open source projects by devoting developer resources to it (jQuery) or by hiring top contributors (John Lam/Iron Ruby). 4. Until the commercial .NET community embraces Lucene.net with developer resources, the health of this project will continue to go up and down because: a. A project mainly consisting of porting code line by line from Java is not particularly sexy to most developers for them to spend nights and weekends on it without financial reward. b. If a branch is created that would .NETify the project and rewrite some logic, it will need a significant number of highly capable developers with prior search experience. Putting a group like that together without financial backing would be very difficult. In my opinion, the best way forward for this project is: a. For those on this list, who are part of a commercial entity that is benefiting from Lucene.net, to have a frank discussion about devoting developer resources to this project with their managers. b. Microsoft devote developer resources to the project and encourage other companies to join it in this effort. The Microsoft ecosystem is very much a top down world. Examples are set by Microsoft, and Microsoft has sufficient convincing power to get partners involved. Only with commercial vendor backing, we can guarantee the health of this project going forward. Thanks to you all! p.s. If a sufficiently powerful tool is found that would do most of the heavy lifting, then the above will no longer apply. On Tue, Nov 2, 2010 at 11:13 PM, Ciaran Roarty <ciaran.roa...@gmail.com>wrote: > Also, and this is a persistent theme in many people's emails, why is > there such a determined view that the search internals - the crown > jewels as described elsewhere - cannot be understood? > > It may take a long time to innovate in that area but I can't see that > it is impossible. > > Ciaran > > > > On 3 Nov 2010, at 04:06, George Aroush <geo...@aroush.net> wrote: > > > Hi Everyone, > > > > > > > > Rather than responding to each email, I will write up one response. The > > points is in no significant order or priority. > > > > > > > > 1) IKVM: Since it doesn't give you source code, you end up with Java look > > and fell, all the way from API to classes to exceptions. If this is > > valuable option for your need, you can do it with ease; you don't need > the > > support of ASF or Lucene developers. Just use IKVM and off you go. With > > this option, you are now further away from .NET'nes that's being asked of > > Lucene.Net, but all exiting Lucene resources (books, examples, support, > > etc.) is available for you and you can have a .NET version of Lucene the > day > > Java Lucene is released. > > > > > > > > 2) Other conversion tools: Using other converter tools (beside JLCA which > is > > the one I'm familiar with) should be looked at. Keep in mind that until > > when they are tried out, and their quality is analyzed, they are just > > another tools beside JLCA. In addition, since those are different tools, > > the output C# code may not be consistent with exiting Lucene.Net code. > If > > so, this will cause issue if such a change is at the public API layer; > the > > port will no longer be backward compatible (at API level) with existing > > clients. My preference is to stick with JLCA, since I'm familiar with it > > and know have written scripts to highlight where it falls short. > However, I > > would like to see others try out other tools and report back. I would be > > really surprise to see any tool doing much better than JLCA because if > such > > a tool exist, there would be many ports of other Java projects. In > another > > email, I will outline a use-case to test those other tools. > > > > > > > > 3) .NET'fying Lucene.Net: If you really want this, just start a new > project > > at ASF or someone where else. I really don't see Lucene.Net achieving > this > > anytime soon per reasons that I pointed out earlier and over the years on > > this mailing list. If you start such a project, it shouldn't be called > > Lucene.Net because that new project will produce a C# Lucene which is no > > longer compatible with existing Lucene.Net clients as the public API will > > now diverge. In addition, you will also lose, based on how deep .NET'es > you > > make your Lucene, existing available resources about Lucene (web, books, > > mailing list, etc). You will also need good knowledge of search engines, > > and the internals of Lucene to make this happen. > > > > > > > > 4) Adding a .NET'es layer: Have a look at the list of classes and APIs > > Lucene.Net has to offer (see: > > http://lucene.apache.org/lucene.net/docs/2.4.0/ -- hmm, looks like I > never > > created doc for 2.9.x). Do you plan to cover them all? Only part of it? > > Are you ready to support it? If so, you can start such a project at ASF > or > > somewhere else. > > > > > > > > 5) Support VS 2010: This is a minor issue (if an issue at all). Just > open > > the existing project and VS 2010 will ask you if you want to convert it. > > Personally, it's always best to support the lowest common compiler, > > environment and .NET Framework. This way, you can support a wider > audience > > as possible (even mono). Remember, not everyone wants the source code, > or > > can use the latest compiler or IDE, most just want the release DLL. Java > > Lucene has always supported older ver. of Java till Lucene 3.0. > > > > > > > > 6) Lucene.Net on ASF: This is a big one. Many corporation and > > organization, big and small, will use and ship ASF software over other > open > > source software with very little, if any, reservation. The license model > of > > ASF, the opens, brand reorganization and the process that ASF demands of > its > > project is well known and sound. When you grab an ASF project, which has > > gone through incubation and graduated, you know you are getting a > software > > which has been well vetted, is backed with a team that knows about the > > software, and the team will be around to back it up and support it. At > ASF, > > there is a established process which all graduated projects fallow. > > Lucene.Net, since it graduated, has NOT stood up to this level of > standard. > > Heck, there was only 1 official release back in 2006 of Lucene 1.9 which > was > > pre-graduation. This is why Grant has raised this issue, to send us back > > into incubation or attic (retire). > > > > > > > > 7) Committers: There are several committers, few are more dedicated and > > active than others. I was the initial and sole committer since 2004 > (even > > prior to that on SourceForg.net). This change since 2008 when we added > > DIGY, Doug and Michael; they all have contributed -- they took my initial > > port and cleaned up open issues. When folks are saying there is 1 > > committer, I think they mean to say there is only 1 committer who has > done > > the initial ports. Let us not forgot to give credit where it's due. > > > > > > > > 8) Not .NET'fying Lucene.Net and line-by-line port: I want to say few > > things about this even though I pointed them earlier, but just to make it > > clear. While it is true Lucene.Net doesn't have the full fell of > .NET'nes > > (it's more like the first and second generation of C#) the fact that > > Lucene.Net fallows this port model means you can post a question on Java > > Lucene mailing list and everyone will know what you are talking about. > It > > means if there is a bug in Lucene.Net, you can debug it by doing a > > side-by-side run of Java and C# Lucene (no need for deep Lucene or search > > engine expertise). It means existing Lucene resources are available for > > you. It means a bug in Java Lucene also exist in Lucene.Net. It means a > > rock solid Java Lucene is what Lucene.Net will be. > > > > > > > > 9) Back to incubation: The reason to go back to incubation is mainly to > > make sure the ASF brand that a graduated ASF project is stamped with, > holds > > to ASF's core. As is, since Lucene.Net was prompted into graduation, has > > failed on this front. As I pointed out earlier, there hasn't been any > > official release other than the one I did way back in 2006 for 1.9. > Having > > ASF to offer Lucene.Net as a "graduated" and "stable" project does > injustice > > to existing graduated ASF projects not to mention the brand ASF. > > > > > > > > 10) Comparing this project to X: You can look hard and deep to find > reason > > why Lucene.Net isn't as successful as project X. My take on it is, > unlike > > other most successful open source projects, on ASF or somewhere else, > > Lucene.Net has NO active and continues committers who actually get paid > to > > work on it. Until when we have a sponsoring entity, any cycles or effort > > spent on this project by anyone is going to be an after though even if > you > > are a dedicated user who is in need of Lucene.Net -- you will most likely > > commit a fix or a port to mainly get your need done. This is also true > for > > a sponsoring entity, but the sponsoring entity has a broader need. > > > > > > > > 11) Lucene contrib: I don't know how many folks know this, but I also > > ported a number of Java Lucene contrib codes. Check the ported list: > > https://svn.apache.org/repos/asf/lucene/lucene.net/trunk/C%23/contrib/ > > > > > > > > So where do we go from here? Unless if there are further discussions or > > questions, I suggest we put our energy and effort on getting actual > results > > done. To do so, I will start a new email thread on this subject sometime > > tomorrow. > > > > > > > > Thanks, > > > > > > > > -- George > > > -- Nima
