> > > Now that you mention upgrade solutions from 1.x to 2.0 I suggest that we > open > > a JIRA to discuss this. IMHO we probably don't want to keep the 'old' > code in > > src/java when we merge but could have the code for the conversion > utilities > > and the Nutch 1.x jars in a the contrib/ directory instead. > > I wouldn't favor a Nutch contrib going forward. Contribs lead to > umbrella-projects which Apache is moving away from b/c it typically creates > different committer lists (those who can commit to contrib and those with > commit privs to the full source code base, etc.), different lifecycles and > ultimately incubates/grows mini-projects within larger ones. >
I meant putting the migration code and 1.x Nutch jars in the contrib directory of the trunk - that shouldn't require a different committers list or should it? > > If someone needs Nutch 1.x jars they can grab them from the Apache distros > or we can publish them to Maven central. As for conversion and removal of > src/java, I'm not sure I get that? Why should we remove src/java? Merge > means "adapt existing" rather than "replace". > I was talking about removing deprecated Nutch objects (old Writables which we needed for storing things in Hadoop MapFiles) from the src after the merge once they are not used by Nutch2.0. The point made by Dogacan was that they would be needed if we want to provide conversion tools so that people could convert their old crawldbs and segments into our shiny new Gora-based architecture. > > > >> > >> Also, I realize that I am the last person to talk about this, but can we > get > >> some reviews for these changes? > > > > I could have filed a JIRA for the branch NutchBase indeed (but haven't). > > Again, NutchBase is a transitional / test / development repository before > we > > merge things into trunk. Changes to the trunk are made properly i.e. > through > > JIRA with patches and peer review. Or maybe I should indeed open a JIRA > for > > NutchBase every time I do a bit of cleanup or port new plugins to the 2.0 > API? > > Nah, IMHO I think it's OK to muck around in the branch, so long as when the > branch gets merged (incrementally rather than wholesale), we can review > those. So, the way it would work is this: > > A. branch cleaned up, SVN commits, etc., stable working > B. at some point, branch ready to be merged (assumption: branch devel > stops) > C. define branch merge into 3-5 patches > D. foreach patch in C: > create JIRA issue for patch > call for review of patch > if no objections, then commit in 24-48 hours > > E. trunk now ready for 2.0 development > F. schedule current open issues for 2.0, grab any low hanging fruit (1-2 > days) > G. all other issues pushed out to 2.1 > H. release 2.0 > > Andrzej and myself are in the process of porting the last missing tests in NutchBase and debugging Gora along the way. There is just a handful of plugins which have not been ported and I should have finished that pretty quickly. Hopefully we'll get to (A) soonish and can then follow the plan above. However we still need to address the issue raise by Dogacan i.e shall we provide tools to convert from 1.x structures to 2.0 and if so how shall we organise it. Again - some things have been removed fom NutchBase for the sake of clarity but since they are in the trunk they are not lost and we can decide what to do with them later. J. -- DigitalPebble Ltd Open Source Solutions for Text Engineering http://www.digitalpebble.com
