Re: VOTE Apache Nutch 2.0 RC1

2012-06-15 Thread Ferdy Galema
Agree with only releasing src. On Thu, Jun 14, 2012 at 11:32 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Or just not ship a bin release at all. Src is the only thing we really VOTE on legally though bin is provided for convenience purposes. Will type more on this

Re: VOTE Apache Nutch 2.0 RC1

2012-06-15 Thread Julien Nioche
+1 On 15 June 2012 09:00, Ferdy Galema ferdy.gal...@kalooga.com wrote: Agree with only releasing src. On Thu, Jun 14, 2012 at 11:32 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Or just not ship a bin release at all. Src is the only thing we really VOTE on legally

Re: VOTE Apache Nutch 2.0 RC1

2012-06-15 Thread Lewis John Mcgibbney
I'll push this in an hour or so guys. Thanks for the input. Lewis On Fri, Jun 15, 2012 at 9:39 AM, Julien Nioche lists.digitalpeb...@gmail.com wrote: +1 On 15 June 2012 09:00, Ferdy Galema ferdy.gal...@kalooga.com wrote: Agree with only releasing src. On Thu, Jun 14, 2012 at 11:32

Re: VOTE Apache Nutch 2.0 RC1

2012-06-15 Thread Julien Nioche
Before you do, could you check that NutchGora passes ant test successfully. I just tried and got an error related to the parse-tika tests. Am about to open a JIRA to update to the latest version of Tika for NutchGora which should fix the problem and put it at the same level as trunk J On 15 June

Re: VOTE Apache Nutch 2.0 RC1

2012-06-15 Thread Julien Nioche
see https://issues.apache.org/jira/browse/NUTCH-1396 On 15 June 2012 10:43, Julien Nioche lists.digitalpeb...@gmail.com wrote: Before you do, could you check that NutchGora passes ant test successfully. I just tried and got an error related to the parse-tika tests. Am about to open a JIRA to

Re: VOTE Apache Nutch 2.0 RC1

2012-06-15 Thread Mattmann, Chris A (388J)
OK you are just making us all look bad now Juls ;) Super fast! Cheers, Chris On Jun 15, 2012, at 2:54 AM, Julien Nioche wrote: see https://issues.apache.org/jira/browse/NUTCH-1396 On 15 June 2012 10:43, Julien Nioche lists.digitalpeb...@gmail.com wrote: Before you do, could you check

Re: VOTE Apache Nutch 2.0 RC1

2012-06-15 Thread Julien Nioche
That was not intented. Just that am on holidays, it's raining and the children were either asleep or playing nicely :-) On 15 June 2012 18:19, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: OK you are just making us all look bad now Juls ;) Super fast! Cheers, Chris On

Re: VOTE Apache Nutch 2.0 RC1

2012-06-14 Thread Ferdy Galema
Maybe just 1392? I went ahead and made a patch that should fix this. Feel free to commit or ignore prior to RC2. On Thu, Jun 14, 2012 at 1:44 AM, Lewis John Mcgibbney lewis.mcgibb...@gmail.com wrote: Hi Sebastian, On Wed, Jun 13, 2012 at 11:30 PM, Sebastian Nagel wastl.na...@googlemail.com

Re: VOTE Apache Nutch 2.0 RC1

2012-06-14 Thread Sebastian Nagel
We only supply src distributions... Does this principle apply to Nutch 2 as well? Maybe, yes. The situation with the current binary package is uncomfortable: I had to copy/link gora-hbase and hbase jars into lib/ to get nutch running. 2012/6/13 Lewis John Mcgibbney lewis.mcgibb...@gmail.com

Re: VOTE Apache Nutch 2.0 RC1

2012-06-14 Thread Lewis John Mcgibbney
Aye this is no good at all. Depending on which backend you wish to use with Gora, you will need to go and manually fetch the correct .jar's from maven central. Does anyone else have either solution or a workaround before I push RC2 with just src dists? Thanks Lewis On Thu, Jun 14, 2012 at 4:52

Re: VOTE Apache Nutch 2.0 RC1

2012-06-14 Thread Mattmann, Chris A (388J)
Hey Guys, I think the annoyance is probably something folks can live with as they have been waiting for an official release of 2.x for years :) My +1 to roll RC #2 with or without a solution to this and mark it as a TODO. release eary, release often :) Cheers, Chris On Jun 14, 2012, at 10:04

Re: VOTE Apache Nutch 2.0 RC1

2012-06-14 Thread Julien Nioche
I disagree. You'd expect a binary release to work out of the box - which is not the case. Plus we'd have to spend more time explaining the workaround, answering the same questions over and over on the ML etc... Fixing this should not be a big deal (i.e. add the gore-x modules for the backends to

Re: VOTE Apache Nutch 2.0 RC1

2012-06-14 Thread Lewis John Mcgibbney
Hi Julien, Do you suggest with the binary release that we simply open up all gora-* deps and ship it with every jar available? Lewis On Thu, Jun 14, 2012 at 9:39 PM, Julien Nioche lists.digitalpeb...@gmail.com wrote: I disagree. You'd expect a binary release to work out of the box - which

Re: VOTE Apache Nutch 2.0 RC1

2012-06-14 Thread Julien Nioche
yep, remember that you can't build from the bin package so inevitably someone will wonder why only such or such backend is available etc... another option is to NOT have a binary release at all, in which case it is acceptable I think not to include the deps in ivy. Maybe we should at least add

Re: VOTE Apache Nutch 2.0 RC1

2012-06-14 Thread Lewis John Mcgibbney
This is what is currently done and what I was essentially proposing. I really don't know about the size of the bin artifact if we enable all gora-* dependencies before packaging it for distribution... thanks to input from yourselves we recently sorted out some size issues with 1.5, it would be

Re: VOTE Apache Nutch 2.0 RC1

2012-06-14 Thread Mattmann, Chris A (388J)
Or just not ship a bin release at all. Src is the only thing we really VOTE on legally though bin is provided for convenience purposes. Will type more on this later... Sent from my iPhone On Jun 14, 2012, at 2:18 PM, Lewis John Mcgibbney

Re: VOTE Apache Nutch 2.0 RC1

2012-06-13 Thread Ferdy Galema
Findings about Nutch-2.0 RC 1. The Nutch job jar is not present in the binary archive. This means distributed running of jobs is not supported. I'm not sure if this is a problem (since users can always build one themselves), merely pointing it out. The recently released 1.5 also lacks this job

Re: VOTE Apache Nutch 2.0 RC1

2012-06-13 Thread Ferdy Galema
Hmm please ignore the parse text limited to 100 chars, this is actually not the case. (Only in our branch that has a fix for limiting anchor texts; not yet present in in the nutchgora branch because it still needs polishing). So no need to wait for commits on my part. On Wed, Jun 13, 2012 at

Re: VOTE Apache Nutch 2.0 RC1

2012-06-13 Thread Lewis John Mcgibbney
Hi Seb, As Chris said, the issues you highlight well justify another RC. I can shift it by the end of play today. Thanks very much for having a look through guys Lewis On Tue, Jun 12, 2012 at 11:33 PM, Sebastian Nagel wastl.na...@googlemail.com wrote: Hi Lewis, my first steps with 2.0 (to

Re: VOTE Apache Nutch 2.0 RC1

2012-06-13 Thread Lewis John Mcgibbney
Hi Seb, Quick update On Tue, Jun 12, 2012 at 11:33 PM, Sebastian Nagel wastl.na...@googlemail.com wrote: 1 some guidance would be nice. README.txt points to http://wiki.apache.org/nutch/NutchTutorial which refers to 1.x Please see http://wiki.apache.org/nutch/Nutch2Tutorial which is an update

Re: VOTE Apache Nutch 2.0 RC1

2012-06-13 Thread Julien Nioche
Ferdy The Nutch job jar is not present in the binary archive. This means distributed running of jobs is not supported. I'm not sure if this is a problem (since users can always build one themselves), merely pointing it out. The recently released 1.5 also lacks this job jar, so at least no

Re: VOTE Apache Nutch 2.0 RC1

2012-06-13 Thread Lewis John Mcgibbney
Hi Guys, Whilst updating the Nutch2Tutorial I got thinking that within Gora we don't supply binary distributions of the code, this is because when using Gora a user may wish/require to recompile the code to accomodate config changes etc. We only supply src distributions... Does this principle

Re: VOTE Apache Nutch 2.0 RC1

2012-06-13 Thread Sebastian Nagel
Hi Lewis, Please see http://wiki.apache.org/nutch/Nutch2Tutorial which is an update of Julien's (I think) page on GORA_HBase. Thsi will get you rocking with HBase. The changes between Cassandra, Accumulo and the other data stores are fairly trivial. I'll managed to perform a crawl with 2.0

Re: VOTE Apache Nutch 2.0 RC1

2012-06-13 Thread Lewis John Mcgibbney
Hi Sebastian, On Wed, Jun 13, 2012 at 11:30 PM, Sebastian Nagel wastl.na...@googlemail.com wrote: I'll managed to perform a crawl with 2.0 and HBase: it rocks, indeed. Much simpler than 1.x (no segments!). :0) % ./bin/nutch readdb -stats WebTable statistics start WebTableReader:

Re: VOTE Apache Nutch 2.0 RC1

2012-06-12 Thread Lewis John Mcgibbney
Hi Everyone, I appreciate that most of the core dev's are using trunk, however I would appeal to you guys to at least check out the artifacts and check sigs, tests, license headers if possible. Although this does not fully satisfy the requirements of a thoroughly reviewed RC, hopefully the

Re: VOTE Apache Nutch 2.0 RC1

2012-06-12 Thread Mattmann, Chris A (388J)
Hey Lewis, I will get to this tonight, for sure. Thanks! Cheers, Chris On Jun 12, 2012, at 1:16 PM, Lewis John Mcgibbney wrote: Hi Everyone, I appreciate that most of the core dev's are using trunk, however I would appeal to you guys to at least check out the artifacts and check sigs,

Re: VOTE Apache Nutch 2.0 RC1

2012-06-12 Thread Lewis John Mcgibbney
Thank you On Tue, Jun 12, 2012 at 9:19 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Lewis, I will get to this tonight, for sure. Thanks! Cheers, Chris On Jun 12, 2012, at 1:16 PM, Lewis John Mcgibbney wrote: Hi Everyone, I appreciate that most of the core

Re: VOTE Apache Nutch 2.0 RC1

2012-06-12 Thread Sebastian Nagel
Hi Lewis, my first steps with 2.0 (to be continued, still struggling). Two points (I'll try to give a final vote tomorrow): 1 some guidance would be nice. README.txt points to http://wiki.apache.org/nutch/NutchTutorial which refers to 1.x (I'm using

Re: VOTE Apache Nutch 2.0 RC1

2012-06-12 Thread Mattmann, Chris A (388J)
Hey Guys, #2 is probably reason enough for a respin. Lewis if you don't have time to do it before Thursday, I could probably give it a whack. Let me know. Cheers, Chris On Jun 12, 2012, at 3:33 PM, Sebastian Nagel wrote: Hi Lewis, my first steps with 2.0 (to be continued, still

VOTE Apache Nutch 2.0 RC1

2012-06-08 Thread lewis john mcgibbney
Good Evening Everyone, A candidate for the Apache Nutch 2.0 RC1 is available at: http://people.apache.org/~lewismc/nutch-2.0 The release candidate is a src.zip, bin.zip, src.tar.gz and bin.tar.gz archive of the sources in: http://svn.apache.org/repos/asf/nutch/tags/release-2.0rc1 Further, a