You will need to upgrade the HDFS cluster but it goes pretty quick this
time. Steps are:
1) Stop DFS and Map Reduce
2) Upgrade all slaves with new nutch/hadoop 0.16
3) Run bin/start-dfs.sh -upgrade
4) When finished run some MR jobs to test
5) When satisfied everything is working run bin/hadoop dfsadmin
-finalizeUpgrade
Pretty easy, took me about 20 mins to upgrade the Search Wikia cluster.
Dennis
Dennis & Co.
Is the 0.15.* -> 0.16 upgrade seamless? That is, a jar replacement and
that's it, or is there an explicit HDFS upgrade step involved?
Thanks,
Otis
--
Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch
----- Original Message ----
> From: Dennis Kubes <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Sent: Saturday, February 9, 2008 11:44:34 PM
> Subject: Re: nutch vs hadoop versions
>
> Ah, time to upgrade to 0.16 :) I will put it through the tests and some
> fetch cycles. I was seeing some weird errors with 0.17 though for
> injecting with no urls passing filtering, I want to make sure these
> aren't there for 0.16.
>
> Dennis
>
> Andrzej Bialecki wrote:
> > Kenji Kawai wrote:
> >> Does anybody know when/how the nutch would catch up with the hadoop
> >> versions? Currently the nutch trunk uses hadoop 0.15.0, and result in
> >> a runtime no-method error when run with the .17 hadoop. We need to
> >> use the 0.17.0 for our hbase applications.
> >
> > We try to upgrade Nutch to the latest official release of Hadoop soon
> > after it becomes available. Yesterday it was 0.15.3, today it's 0.16.0
> > ;) so in a couple days we will upgrade to that. 0.17 is still under
> > development, so we don't plan to upgrade Nutch to this version until
> > it's released or very close to be released.
> >
> >
>
... [show rest of quote]
