Hi Dennis, 1) I've initially tried to run on my existing DFS and it didn't work. I then made a backup of my DFS and performed a format and it still didn't work...
2) I'm using: java version "1.6.0_0" OpenJDK Runtime Environment (IcedTea6 1.4.1) (6b14-1.4.1-0ubuntu12) OpenJDK Client VM (build 14.0-b08, mixed mode, sharing) 3) My environment variables: ORBIT_SOCKETDIR=/tmp/orbit-eran SSH_AGENT_PID=3533 GPG_AGENT_INFO=/tmp/seahorse-Gq6lRI/S.gpg-agent:3557:1 TERM=xterm SHELL=/bin/bash XDG_SESSION_COOKIE=1a02c2275727547fa7209ad54a91276c-1260199857.905267-2000911890 GTK_RC_FILES=/etc/gtk/gtkrc:/home/eran/.gtkrc-1.2-gnome2 WINDOWID=54653392 GTK_MODULES=canberra-gtk-module USER=eran LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.svgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36: GNOME_KEYRING_SOCKET=/tmp/keyring-0Vt0yu/socket SSH_AUTH_SOCK=/tmp/keyring-0Vt0yu/socket.ssh SESSION_MANAGER=local/eran:/tmp/.ICE-unix/3387 USERNAME=eran DESKTOP_SESSION=default PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games GDM_XSERVER_LOCATION=local PWD=/home/eran JAVA_HOME=/usr/lib/jvm/default-java/ LANG=en_US.UTF-8 GDM_LANG=en_US.UTF-8 GDMSESSION=default HISTCONTROL=ignoreboth SHLVL=1 HOME=/home/eran GNOME_DESKTOP_SESSION_ID=this-is-deprecated LOGNAME=eran XDG_DATA_DIRS=/usr/local/share/:/usr/share/:/usr/share/gdm/ DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-E4IJ0hMrD8,guid=c3caaf3e590c65a58904ca7f4b1d1fb3 LESSOPEN=| /usr/bin/lesspipe %s WINDOWPATH=7 DISPLAY=:0.0 LESSCLOSE=/usr/bin/lesspipe %s %s XAUTHORITY=/home/eran/.Xauthority COLORTERM=gnome-terminal _=/usr/bin/printenv Thanks, Eran On Wed, Dec 9, 2009 at 2:38 PM, Dennis Kubes <[email protected]> wrote: > 1) Is this a new or existing Hadoop cluster? > 2) What Java version are you using and what is your environment? > > Dennis > > > Eran Zinman wrote: > >> Hi, >> >> Running new Nutch version status: >> >> 1. Nutch runs perfectly if Hadoop is disabled (i.e. running in normal >> mode). >> 2. Nutch doesn't work when I setup it to work with Hadoop either in a >> single >> or cluster setup. >> >> *I'm getting an exception: * >> ERROR namenode.NameNode - java.lang.SecurityException: sealing violation: >> can't seal package org.mortbay.util: already loaded >> >> I thought it might be a good idea that I'll attach my Hadoop conf files, >> so >> here they are: >> >> *core-site.xml* >> <configuration> >> <property> >> <name>fs.default.name</name> >> <value>hdfs://10.0.0.2:9000/</value> >> <description> >> The name of the default file system. Either the literal string >> "local" or a host:port for NDFS. >> </description> >> </property> >> </configuration> >> >> *mapred-site.xml* >> <configuration> >> <property> >> <name>mapred.job.tracker</name> >> <value>10.0.0.2:9001</value> >> <description> >> The host and port that the MapReduce job tracker runs at. If >> "local", then jobs are run in-process as a single map and >> reduce task. >> </description> >> </property> >> >> <property> >> <name>mapred.system.dir</name> >> <value>/my_crawler/filesystem/mapreduce/system</value> >> </property> >> >> <property> >> <name>mapred.local.dir</name> >> <value>/my_crawler/filesystem/mapreduce/local</value> >> </property> >> </configuration> >> >> *hdfs-site.xml* >> <configuration> >> <property> >> <name>dfs.name.dir</name> >> <value>/my_crawler/filesystem/name</value> >> </property> >> >> <property> >> <name>dfs.data.dir</name> >> <value>/my_crawler/filesystem/data</value> >> </property> >> >> <property> >> <name>dfs.replication</name> >> <value>2</value> >> </property> >> </configuration> >> >> Thanks, >> Eran >> >> On Wed, Dec 9, 2009 at 12:22 PM, Eran Zinman <[email protected]> wrote: >> >> Hi Andrzej, >>> >>> Thanks for your help (as always). >>> >>> Still getting same exception when running on standalone Hadoop cluster. >>> Getting same exceptions as before - also in the datanode log I'm >>> getting: >>> >>> 2009-12-09 12:20:37,805 ERROR datanode.DataNode - java.io.IOException: >>> Call >>> to 10.0.0.2:9000 failed on local exception: java.io.IOException: >>> Connection reset by peer >>> at org.apache.hadoop.ipc.Client.wrapException(Client.java:774) >>> at org.apache.hadoop.ipc.Client.call(Client.java:742) >>> at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220) >>> at $Proxy4.getProtocolVersion(Unknown Source) >>> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359) >>> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346) >>> at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383) >>> at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314) >>> at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291) >>> at >>> >>> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:269) >>> at >>> org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216) >>> at >>> >>> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283) >>> at >>> >>> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238) >>> at >>> >>> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246) >>> at >>> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368) >>> Caused by: java.io.IOException: Connection reset by peer >>> at sun.nio.ch.FileDispatcher.read0(Native Method) >>> at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:21) >>> at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:233) >>> at sun.nio.ch.IOUtil.read(IOUtil.java:206) >>> at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:236) >>> at >>> >>> org.apache.hadoop.net.SocketInputStream$Reader.performIO(SocketInputStream.java:55) >>> at >>> >>> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142) >>> at >>> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155) >>> at >>> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128) >>> at java.io.FilterInputStream.read(FilterInputStream.java:116) >>> at >>> >>> org.apache.hadoop.ipc.Client$Connection$PingInputStream.read(Client.java:276) >>> at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) >>> at java.io.BufferedInputStream.read(BufferedInputStream.java:237) >>> at java.io.DataInputStream.readInt(DataInputStream.java:370) >>> at >>> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:501) >>> at org.apache.hadoop.ipc.Client$Connection.run(Client.java:446) >>> >>> Thanks, >>> Eran >>> >>> >>> On Wed, Dec 9, 2009 at 12:12 PM, Andrzej Bialecki <[email protected]> wrote: >>> >>> Eran Zinman wrote: >>>> >>>> Hi, >>>>> >>>>> Sorry to bother you guys again, but it seems that no matter what I do I >>>>> can't run the new version of Nutch with Hadoop 0.20. >>>>> >>>>> I am getting the following exceptions in my logs when I execute >>>>> bin/start-all.sh >>>>> >>>>> Do you use the scripts in place, i.e. without deploying the nutch*.job >>>> to >>>> a separate Hadoop cluster? Could you please try it with a standalone >>>> Hadoop >>>> cluster (even if it's a pseudo-distributed, i.e. single node)? >>>> >>>> >>>> -- >>>> Best regards, >>>> Andrzej Bialecki <>< >>>> ___. ___ ___ ___ _ _ __________________________________ >>>> [__ || __|__/|__||\/| Information Retrieval, Semantic Web >>>> ___|||__|| \| || | Embedded Unix, System Integration >>>> http://www.sigram.com Contact: info at sigram dot com >>>> >>>> >>>> >>
