On Fri, 2005-11-04 at 20:41 -0800, Doug Cutting wrote:
> Rod Taylor wrote:
> > Here you go. local filesystem and a single job tracker on another
> > machine. When the tasktracker and jobtracker are on the same box there
> > isn't a problem. When they are on different machines it runs into
> > issues.
> >
> > This is using mapred.local.dir on the local machine (not sharedd between
> > sbider4 and sbider5):
>
> > parsing /home/sitesell/localt/taskTracker/task_m_o59djj/job.xml
> > [Fatal Error] :-1:-1: Premature end of file.
>
> What is mapred.system.dir? That must be shared. Also, filenames you
> pass to commands must be pathnames that work on all hosts.
Had the rest, but failed to override system.dir (description is "local
directory" which isn't really true if it is shared).
That worked through the map but failed at the reduce. Both the remote
task tracker and the task tracker on the same physical machine as the
job tracker failed.
Both had similar errors logged:
051104 235758 task_m_r2dcvc
0.6336343% /opt/sitesell/sbider_data/test/urls/list-oct31:167034415
+1758257
051104 235758 Server connection on port 45644 from 192.168.100.13:
exiting
051104 235759 task_m_r2dcvc
0.7225661% /opt/sitesell/sbider_data/test/urls/list-oct31:167034415
+1758257
051104 235800 task_m_r2dcvc
0.8255505% /opt/sitesell/sbider_data/test/urls/list-oct31:167034415
+1758257
051104 235801 task_m_r2dcvc
0.9183419% /opt/sitesell/sbider_data/test/urls/list-oct31:167034415
+1758257
051104 235802 task_m_r2dcvc
1.0% /opt/sitesell/sbider_data/test/urls/list-oct31:167034415+1758257
051104 235802 Task task_m_r2dcvc is done.
051104 235802 Server connection on port 45644 from 192.168.100.13:
exiting
java.io.FileNotFoundException:
/opt/sitesell/sbider_data/test/system/submit_fubqfe/job.xml (No such file or
directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:106)
at org.apache.nutch.fs.LocalFileSystem
$LocalNFSFileInputStream.<init>(LocalFileSystem.java:64)
at
org.apache.nutch.fs.LocalFileSystem.openRaw(LocalFileSystem.java:108)
at org.apache.nutch.fs.FileUtil.copyContents(FileUtil.java:57)
at
org.apache.nutch.fs.LocalFileSystem.copyToLocalFile(LocalFileSystem.java:297)
at org.apache.nutch.mapred.TaskTracker
$TaskInProgress.localizeTask(TaskTracker.java:328)
at org.apache.nutch.mapred.TaskTracker
$TaskInProgress.<init>(TaskTracker.java:314)
at
org.apache.nutch.mapred.TaskTracker.offerService(TaskTracker.java:214)
at org.apache.nutch.mapred.TaskTracker.run(TaskTracker.java:268)
at
org.apache.nutch.mapred.TaskTracker.main(TaskTracker.java:633)
051104 235806 Lost connection to JobTracker
[sbider5.sitebuildit.com/192.168.100.14:5464]. Retrying...
051104 235811 parsing file:/opt/nutch-0.8_7/conf/nutch-default.xml
051104 235811 parsing file:/opt/nutch-0.8_7/conf/mapred-default.xml
051104 235811
parsing /home/sitesell/local/taskTracker/task_r_mdnul7/job.xml
[Fatal Error] :-1:-1: Premature end of file.
051104 235811 SEVERE error parsing conf file:
org.xml.sax.SAXParseException: Premature end of file.
java.lang.RuntimeException: org.xml.sax.SAXParseException: Premature end
of file.
at
org.apache.nutch.util.NutchConf.loadResource(NutchConf.java:358)
at org.apache.nutch.util.NutchConf.getProps(NutchConf.java:293)
at org.apache.nutch.util.NutchConf.get(NutchConf.java:94)
at org.apache.nutch.mapred.JobConf.getJar(JobConf.java:81)
at org.apache.nutch.mapred.TaskTracker
$TaskInProgress.localizeTask(TaskTracker.java:332)
at org.apache.nutch.mapred.TaskTracker
$TaskInProgress.<init>(TaskTracker.java:314)
at
org.apache.nutch.mapred.TaskTracker.offerService(TaskTracker.java:214)
at org.apache.nutch.mapred.TaskTracker.run(TaskTracker.java:268)
at
org.apache.nutch.mapred.TaskTracker.main(TaskTracker.java:633)
Caused by: org.xml.sax.SAXParseException: Premature end of file.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown
Source)
at
javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:172)
at
org.apache.nutch.util.NutchConf.loadResource(NutchConf.java:318)
... 8 more
051104 235811 Lost connection to JobTracker
[sbider5.sitebuildit.com/192.168.100.14:5464]. Retrying...
--
Rod Taylor <[EMAIL PROTECTED]>
