The message which was appearing in the logs is pasted below. Basically, in org.apache.nutch.crawl.MapWritable#getKeyValueEntry the Writable is instantiated. It's class is determined by a two byte code (which is written to crawldb I guess), if there is no entry for the class it fails to create it, regardless of if it's a Writable. You're right in that it can potentially handle any Writabl object, but only if it has a maping for it's class.
Cheers Rob 07/07/25 11:52:00 WARN crawl.MapWritable: Unable to load meta data entry, ignoring.. : java.io.IOException: unable to load class for id: 36 On 7/25/07, Doğacan Güney (JIRA) <[EMAIL PROTECTED]> wrote: > > [ > https://issues.apache.org/jira/browse/NUTCH-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12515283 > ] > > Doğacan Güney commented on NUTCH-527: > ------------------------------------- > > What was the error you were having? MapWritable supports reading and writing > *all* writables. The ones defined at the top of the file are an optimization > and shouldn't affect correctness (basically mapwritable first writes a byte > and the associated classname, then writes that byte to indicate classname > everywhere else. For commonly used types we statically define the association > so that "first write the byte then the classname" phase is not necessary). > > > MapWritable doesn't support all hadoops writable types > > ------------------------------------------------------ > > > > Key: NUTCH-527 > > URL: https://issues.apache.org/jira/browse/NUTCH-527 > > Project: Nutch > > Issue Type: Bug > > Affects Versions: 0.9.0 > > Environment: Tested on Solaris and Windows with Java 1.5 > > Reporter: Rob Young > > Attachments: mapwritable.patch > > > > > > The map of classes which implement org.apache.hadoop.io.Writable is not > > complete. It does not, for example, include > > org.apache.hadoop.io.BooleanWritable. I would happily provide a patch if > > someone would explain what the Byte parameter is. > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > > ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Nutch-developers mailing list Nutch-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nutch-developers
