Firstly, jira seems to be down so I'm replying by email again, I'll update jira when it's back up.
I see your point and I see the code for it (339-350 right?) but I'm still getting the error listed below, which disapears when I apply the patch. I guess the problem must be in the adding of new Writables to the map. WARN crawl.MapWritable: Unable to load meta data entry, ignoring.. : java.io.IOException: unable to load class for id: 36 On 7/25/07, Doğacan Güney <[EMAIL PROTECTED]> wrote: > On 7/25/07, Robert Young <[EMAIL PROTECTED]> wrote: > > The message which was appearing in the logs is pasted below. > > > > Basically, in org.apache.nutch.crawl.MapWritable#getKeyValueEntry the > > Writable is instantiated. It's class is determined by a two byte code > > (which is written to crawldb I guess), if there is no entry for the > > class it fails to create it, regardless of if it's a Writable. You're > > right in that it can potentially handle any Writabl object, but only > > if it has a maping for it's class. > > If you add a writable that does not have a mapping, MapWritable > automatically creates it and then stores the mapping internally. When > a MapWritable is written, any new mapping is also written (as a byte > and the corresponding class name). So, when you read a MapWritable it > first reads all the mappings (that are not already statically defined) > then proceeds to reading the <Writable,Writable> map. So I think your > problem is caused by something else (perhaps, there is a bug in > MapWritable's implementation but that is what that code is trying to > do). > > Also, replying to JIRA generated emails does not add comments to > issues (despite what the email is saying). So please use JIRA to > reply. > > > > > Cheers > > Rob > > > > 07/07/25 11:52:00 WARN crawl.MapWritable: Unable to load meta data > > entry, ignoring.. : java.io.IOException: unable to load class for id: > > 36 > > > > On 7/25/07, Doğacan Güney (JIRA) <[EMAIL PROTECTED]> wrote: > > > > > > [ > > > https://issues.apache.org/jira/browse/NUTCH-527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12515283 > > > ] > > > > > > Doğacan Güney commented on NUTCH-527: > > > ------------------------------------- > > > > > > What was the error you were having? MapWritable supports reading and > > > writing *all* writables. The ones defined at the top of the file are an > > > optimization and shouldn't affect correctness (basically mapwritable > > > first writes a byte and the associated classname, then writes that byte > > > to indicate classname everywhere else. For commonly used types we > > > statically define the association so that "first write the byte then the > > > classname" phase is not necessary). > > > > > > > MapWritable doesn't support all hadoops writable types > > > > ------------------------------------------------------ > > > > > > > > Key: NUTCH-527 > > > > URL: https://issues.apache.org/jira/browse/NUTCH-527 > > > > Project: Nutch > > > > Issue Type: Bug > > > > Affects Versions: 0.9.0 > > > > Environment: Tested on Solaris and Windows with Java 1.5 > > > > Reporter: Rob Young > > > > Attachments: mapwritable.patch > > > > > > > > > > > > The map of classes which implement org.apache.hadoop.io.Writable is not > > > > complete. It does not, for example, include > > > > org.apache.hadoop.io.BooleanWritable. I would happily provide a patch > > > > if someone would explain what the Byte parameter is. > > > > > > -- > > > This message is automatically generated by JIRA. > > > - > > > You can reply to this email to add a comment to the issue online. > > > > > > > > > > > -- > Doğacan Güney > ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ _______________________________________________ Nutch-developers mailing list Nutch-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nutch-developers
