Do as much as poaaible offline make much sense from my point of view.
We should make it easier to add new fields that are stored with each page in the database. We could simply add an attribute-value list, but that could get large and slow database updates considerably. What would be better is to have a registry per database of field names and types, so that field names can be written as single bytes and values can be written untyped. I'll think a bit more about how to implement this...
I know this is very hard but i would really love to see if its possible to update the database later and set by step for each meta data.
Do you remeber my question about this 1:n thing? Whats about if we just leave our link and page / index database as it is and run for each meta data field a own database?
So we still have the fast page, link and index db, but can do the slow prossess like whois queries in the baclround. That would have the advantage in case we deside to add a new meta data field to our db, that we do not need to rebuild a already existing db, since we can just "extend" it. Realtions can be habdled about unique keys for each entry in the dbs.
Make that sense?
Stefan
--------------------------------------------------------------- enterprise information technology consultanting open technology: http://www.media-style.com open source: http://www.weta-group.net open discussion: http://www.text-mining.org
------------------------------------------------------- This SF.Net email is sponsored by: GNOME Foundation Hackers Unite! GUADEC: The world's #1 Open Source Desktop Event. GNOME Users and Developers European Conference, 28-30th June in Norway http://2004/guadec.org _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
