Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Nutch Wiki" for change 

The "Nutch2Roadmap" page has been changed by JulienNioche.


    * Storage Abstraction
      * initially with back end implementations for HBase and HDFS  
      * extend it to other storages later e.g. MySQL etc...
-   * Plugin cleanup : Tika only for parsing document formats
+   * Plugin cleanup : Tika only for parsing document formats (see
      * keep only stuff HtmlParseFilters (probably with a different API) so 
that we can post-process the DOM created in Tika from whatever original format.
    * Externalize functionalities to crawler-commons project 
      * robots handling, url filtering and url normalization, URL state 
management, perhaps deduplication. We should coordinate our efforts, and share 
code freely so that other projects (bixo, heritrix,droids) may contribute to 
this shared pool of functionality, much like Tika does for the common need of 
parsing complex formats.

Reply via email to