[
https://issues.apache.org/jira/browse/NUTCH-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14165632#comment-14165632
]
Sebastian Nagel commented on NUTCH-1870:
----------------------------------------
Thanks, [~Albinscode], for the patch. Looks nice, code is well formatted, ...
I'll continue testing, but a few first comments:
* could load resources in setConf(conf) and not do it on-demand in the "filter"
method:
** setConf() is called early, so failures in reading configuration resources
are reported soon
** filter() may be called concurrently because for every plugin only one
instance is hold per extension point
* thread-safety: the filter() method must be thread-safe, and so must be all
used object instances.
[Transformer|http://docs.oracle.com/javase/7/docs/api/javax/xml/transform/Transformer.html]
instances are not safe and may not be shared by threads. That's also true for
other DOM/XML related classes, cf.
[1|http://mail-archives.apache.org/mod_mbox/nutch-user/201301.mbox/%[email protected]%3E],
[2|http://mail-archives.apache.org/mod_mbox/nutch-user/201301.mbox/%[email protected]%3E],
or NUTCH-1596. Possible solutions are, e.g., to make this variables local or
[thread
local|http://docs.oracle.com/javase/7/docs/api/java/lang/ThreadLocal.html].
> there are some unit tests strongly related to sites I'm crawling
* would be better to take sample pages where we are sure not to violate any
copyright
> Generic xsl parser plugin
> -------------------------
>
> Key: NUTCH-1870
> URL: https://issues.apache.org/jira/browse/NUTCH-1870
> Project: Nutch
> Issue Type: New Feature
> Components: indexer, parser
> Affects Versions: 1.9
> Reporter: Albinscode
> Fix For: 1.10
>
> Attachments: xsl-parse-plugin.patch
>
>
> The aim of this plugin is to use XSLT to extract metadata from HTML DOM
> structures.
> | Your Data | --> | Parse-html plugin or TIKA plugin | --> | DOM structure |
> --> |XSLT plugin |
>
>
> The main advantage is that:
> - You won't have to produce any java code, only XSLT and configuration
> - It can process DOM structure from DocumentFragment (@see NekoHtml and @see
> TagSoup)
> - It is HtmlParseFilter plugin compatible and can be plugged as any other
> plugin (parse-js, parse-swf, etc...)
> This topic has been discussed on
> http://www.mail-archive.com/dev%40nutch.apache.org/msg15257.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
