[
https://issues.apache.org/jira/browse/NUTCH-1870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sebastian Nagel updated NUTCH-1870:
-----------------------------------
Attachment: NUTCH-1870-trunk-v4.patch
New patch including:
* load all configuration files from class path
* load once initially via setConf(conf)
* made {{javax.xml.transform.Transformer}} thread-local
* renamed ParseTechnicalTests to make them run by ant (-> TestParseTechnical)
Further / open points:
* verified (I was wrong in my last comment): it's possible to fill fields with
multiple values if an expression matches more than once
* in the transformer file (cf. src/plugin/parse-xsl/: conf/documents.xsd resp.
sample/sample1/transformer_book.xsl) there is "documents" node containing an
open number of "document" nodes. This could also be used to (optionally) add
multiple "subdocuments" (cf. NUTCH-443).
* selection of root node via XPath "html" has one disadvantage: it makes it
impossible to use parse-xsl for make pure XML documents indexable and --
extract documents and fields (key-value pairs) from arbitrary XML. Also the
special treatment of Neko (parse-html) and tagsoup (parse-tika) via
lower/upper-cased tag names could be needless after NUTCH-1592.
* there is still some extra debugging code (saveDOMOutput(),
displayMemoryUsage, timing) which could be moved to some general utility
classes, it could be useful also somewhere else.
* ev. we should simplify how the parsers are called in AbstractCrawlTest: just
call ParseUtil.parse(). This would also make the parse-xsl tests sensitive to
changes in parse-html (or parse-tika).
> Generic xsl parser plugin
> -------------------------
>
> Key: NUTCH-1870
> URL: https://issues.apache.org/jira/browse/NUTCH-1870
> Project: Nutch
> Issue Type: New Feature
> Components: indexer, parser
> Affects Versions: 1.9
> Reporter: Albinscode
> Fix For: 1.11
>
> Attachments: NUTCH-1870-trunk-v3.patch, NUTCH-1870-trunk-v4.patch,
> nutch-site.xml, xsl-parse-plugin.patch, xsl-parse-plugin2.patch
>
>
> The aim of this plugin is to use XSLT to extract metadata from HTML DOM
> structures.
> | Your Data | --> | Parse-html plugin or TIKA plugin | --> | DOM structure |
> --> |XSLT plugin |
>
>
> The main advantage is that:
> - You won't have to produce any java code, only XSLT and configuration
> - It can process DOM structure from DocumentFragment (@see NekoHtml and @see
> TagSoup)
> - It is HtmlParseFilter plugin compatible and can be plugged as any other
> plugin (parse-js, parse-swf, etc...)
> This topic has been discussed on
> http://www.mail-archive.com/dev%40nutch.apache.org/msg15257.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
