[
https://issues.apache.org/jira/browse/NUTCH-2464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16262795#comment-16262795
]
Jorge Luis Betancourt Gonzalez commented on NUTCH-2464:
-------------------------------------------------------
I've tested this on master and I can reproduce the bug, but you reported that
this was happening on the {{2.3}} version which doesn't ship with the headings
plugin.
Using the attached example by [~wastl-nagel] the extracted metadata is:
{code}
Parse Metadata: ... h1=header with
{code}
The problem is that the method is stopping after finding the first
{{TEXT_NODE}}. So we get a truncated title in this case. One option would be to
just allow it to continue traversing the DOM tree. Your patch only needed one
more tweak:
{code}
for (int i = 0; i < children.getLength(); i++) {
if (children.item(i).getNodeType() == Node.TEXT_NODE) {
buffer.append(children.item(i).getNodeValue());
} else {
buffer.append(getNodeValue(children.item(i)));
}
}
{code}
We could move this into using the {{NodeWalker}} as suggested by [~wastl-nagel]
> Headers That Contain HTML Elements Are Not Parsed
> -------------------------------------------------
>
> Key: NUTCH-2464
> URL: https://issues.apache.org/jira/browse/NUTCH-2464
> Project: Nutch
> Issue Type: Bug
> Components: plugin
> Affects Versions: 2.3
> Environment: Internal development/test environments.
> Reporter: Cass Pallansch
> Attachments: NUTCH-2464-complex-header.html
>
>
> Nutch does not appear to traverse the HTML elements that may be contained
> within header elements (e.g., H1, H2, H3, etc. tags). Many times there are
> anchors and/or <span> tags within these elements that contain the actual text
> nodes that should be picked up as the header value for indexing purposes.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
