[
https://issues.apache.org/jira/browse/NUTCH-855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896223#action_12896223
]
Scott Gonyea edited comment on NUTCH-855 at 8/7/10 4:52 AM:
------------------------------------------------------------
FYI for anyone who might use this:
The "urlmeta.tags" must be comma-delimited, with no white-space to pad the
boundaries.
was (Author: sgonyea):
Let it be known, to anyone who uses this:
The "urlmeta.tags" must be comma-delimited, with no white-space to pad the
boundaries. Why?
http://svn.apache.org/viewvc/lucene/hadoop/trunk/src/java/org/apache/hadoop/conf/Configuration.java?annotate=394984&pathrev=394984
I have no damn clue. After having 1.5 days wasted, trying to figure out why my
metatags were not working... I finally found the answer. I originally tested
against a hadoop-0.21 build, not thinking that white-space would trash my
numerous days worth of time.
I seriously spent the last 2-hours wondering who it was that I hated so much,
for not thinking ahead... Then I saw that commit, where it used to split based
upon commas and bounded white-space... and vanished for no stated reason. So
odd. This was my pedantic deed for the month.
Grrrr, Doug Cutting!
> ScoringFilter and IndexingFilter: To allow for the propagation of URL
> Metatags and their subsequent indexing.
> -------------------------------------------------------------------------------------------------------------
>
> Key: NUTCH-855
> URL: https://issues.apache.org/jira/browse/NUTCH-855
> Project: Nutch
> Issue Type: New Feature
> Components: generator, indexer
> Affects Versions: 1.1
> Reporter: Scott Gonyea
> Assignee: Chris A. Mattmann
> Fix For: 1.2
>
> Attachments: nutch-855.txt
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> This plugin is designed to enhance the NUTCH-655 patch, by doing two things:
> 1. Meta Tags that are supplied with your Crawl URLs, during injection, will
> be propagated throughout the outlinks of those Crawl URLs.
> 2. When you index your URLs, the meta tags that you specified with your URLs
> will be indexed alongside those URLs--and can be directly queried, assuming
> you have done everything else correctly.
> The flat-file of URLs you are injecting should, per NUTCH-655, be
> tab-delimited in the form of:
> www.url.com\tkey1=value1\tkey2=value2\t...\tkeyN=valueN
> or:
> http://slashdot.org/ corp_owner=Geeknet will_it_blend=indubitably
> http://engadget.com/ corp_owner=Weblogs genre=geeksquad_thriller
> To activate this plugin, you must modify two properties in your
> nutch-sites.xml:
> 1. plugin.includes
> add: urlmeta
> to: <value>...</value>
> ie: <value>urlmeta|parse-tika|scoring-opic|...</value>
> 2. urlmeta.tags
> Insert a comma-delimited list of metatags. Using the above example:
> <value>corp_owner, will_it_blend, genre</value>
> Note that you do not need to include the tag with every URL. However, you
> must specify each tag if you want it to be propagated and later indexed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
