[
https://issues.apache.org/jira/browse/NUTCH-863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrzej Bialecki updated NUTCH-863:
------------------------------------
Attachment: proxy.patch
Patch with the tools. If there are no objections I'd like to commit this soon,
and track improvements in other JIRA issues.
> Benchmark and a testbed proxy server
> ------------------------------------
>
> Key: NUTCH-863
> URL: https://issues.apache.org/jira/browse/NUTCH-863
> Project: Nutch
> Issue Type: New Feature
> Affects Versions: 2.0
> Reporter: Andrzej Bialecki
> Assignee: Andrzej Bialecki
> Attachments: proxy.patch
>
>
> This issue adds two components:
> * a testbed proxy server that can serve various content: pre-fetched Nutch
> segments, forward requests to original URLs, or create a lot of unique but
> predictable fake content (with outlinks) on the fly.
> * a simple Benchmark class to measure the time taken to complete several
> crawl cycles using fake content.
> * 'ant proxy' and 'ant benchmark' targets to execute a benchmark run.
> Together these tools should provide a more or less objective method to
> measure the end-to-end crawl performance. This initial version can be further
> instrumented to collect statistics about various stages of data processing.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
