Author: kwright
Date: Fri Mar 29 19:21:00 2013
New Revision: 1462614
URL: http://svn.apache.org/r1462614
Log:
Update Solr connector documentation. Part of CONNECTORS-665.
Added:
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-solr-type.PNG
(with props)
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-zookeeper.PNG
(with props)
Modified:
manifoldcf/trunk/CHANGES.txt
manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-schema.PNG
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-server.PNG
Modified: manifoldcf/trunk/CHANGES.txt
URL:
http://svn.apache.org/viewvc/manifoldcf/trunk/CHANGES.txt?rev=1462614&r1=1462613&r2=1462614&view=diff
==============================================================================
--- manifoldcf/trunk/CHANGES.txt (original)
+++ manifoldcf/trunk/CHANGES.txt Fri Mar 29 19:21:00 2013
@@ -3,6 +3,9 @@ $Id$
======================= 1.2-dev =====================
+CONNECTORS-665: Update Solr connector type end-user documentation.
+(Karl Wright)
+
CONNECTORS-667: Fix livelink authority caching to work properly with
new SSL connection support.
(David Morana, Karl Wright)
Modified:
manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml
URL:
http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml?rev=1462614&r1=1462613&r2=1462614&view=diff
==============================================================================
---
manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml
(original)
+++
manifoldcf/trunk/site/src/documentation/content/xdocs/en_US/end-user-documentation.xml
Fri Mar 29 19:21:00 2013
@@ -435,41 +435,60 @@
<section id="solroutputconnector">
<title>Solr Output Connection</title>
- <p>The Solr output connection type is designed to allow
ManifoldCF to submit documents to an appropriate Solr pipeline, via the Solr
- HTTP ingestion API. The configuration parameters are
set to the default Solr values, which can be changed (since Solr's
configuration can be changed).
- The Solr output connection type furthermore makes no
judgment as to whether a given document is indexable or not - it accepts
everything, and passes all documents
- on to the pipeline, where presumably the configured
pipeline will decide if a document should be rejected or not. (All of that
happens without a Solr connection
- being aware of it in any way.)</p>
- <p>Unfortunately, this lack of specificity comes at a cost.
Unless you take care to filter documents properly in each job, large movie
files or other opaque
- content may well be picked up and sent to Solr for
indexing, which will greatly increase the dead load on the overall system. It
is therefore a good idea to review
- all crawls done through a Solr connection while they
are underway, to be sure there isn't a misconfiguration of this kind.</p>
- <p>When you create a Solr output connection, five
configuration tabs appear. The "Server" tab allows you to configure the HTTP
target of the connection:</p>
+ <p>The Solr output connection type is designed to allow
ManifoldCF to submit documents to either an appropriate Apache Solr instance,
+ via the Solr HTTP API, or alternatively to a Solr Cloud
cluster. The configuration parameters are initially set to appropriate default
+ values for a stand-alone Solr instance.</p>
+ <p>When you create a Solr output connection, multiple
configuration tabs appear. The first tab is the "Solr type" tab. Here you
select
+ whether you want your connection to communicate to a
standalone Solr instance, or to a Solr Cloud cluster:</p>
+ <br/><br/>
+ <figure src="images/en_US/solr-configure-solr-type.PNG"
alt="Solr Configuration, Solr type tab" width="80%"/>
+ <br/><br/>
+ <p>Select which kind of Solr installation you want to
communicate with. Based on your selection, you can proceed to either the
"Server"
+ tab (if a standalone instance) or to the "ZooKeeper"
tab (if a SolrCloud cluster).</p>
+ <p>The "Server" tab allows you to configure the HTTP
parameters appropriate for communicating with a standalone Solr instance:</p>
<br/><br/>
<figure src="images/en_US/solr-configure-server.PNG" alt="Solr
Configuration, Server tab" width="80%"/>
<br/><br/>
- <p>Fill in the fields according to your Solr configuration.
The Solr connection type supports only basic authentication at this time; if
you have this enabled, supply the credentials
- as requested on the bottom part of the form.</p>
- <p>The second tab is the "Schema" tab, which allows you to
specify the name of the Solr field to use as a document identifier. The Solr
connection type will treat
- this field as being a unique key for locating the
indexed document for further modification or deletion:</p>
+ <p>If your Solr setup is a standalone instance, fill in the
fields according to your Solr configuration. The Solr connection type supports
+ only basic authentication at this time; if you have
this enabled, supply the credentials as requested on the bottom part of the
form.</p>
+ <p>The "Zookeeper" tab allows your to configure the connection
type to communicate with a Solr Cloud cluster:</p>
+ <br/><br/>
+ <figure src="images/en_US/solr-configure-zookeeper.PNG"
alt="Solr Configuration, Zookeeper tab" width="80%"/>
+ <br/><br/>
+ <p>Here, add each ZooKeeper instance in the SolrCloud cluster
to the list of ZooKeeper instances. The connection comes preconfigured with
+ "localhost" as being a ZooKeeper instance. You may
delete this if it is not the case.</p>
+ <p>The next tab is the "Schema" tab, which allows you to
specify the names of various Solr fields into which the Solr connection type
will
+ place built-in document metadata:</p>
<br/><br/>
<figure src="images/en_US/solr-configure-schema.PNG" alt="Solr
Configuration, Schema tab" width="80%"/>
<br/><br/>
- <p>The third tab is the "Arguments" tab, which allows you to
specify arbitrary arguments to be sent to Solr. All valid Solr update request
parameters
- can be specified here. You can for instance add <a
href="http://wiki.apache.org/solr/UpdateRequestProcessor";>update.chain=myChain</a>
to select the document processing pipeline/chain to use for
- processing documents in Solr. See the Solr
documentation for more valid arguments. The tab looks like:</p>
+ <p>The most important of these is the document identifier
field, which MUST be present for the connection type to function. This field
will
+ be used to uniquely identify the document within Solr,
and will contain the document's URL. The Solr connection type will treat this
field as being
+ a unique key for locating the indexed document for
further modification or deletion. The other Solr fields are optional, and
largely self-
+ explanatory.</p>
+ <p>The next tab is the "Arguments" tab, which allows you to
specify arbitrary arguments to be sent to Solr:</p>
<br/><br/>
<figure src="images/en_US/solr-configure-arguments.PNG"
alt="Solr Configuration, Arguments tab" width="80%"/>
<br/><br/>
<p>Fill in the argument name and value, and click the "Add"
button. Bear in mind that if you add an argument with the same name as an
existing one, it will replace the
existing one with the new specified value. You can
delete existing arguments by clicking the "Delete" button next to the argument
you want to delete.</p>
- <p>The fourth tab is the "Documents" tab, which allows you to
do document filtering based on size and mime types. By specifying a maximum
document length in bytes, you can filter out documents which exceed that size
(e.g. 10485760 which is equivalent to 10 MB). If you only want to add documents
with specific mime types, you can enter them into the "included mime types"
field (e.g. "text/html" for filtering out all documents but HTML). The
"excluded mime types" field is for excluding documents with specific mime types
(e.g. "image/jpeg" for filtering out JPEG images). The tab looks like:</p>
+ <p>Use this tab to specify any and all desired Solr update
request parameters. You can, for instance, add
+ <a
href="http://wiki.apache.org/solr/UpdateRequestProcessor";>update.chain=myChain</a>
to select a specific document processing pipeline/chain to
+ use for processing documents. See the Solr
documentation for more valid arguments.</p>
+ <p>The next tab is the "Documents" tab, which allows you to do
document filtering based on size and mime types. By specifying a maximum
document
+ length in bytes, you can filter out documents which
exceed that size (e.g. 10485760 which is equivalent to 10 MB). If you only want
to add
+ documents with specific mime types, you can enter them
into the "included mime types" field (e.g. "text/html" for filtering out all
documents but HTML).
+ The "excluded mime types" field is for excluding
documents with specific mime types (e.g. "image/jpeg" for filtering out JPEG
images). The tab looks like:</p>
<figure src="images/en_US/solr-configure-documents.PNG"
alt="Solr Configuration, Documents tab" width="80%"/>
<br/><br/>
- <p>The fifth tab is the "Commits" tab, which allows you to
control the commit strategies. As well as committing documents at the end of
every job, an option which is enabled by default, you may also commit each
document within a certain time in milliseconds (e.g. "10000" for committing
within 10 seconds). The <a
href="http://wiki.apache.org/solr/CommitWithin";>commit within</a> strategy will
leave the responsibility to Solr instead of ManifoldCF. The tab looks like:</p>
+ <p>The fifth tab is the "Commits" tab, which allows you to
control the commit strategies. As well as committing documents at the end of
every job, an
+ option which is enabled by default, you may also commit
each document within a certain time in milliseconds (e.g. "10000" for
committing within
+ 10 seconds). The <a
href="http://wiki.apache.org/solr/CommitWithin";>commit within</a> strategy will
leave the responsibility to Solr instead
+ of ManifoldCF. The tab looks like:</p>
<figure src="images/en_US/solr-configure-commits.PNG"
alt="Solr Configuration, Documents tab" width="80%"/>
<br/><br/>
- <p>When you are done, don't forget to click the "Save" button
to save your changes! When you do, a connection summary and status screen will
be presented, which
- may look something like this:</p>
+ <p>When you are done, don't forget to click the "Save" button
to save your changes! When you do, a connection summary and status screen will
be
+ presented, which may look something like this:</p>
<br/><br/>
<figure src="images/en_US/solr-status.PNG" alt="Solr Status"
width="80%"/>
<br/><br/>
Modified:
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-schema.PNG
URL:
http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-schema.PNG?rev=1462614&r1=1462613&r2=1462614&view=diff
==============================================================================
Binary files - no diff available.
Modified:
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-server.PNG
URL:
http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-server.PNG?rev=1462614&r1=1462613&r2=1462614&view=diff
==============================================================================
Binary files - no diff available.
Added:
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-solr-type.PNG
URL:
http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-solr-type.PNG?rev=1462614&view=auto
==============================================================================
Binary file - no diff available.
Propchange:
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-solr-type.PNG
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added:
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-zookeeper.PNG
URL:
http://svn.apache.org/viewvc/manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-zookeeper.PNG?rev=1462614&view=auto
==============================================================================
Binary file - no diff available.
Propchange:
manifoldcf/trunk/site/src/documentation/resources/images/en_US/solr-configure-zookeeper.PNG
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
