Repository: geode Updated Branches: refs/heads/develop f271667b1 -> 096c22d5c
GEODE-2913 Update Lucene index documentation Project: http://git-wip-us.apache.org/repos/asf/geode/repo Commit: http://git-wip-us.apache.org/repos/asf/geode/commit/096c22d5 Tree: http://git-wip-us.apache.org/repos/asf/geode/tree/096c22d5 Diff: http://git-wip-us.apache.org/repos/asf/geode/diff/096c22d5 Branch: refs/heads/develop Commit: 096c22d5c73dc609651caf2887b4d95f162230ad Parents: f271667 Author: Karen Miller <kmil...@pivotal.io> Authored: Wed May 17 14:10:42 2017 -0700 Committer: Karen Miller <kmil...@pivotal.io> Committed: Wed May 24 17:07:49 2017 -0700 ---------------------------------------------------------------------- .../source/subnavs/geode-subnav.erb | 22 +- .../implementing_authorization.html.md.erb | 5 + .../statistics/statistics_list.html.md.erb | 24 ++ .../topics/cache-elements-list.html.md.erb | 4 +- .../reference/topics/cache_xml.html.md.erb | 63 +++++ ...mory_requirements_for_cache_data.html.md.erb | 2 + .../gfsh/command-pages/create.html.md.erb | 2 +- .../gfsh/command-pages/destroy.html.md.erb | 13 +- .../lucene_integration.html.md.erb | 283 +++++++++++++++---- 9 files changed, 339 insertions(+), 79 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/geode/blob/096c22d5/geode-book/master_middleman/source/subnavs/geode-subnav.erb ---------------------------------------------------------------------- diff --git a/geode-book/master_middleman/source/subnavs/geode-subnav.erb b/geode-book/master_middleman/source/subnavs/geode-subnav.erb index c97e5ec..12b2151 100644 --- a/geode-book/master_middleman/source/subnavs/geode-subnav.erb +++ b/geode-book/master_middleman/source/subnavs/geode-subnav.erb @@ -2308,16 +2308,7 @@ gfsh</a> <a href="/docs/guide/12/tools_modules/lucene_integration.html#using-the-apache-lucene-integration">Using the Apache Lucene Integration</a> </li> <li> - <a href="/docs/guide/12/tools_modules/lucene_integration.html#java-api-example">Java API Example</a> - </li> - <li> - <a href="/docs/guide/12/tools_modules/lucene_integration.html#search-example">Search Example</a> - </li> - <li> - <a href="/docs/guide/12/tools_modules/lucene_integration.html#gfsh-api">Gfsh API</a> - </li> - <li> - <a href="/docs/guide/12/tools_modules/lucene_integration.html#xml-configuration">XML Configuration</a> + <a href="/docs/guide/12/tools_modules/lucene_integration.html#LuceneRandC">Requirements and Caveats</a> </li> </ul> </li> @@ -2557,6 +2548,14 @@ gfsh</a> <a href="/docs/guide/12/reference/topics/cache_xml.html#index"><index></a> </li> <li class="has_submenu"> + <a href="/docs/guide/12/reference/topics/cache_xml.html#luceneindex"><lucene:index></a> + <ul> + <li> + <a href="/docs/guide/12/reference/topics/cache_xml.html#lucenefield"><lucene:field></a> + </li> + </ul> + </li> + <li class="has_submenu"> <a href="/docs/guide/12/reference/topics/cache_xml.html#entry"><entry></a> <ul> <li class="has_submenu"> @@ -3037,6 +3036,9 @@ gfsh</a> <a href="/docs/guide/12/reference/statistics/statistics_list.html#section_C48B654F973E4B44AD825D459C23A6CD">Locator (LocatorStatistics)</a> </li> <li> + <a href="/docs/guide/12/reference/statistics/statistics_list.html#LuceneStats">Lucene Indexes (LuceneIndexStats)</a> + </li> + <li> <a href="/docs/guide/12/reference/statistics/statistics_list.html#topic_ohc_tjk_w5">Off-Heap (OffHeapMemoryStats)</a> </li> <li> http://git-wip-us.apache.org/repos/asf/geode/blob/096c22d5/geode-docs/managing/security/implementing_authorization.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/managing/security/implementing_authorization.html.md.erb b/geode-docs/managing/security/implementing_authorization.html.md.erb index f897e4c..d16280e 100644 --- a/geode-docs/managing/security/implementing_authorization.html.md.erb +++ b/geode-docs/managing/security/implementing_authorization.html.md.erb @@ -117,18 +117,21 @@ This table classifies the permissions assigned for `gfsh` operations. | create gateway-receiver | DATA:MANAGE | | create gateway-sender | DATA:MANAGE | | create index | DATA:MANAGE:RegionName | +| create lucene index | DATA:MANAGE:RegionName | | create region | DATA:MANAGE | | define index | DATA:MANAGE:RegionName | | deploy | DATA:MANAGE, DATA:WRITE, CLUSTER:MANAGE, and CLUSTER:WRITE | | describe client | CLUSTER:READ | | describe config | CLUSTER:READ | | describe disk-store | CLUSTER:READ | +| describe lucene index | CLUSTER:READ | | describe member | CLUSTER:READ | | describe offline-disk-store | CLUSTER:READ | | describe region | CLUSTER:READ | | destroy disk-store | DATA:MANAGE | | destroy function | DATA:MANAGE | | destroy index | DATA:MANAGE or DATA:MANAGE:RegionName | +| destroy lucene index | DATA:MANAGE:RegionName | | destroy region | DATA:MANAGE | | disconnect | DATA:MANAGE | | echo | DATA:MANAGE | @@ -152,6 +155,7 @@ This table classifies the permissions assigned for `gfsh` operations. | list functions | CLUSTER:READ | | list gateways | CLUSTER:READ | | list indexes | CLUSTER:READ | +| list lucene indexes | CLUSTER:READ | | list members | CLUSTER:READ | | list regions | DATA:READ | | load-balance gateway-sender | DATA:MANAGE | @@ -165,6 +169,7 @@ This table classifies the permissions assigned for `gfsh` operations. | remove | DATA:WRITE:RegionName or DATA:WRITE:RegionName:Key | | resume gateway-sender | DATA:MANAGE | | revoke mising-disk-store | DATA:MANAGE | +| search lucene | DATA:WRITE | | show dead-locks | CLUSTER:READ | | show log | CLUSTER:READ | | show metrics | CLUSTER:READ | http://git-wip-us.apache.org/repos/asf/geode/blob/096c22d5/geode-docs/reference/statistics/statistics_list.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/reference/statistics/statistics_list.html.md.erb b/geode-docs/reference/statistics/statistics_list.html.md.erb index c38b2f7..49e416e 100644 --- a/geode-docs/reference/statistics/statistics_list.html.md.erb +++ b/geode-docs/reference/statistics/statistics_list.html.md.erb @@ -60,6 +60,8 @@ Performance statistics are collected for each Java application or cache server t - **[Locator (LocatorStatistics)](#section_C48B654F973E4B44AD825D459C23A6CD)** +- **[Lucene Indexes (LuceneIndexStats)](#LuceneStats)** + - **[Off-Heap (OffHeapMemoryStats)](#topic_ohc_tjk_w5)** - **[Operating System Statistics - Linux](#section_923B28F01BC3416786D3AFBD87F22A5E)** @@ -1006,6 +1008,28 @@ These statistics are on the Geode locator. The primary statistics are: | `RESPONSES_FROM_LOCATOR` | Number of responses this locator has sent to clients. | | `SERVER_LOAD_UPDATES` | Total number of times a server load update has been received. | +## <a id="LuceneStats" class="no-quick-link"></a>Lucene Indexes (LuceneIndexStats) + +These statistics quantify the use of Lucene indexes. The primary statistics are: + +| Statistic | Description | +|-----------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `queryExecutions` | The number of Lucene queries executed on this member. | +| `queryExecutionTime` | The amount of time in nanoseconds spent executing Lucene queries. | +| `queryExecutionsInProgress` | The number of query executions currently in progress. | +| `queryExecutionTotalHits` | The total number of documents returned by query executions. | +| `repositoryQueryExecutions` | The number of Lucene repository queries executed on this member. | +| `repositoryQueryExecutionTime` | The amount of time in nanoseconds spent executing Lucene repository queries. | +| `repositoryQueryExecutionsInProgress` | The number of repository query executions currently in progress. | +| `repositoryQueryExecutionTotalHits` | The total number of documents returned by repository query executions. | +| `updates` | The number of Lucene index documents added or removed on this member. | +| `updateTime` | The amount of time in nanoseconds spent adding or removing documents from the index. | +| `updatesInProgress` | The number of index updates in progress. | +| `commits` | The number of Lucene index commits on this member. | +| `commitTime` | The amount of time in nanoseconds spent in Lucene index commits. | +| `commitsInProgress` | The number of Lucene index commits in progress. | +| `documents` | The number of documents in the index. | + ## <a id="topic_ohc_tjk_w5" class="no-quick-link"></a>Off-Heap (OffHeapMemoryStats) These statistics quantify the use of off-heap memory. The primary statistics are: http://git-wip-us.apache.org/repos/asf/geode/blob/096c22d5/geode-docs/reference/topics/cache-elements-list.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/reference/topics/cache-elements-list.html.md.erb b/geode-docs/reference/topics/cache-elements-list.html.md.erb index 71e092b..2b1c035 100644 --- a/geode-docs/reference/topics/cache-elements-list.html.md.erb +++ b/geode-docs/reference/topics/cache-elements-list.html.md.erb @@ -151,7 +151,9 @@ For details, see [<cache> Element Reference](cache_xml.html#cache_xml_cach <config-property-value> <region> <region-attributes> - <index>> + <index> + <lucene:index> + <field> <entry> <key> <string> http://git-wip-us.apache.org/repos/asf/geode/blob/096c22d5/geode-docs/reference/topics/cache_xml.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/reference/topics/cache_xml.html.md.erb b/geode-docs/reference/topics/cache_xml.html.md.erb index a934b62..cf5d2b3 100644 --- a/geode-docs/reference/topics/cache_xml.html.md.erb +++ b/geode-docs/reference/topics/cache_xml.html.md.erb @@ -2685,6 +2685,7 @@ Defines a region in the cache. See [<region-attributes>](#region-attribute See [<region-attributes>](#region-attributes) for a complete listing of region attributes. + ## <a id="index" class="no-quick-link"></a><index> Describes an index to be created on a region. The index node, if any, should all come immediately after the "region-attributes" node. The "name" attribute is a required field which identifies the name of the index. See [Working with Indexes](../../developing/query_index/query_index.html) for more information on indexes. @@ -2728,6 +2729,68 @@ Describes an index to be created on a region. The index node, if any, should all </region> ``` +<!-- start of Lucene index description --> +## <a id="luceneindex" class="no-quick-link"></a><lucene:index> + +Describes a Lucene index to be created on a region. The `lucene` namespace +and the scoping operator (`:`) must be specified, as the Geode `cache` +namespace also defines an `index` element (for OQL indexes). + +**API:** `org.apache.geode.cache.lucene` package + +| Attribute | Description | Default | +|-------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| +| name | Required. Name of the Lucene index. | Â | + +**Example:** + +``` pre +<cache + xmlns="http://geode.apache.org/schema/cache"; + xmlns:lucene="http://geode.apache.org/schema/lucene"; + xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; + xsi:schemaLocation="http://geode.apache.org/schema/cache + http://geode.apache.org/schema/cache/cache-1.0.xsd + http://geode.apache.org/schema/lucene + http://geode.apache.org/schema/lucene/lucene-1.0.xsd"; + version="1.0"> + + <region name="regionA" refid="PARTITION"> + <lucene:index name="myIndex"> + <lucene:field name="x" /> + <lucene:field name="y" /> + </lucene:index> + </region> +</cache> +``` +<!-- end of Lucene index description --> + +<!-- start of Lucene field description --> +## <a id="lucenefield" class="no-quick-link"></a><lucene:field> + +Describes a field to be included in a Lucene index. Including the +`lucene` namespace and the scoping operator (`:`) clarifies, +but is not required. + +**API:** `org.apache.geode.cache.lucene` package + +| Attribute | Description | Default | +|-------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| +| name | Required. A string that defines the name of the field. If a single field is defined by the value `"__REGION_VALUE_FIELD"`, then the entire value is used as a single field. | Â | +| analyzer | A string that provides the path to the analyzer to use for this field. A value of `"null"` uses the default analyzer. | `"null"` | + +**Example:** + +``` pre +<region name="dataregion" refid="PARTITION_REDUNDANT"> + <lucene:index name="full_value_index"> + <lucene:field name="__REGION_VALUE_FIELD"/> + </lucene:index> +</region> +``` + +<!-- end of Lucene field description --> + ## <a id="entry" class="no-quick-link"></a><entry> An "entry" element describes an entry to be added to a region. Note that if an entry with the given key already exists in the region, it will be replaced. http://git-wip-us.apache.org/repos/asf/geode/blob/096c22d5/geode-docs/reference/topics/memory_requirements_for_cache_data.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/reference/topics/memory_requirements_for_cache_data.html.md.erb b/geode-docs/reference/topics/memory_requirements_for_cache_data.html.md.erb index 1509a40..150814a 100644 --- a/geode-docs/reference/topics/memory_requirements_for_cache_data.html.md.erb +++ b/geode-docs/reference/topics/memory_requirements_for_cache_data.html.md.erb @@ -175,6 +175,8 @@ For indexes used in querying, the overhead varies greatly depending on the type - If the index has a single value per region entry for the indexed expression, the index introduces at most 243 bytes per region entry. An example of this type of index is: `fromClause="/portfolios", indexedExpression="id"`. The maximum of 243 bytes per region entry is reached if each entry has a unique value for the indexed expression. The overhead is reduced if the entries do not have unique index values. - If each region entry has more than one value for the indexed expression, but no two region entries have the same value for it, then the index introduces at most 236 C + 75 bytes per region entry, where C is the average number of values per region entry for the expression. +- Lucene indexes add approximately 737 bytes per entry. +The other index overhead estimates listed here also apply to Lucene indexes. ## <a id="topic_i1m_stz_j4" class="no-quick-link"></a>Estimating Management and Monitoring Overhead http://git-wip-us.apache.org/repos/asf/geode/blob/096c22d5/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb b/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb index 4fb4c8c..1398352 100644 --- a/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb +++ b/geode-docs/tools_modules/gfsh/command-pages/create.html.md.erb @@ -683,7 +683,7 @@ create lucene index --name=value --region=value --field=value(,value)* [--analyz |----------------------------------------------------|----------------------------------------------------------------------------------------|---------| | <span class="keyword parmname">\\-\\-name</span> | *Required.* Name of the index to create. | Â | | <span class="keyword parmname">\\-\\-region</span> | *Required.* Name/Path of the region which corresponds to the "from" clause in a query. | Â | -| <span class="keyword parmname">\\-\\-field</span> | *Required.* Field of the region values that are referenced by the index. | Â | +| <span class="keyword parmname">\\-\\-field</span> | *Required.* Field of the region values that are referenced by the index. To treat the entire value as a single field, specify `__REGION_VALUE_FIELD`. | Â | | <span class="keyword parmname">‑‑analyzer</span> | Analyzer to extract terms from text | Â | | <span class="keyword parmname">\\-\\-group</span> | The index will be created on all the members in the specified member groups. | Â | http://git-wip-us.apache.org/repos/asf/geode/blob/096c22d5/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb b/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb index e6de426..afd78ee 100644 --- a/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb +++ b/geode-docs/tools_modules/gfsh/command-pages/destroy.html.md.erb @@ -152,25 +152,22 @@ See also [create lucene index](create.html#create_lucene_index), [describe lucen **Syntax:** ``` pre -destroy lucene index [--name=value] [--region=value] +destroy lucene index --region=value [--name=value] ``` -**Note:** -You must specify at least one of the parameter options. If you enter `destroy lucene index` without any parameters, the command will ask you to specify at least one option. - **Parameters, destroy lucene index:** | Name | Description | |------------------------------------------------|------------------------------------------------------------------------------| -| <span class="keyword parmname">\\-\\-name</span> | Name of the index to be removed. | -| <span class="keyword parmname">\\-\\-region</span> | Name of the region from which an index or all indexes are to be removed. | +| <span class="keyword parmname">‑‑region</span> | *Required.* Name of the region from which indexes are to be removed. If no `--name` option is specified, all indexes associated with the region are destroyed.| +| <span class="keyword parmname">‑‑name</span> | Name of the index to be removed. | **Example Commands:** ``` pre -destroy lucene index --member=server2 -destroy lucene index --name=MyKeyIndex +destroy lucene index --region=region1 +destroy lucene index --region=region1 --name=MyKeyIndex ``` ## <a id="topic_BEDACECF4599407794ACBC0E56B30F65" class="no-quick-link"></a>destroy region http://git-wip-us.apache.org/repos/asf/geode/blob/096c22d5/geode-docs/tools_modules/lucene_integration.html.md.erb ---------------------------------------------------------------------- diff --git a/geode-docs/tools_modules/lucene_integration.html.md.erb b/geode-docs/tools_modules/lucene_integration.html.md.erb index e97ce06..b83705b 100644 --- a/geode-docs/tools_modules/lucene_integration.html.md.erb +++ b/geode-docs/tools_modules/lucene_integration.html.md.erb @@ -35,85 +35,73 @@ The Apache Lucene integration: For more details, see Javadocs for the classes and interfaces that implement Apache Lucene indexes and searches, including `LuceneService`, `LuceneQueryFactory`, `LuceneQuery`, and `LuceneResultStruct`. -## <a id="using-the-apache-lucene-integration" class="no-quick-link"></a>Using the Apache Lucene Integration +# <a id="using-the-apache-lucene-integration" class="no-quick-link"></a>Using the Apache Lucene Integration -You can create Apache Lucene indexes through a Java API, through the `gfsh` command-line utility, or by means of -the `cache.xml` configuration file. +You can interact with Apache Lucene indexes through a Java API, +through the `gfsh` command-line utility, +or by means of the `cache.xml` configuration file. -To use Apache Lucene Integration, you will need two pieces of information: +To use Apache Lucene to create and use indexes, +you will need two pieces of information: -1. The name of the region to be indexed or searched +1. The name of the region to be indexed and searched 2. The names of the fields you wish to index +## Key Points ### -### Key Points ### - +- Apache Lucene indexes are supported only on partitioned regions. +Replicated region types are *not* supported. +- Lucene indexes reside on servers. +There is no way to create a Lucene index on a client. - Only top level fields of objects stored in the region can be indexed. -- Apache Lucene indexes are supported only on Partitioned regions. - A single index supports a single region. Indexes do not support multiple regions. -- Heterogeneous objects in single region are supported. -- Join queries between regions are not supported. -- Nested objects are not supported. -- The index needs to be created before the region is created. +- Heterogeneous objects in a single region are supported. + +## <a id="lucene-index-create" class="no-quick-link"></a>Creating an Index + +Create the index before creating the region. -## <a id="java-api-example" class="no-quick-link"></a>Java API Example +When no analyzer is specified, the +`org.apache.lucene.analysis.standard.StandardAnalyzer` will be used. + +### <a id="api-create-example" class="no-quick-link"></a>Java API Example to Create an Index ``` pre // Get LuceneService LuceneService luceneService = LuceneServiceProvider.get(cache); -// Create Index on fields with default analyzer: -luceneService.createIndex(indexName, regionName, "field1", "field2", "field3"); - -Region region = cache.createRegionFactory(RegionShortcut.PARTITION).create(regionName); +// Create the index on fields with default analyzer +// prior to creating the region +luceneService.createIndexFactory() + .addField("name") + .addField("zipcode") + .create(indexName, regionName); +Region region = cache.createRegionFactory(RegionShortcut.PARTITION) + .create(regionName); ``` -## <a id="search-example" class="no-quick-link"></a>Search Example - -``` pre -LuceneQuery<String, Person> query = luceneService.createLuceneQueryFactory() - .setResultLimit(10) - .create(indexName, regionName, "Main Street", "address"); - -Collection<Person> results = query.findValues(); -``` - - -## <a id="gfsh-api" class="no-quick-link"></a>Gfsh API - -The gfsh command-line utility supports five Apache Lucene actions: - -<dt><a href="gfsh/command-pages/create.html#create_lucene_index"><b>create lucene index</b></a></dt> - <dd>Create a Lucene index that can be used to execute queries.</dd> -<dt><a href="gfsh/command-pages/describe.html#describe_lucene_index"><b>describe lucene index</b></a></dt> - <dd>Describe a Lucene index.</dd> -<dt><a href="gfsh/command-pages/destroy.html#destroy_lucene_index"><b>destroy lucene index</b></a></dt> - <dd>Destroy a Lucene index.</dd> -<dt><a href="gfsh/command-pages/list.html#list_lucene_indexes"><b>list lucene indexes</b></a></dt> - <dd>List Lucene indexes created for all members.</dd> -<dt><a href="gfsh/command-pages/search.html#search_lucene"><b>search lucene</b></a></dt> - <dd>Search a Lucene index.</dd> +### <a id="gfsh-create-example" class="no-quick-link"></a>Gfsh Example to Create an Index -**Gfsh command-line examples:** +For details, see the [gfsh create lucene index](gfsh/command-pages/create.html#create_lucene_index") command reference page. ``` pre -// List Index -gfsh> list lucene indexes --with-stats - -// Create Index gfsh>create lucene index --name=indexName --region=/orders --field=customer,tags - -// Create Index, specifying a custom analyzer for the second field -// Note: "null" in the first analyzer position means "use the default analyzer for the first field" -gfsh>create lucene index --name=indexName --region=/orders --field=customer,tags --analyzer=null,org.apache.lucene.analysis.bg.BulgarianAnalyzer - -// Execute Lucene query -gfsh> lucene search --regionName=/orders -queryStrings="John*" --defaultField=field1 --limit=100 ``` +``` pre +// Create an index, specifying a custom analyzer for the second field +// Note: "null" in the first analyzer position uses the default analyzer +// for the first field +gfsh>create lucene index --name=indexName --region=/orders + --field=customer,tags --analyzer=null,org.apache.lucene.analysis.bg.BulgarianAnalyzer +``` +To use the entire value as a single field set the required `--field` +option to be `__REGION_VALUE_FIELD`. +This is only supported when the region entry value is a `String`, `Long`, +`Integer`, `Float`, or `Double`. -## <a id="xml-configuration" class="no-quick-link"></a>XML Configuration +### <a id="xml-configuration" class="no-quick-link"></a>XML Configuration to Create an Index ``` pre <cache @@ -127,12 +115,189 @@ gfsh> lucene search --regionName=/orders -queryStrings="John*" --defaultField=fi version="1.0"> <region name="region" refid="PARTITION"> - <lucene:index name="index"> - <lucene:field name="a" analyzer="org.apache.lucene.analysis.core.KeywordAnalyzer"/> - <lucene:field name="b" analyzer="org.apache.lucene.analysis.core.SimpleAnalyzer"/> - <lucene:field name="c" analyzer="org.apache.lucene.analysis.standard.ClassicAnalyzer"/> + <lucene:index name="myIndex"> + <lucene:field name="a" + analyzer="org.apache.lucene.analysis.core.KeywordAnalyzer"/> + <lucene:field name="b" + analyzer="org.apache.lucene.analysis.core.SimpleAnalyzer"/> + <lucene:field name="c" + analyzer="org.apache.lucene.analysis.standard.ClassicAnalyzer"/> + <lucene:field name="d" /> </lucene:index> </region> </cache> ``` +## <a id="lucene-index-query" class="no-quick-link"></a>Queries + +### <a id="gfsh-query-example" class="no-quick-link"></a>Gfsh Example to Query using a Lucene Index + +For details, see the [gfsh search lucene](gfsh/command-pages/search.html#search_lucene") command reference page. + +``` pre +gfsh> lucene search --regionName=/orders -queryStrings="John*" --defaultField=field1 + --limit=100 +``` + +### <a id="api-query-example" class="no-quick-link"></a>Java API Example to Query using a Lucene Index + +``` pre +LuceneQuery<String, Person> query = luceneService.createLuceneQueryFactory() + .setLimit(10) + .create(indexName, regionName, "name:John AND zipcode:97006", defaultField); + +Collection<Person> results = query.findValues(); +``` + +## <a id="lucene-index-destroy" class="no-quick-link"></a>Destroying an Index + +Since a region destroy operation does not cause the destruction +of any Lucene indexes, +destroy any Lucene indexes prior to destroying the associated region. + +### <a id="API-destroy-example" class="no-quick-link"></a>Java API Example to Destroy a Lucene Index + +``` pre +luceneService.destroyIndex(indexName, regionName); +``` +An attempt to destroy a region with a Lucene index will result in +an `IllegalStateException`, +issuing an error message similar to: + +``` pre +java.lang.IllegalStateException: The parent region [/orders] in colocation chain + cannot be destroyed, unless all its children [[/indexName#_orders.files]] are + destroyed +at org.apache.geode.internal.cache.PartitionedRegion + .checkForColocatedChildren(PartitionedRegion.java:7231) +at org.apache.geode.internal.cache.PartitionedRegion + .destroyRegion(PartitionedRegion.java:7243) +at org.apache.geode.internal.cache.AbstractRegion + .destroyRegion(AbstractRegion.java:308) +at DestroyLuceneIndexesAndRegionFunction + .destroyRegion(DestroyLuceneIndexesAndRegionFunction.java:46) +``` +### <a id="gfsh-destroy-example" class="no-quick-link"></a>Gfsh Example to Destroy a Lucene Index + +For details, see the [gfsh destroy lucene index](gfsh/command-pages/destroy.html#destroy_lucene_index") command reference page. + +The error message that results from an attempt to destroy a region +prior to destroying its associated Lucene index +will be similar to: + +``` pre +Error occurred while destroying region "orders". + Reason: The parent region [/orders] in colocation chain cannot be destroyed, + unless all its children [[/indexName#_orders.files]] are destroyed +``` + +## <a id="lucene-index-change" class="no-quick-link"></a>Changing an Index + +Changing an index requires rebuilding it. +Implement these steps to change an index: +1. Export all region data +2. Destroy the Lucene index +3. Destroy the region +4. Create a new index +5. Create a new region without the user-defined business logic callbacks +6. Import the region data with the option to turn on callbacks. +The callbacks will be to invoke a Lucene async event listener to index +the data. +7. Alter the region to add the user-defined business logic callbacks + +## <a id="addl-gfsh-api" class="no-quick-link"></a>Additional Gfsh Commands + +See the [gfsh describe lucene index](gfsh/command-pages/describe.html#describe_lucene_index") command reference page for the command that prints details about +a specific index. + +See the [gfsh list lucene index](gfsh/command-pages/list.html#list_lucene_index") command reference page +for the command that prints details about the +Lucene indexes created for all members. + +# <a id="LuceneRandC" class="no-quick-link"></a>Requirements and Caveats + +- Join queries between regions are not supported. +- Nested objects are not supported. +- Lucene indexes will not be stored within off-heap memory. +- Lucene queries from within transactions are not supported. +On an attempt to query from within a transaction, +a `LuceneQueryException` is thrown, issuing an error message +on the client (accessor) similar to: + +``` pre +Exception in thread "main" org.apache.geode.cache.lucene.LuceneQueryException: + Lucene Query cannot be executed within a transaction +at org.apache.geode.cache.lucene.internal.LuceneQueryImpl + .findTopEntries(LuceneQueryImpl.java:124) +at org.apache.geode.cache.lucene.internal.LuceneQueryImpl + .findPages(LuceneQueryImpl.java:98) +at org.apache.geode.cache.lucene.internal.LuceneQueryImpl + .findPages(LuceneQueryImpl.java:94) +at TestClient.executeQuerySingleMethod(TestClient.java:196) +at TestClient.main(TestClient.java:59) +``` +- Lucene indexes must be created prior to creating the region. +If an attempt is made to create a Lucene index after creating the region, +the error message will be similar to: + +``` pre + Member | Status +---------------------------- | ------------------------------------------------------ +192.0.2.0(s2:97639)<v2>:1026 | Failed: The lucene index must be created before region +192.0.2.0(s3:97652)<v3>:1027 | Failed: The lucene index must be created before region +192.0.2.0(s1:97626)<v1>:1025 | Failed: The lucene index must be created before region +``` +- An invalidate operation on a region entry does not invalidate a corresponding +Lucene index entry. +A query on a Lucene index that contains values that +have been invalidated can return results that no longer exist. +Therefore, do not combine entry invalidation with queries on Lucene indexes. +- Lucene indexes are not supported for regions that have eviction configured +with a local destroy. +Eviction can be configured with overflow to disk, +but only the region data is overflowed to disk, +not the Lucene index. +On an attempt to create a region with eviction configured to do local destroy +(with a Lucene index), +an `UnsupportedOperationException` will be thrown, +issuing an error message similar to: + +``` pre +[error 2017/05/02 16:12:32.461 PDT <main> tid=0x1] + java.lang.UnsupportedOperationException: + Lucene indexes on regions with eviction and action local destroy are not supported +Exception in thread "main" java.lang.UnsupportedOperationException: + Lucene indexes on regions with eviction and action local destroy are not supported +at org.apache.geode.cache.lucene.internal.LuceneRegionListener + .beforeCreate(LuceneRegionListener.java:85) +at org.apache.geode.internal.cache.GemFireCacheImpl + .invokeRegionBefore(GemFireCacheImpl.java:3154) +at org.apache.geode.internal.cache.GemFireCacheImpl + .createVMRegion(GemFireCacheImpl.java:3013) +at org.apache.geode.internal.cache.GemFireCacheImpl + .basicCreateRegion(GemFireCacheImpl.java:2991) +``` +- Be aware that using the same field name in different objects +where the field has different data types +may have unexpected consequences. +For example, if an index on the field SSN has the following entries + - `Object_1 object_1` has String SSN = "1111" + - `Object_2 object_2` has Integer SSN = 1111 + - `Object_3 object_3` has Float SSN = 1111.0 + + Integers and floats will not be converted into strings. + They remain as `IntPoint` and `FloatPoint` within Lucene. + The standard analyzer will not try to tokenize these values. + The standard analyzer will only try to break up string values. + So, a string search for "SSN: 1111" will return `object_1`. + An `IntRangeQuery` for `upper limit : 1112` and `lower limit : 1110` +will return `object_2`. + And, a `FloatRangeQuery` with `upper limit : 1111.5` and `lower limit : 1111.0` +will return `object_3`. +- Backups should only be made for regions with Lucene indexes +when there are no puts, updates, or deletes in progress. +Incremental backups will not be consistent for the region and +its index upon restart if these operations were in progress, +due to the delayed processing associated with the asynchronous event queue. +If region data needs to be restored from a backup, +follow the same procedure as given for changing an index.
