This is an automated email from the ASF dual-hosted git repository. nightowl888 pushed a commit to branch docs/4.8.0-beta00014 in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit 553b2560b897f66b2ef767420869e45320675b2f Author: Shad Storhaug <[email protected]> AuthorDate: Fri Apr 16 22:19:57 2021 +0700 docs: Lucene.Net.QueryParser + Lucene.Net.Replicator: Fixed formatting and code snippets so they are readable, fixed many broken links (see #284, #300) --- src/Lucene.Net.QueryParser/Classic/package.md | 77 ++++++-- .../Flexible/Core/Builders/package.md | 4 +- .../Flexible/Core/Config/package.md | 8 +- .../Flexible/Core/Messages/package.md | 4 +- .../Flexible/Core/Nodes/package.md | 45 ++++- .../Flexible/Core/Parser/package.md | 7 +- .../Flexible/Core/Processors/package.md | 14 +- .../Flexible/Core/Util/package.md | 4 +- .../Flexible/Core/package.md | 12 +- .../Flexible/Messages/package.md | 61 +++++- .../Flexible/Standard/Builders/package.md | 6 +- .../Flexible/Standard/Config/package.md | 6 +- .../Flexible/Standard/Nodes/package.md | 4 +- .../Flexible/Standard/Parser/package.md | 6 +- .../Flexible/Standard/Processors/package.md | 6 +- .../Flexible/Standard/package.md | 10 +- src/Lucene.Net.QueryParser/Simple/package.md | 4 +- src/Lucene.Net.QueryParser/Xml/package.md | 4 +- src/Lucene.Net.QueryParser/overview.md | 82 ++++---- src/Lucene.Net.Replicator/Http/package.md | 4 +- src/Lucene.Net.Replicator/Http/replicator.md | 162 ---------------- src/Lucene.Net.Replicator/package.md | 210 ++++++++++++++++++--- websites/apidocs/index.md | 2 +- 23 files changed, 436 insertions(+), 306 deletions(-) diff --git a/src/Lucene.Net.QueryParser/Classic/package.md b/src/Lucene.Net.QueryParser/Classic/package.md index 7f2c35f..07217a2 100644 --- a/src/Lucene.Net.QueryParser/Classic/package.md +++ b/src/Lucene.Net.QueryParser/Classic/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Classic summary: *content --- @@ -27,9 +27,6 @@ Note that JavaCC defines lots of public classes, methods and fields that do not need to be public. These clutter the documentation. Sorry. -Note that because JavaCC defines a class named <tt>Token</tt>, <tt>org.apache.lucene.analysis.Token</tt> -must always be fully qualified in source code in this package. - __NOTE__: <xref:Lucene.Net.QueryParsers.Flexible.Standard> has an alternative queryparser that matches the syntax of this one, but is more modular, enabling substantial customization to how a query is created. @@ -45,7 +42,7 @@ enabling substantial customization to how a query is created. * [Wildcard Searches](#wildcard-searches) - * [Regular expression Searches](#regexp-searches) + * [Regular expression Searches](#regular-expression-searches) * [Fuzzy Searches](#fuzzy-searches) @@ -61,7 +58,7 @@ enabling substantial customization to how a query is created. * [AND](#and) - * [+](#+) + * [+](#nbsp) * [NOT](#not) @@ -75,11 +72,15 @@ enabling substantial customization to how a query is created. ## Overview -Although Lucene provides the ability to create your own queries through its API, it also provides a rich query language through the Query Parser, a lexer which interprets a string into a Lucene Query using JavaCC. +Although Lucene.NET provides the ability to create your own queries through its API, it also provides a rich query language through the Query Parser, a lexer which interprets a string into a Lucene.NET Query using JavaCC. -Generally, the query parser syntax may change from release to release. This page describes the syntax as of the current release. If you are using a different version of Lucene, please consult the copy of <span class="codefrag">docs/queryparsersyntax.html</span> that was distributed with the version you are using. +Generally, the query parser syntax may change from release to release. This page describes the syntax as of the current release. If you are using a different version of Lucene.NET, please consult the copy of this document that was distributed with the version you are using. - Before choosing to use the provided Query Parser, please consider the following: 1. If you are programmatically generating a query string and then parsing it with the query parser then you should seriously consider building your queries directly with the query API. In other words, the query parser is designed for human-entered text, not for program-generated text. 2. Untokenized fields are best added directly to queries, and not through the query parser. If a field's values are generate [...] +Before choosing to use the provided Query Parser, please consider the following: + +1. If you are programmatically generating a query string and then parsing it with the query parser then you should seriously consider building your queries directly with the query API. In other words, the query parser is designed for human-entered text, not for program-generated text. +2. Untokenized fields are best added directly to queries, and not through the query parser. If a field's values are generated programmatically by the application, then so should query clauses for this field. An analyzer, which the query parser uses, is designed to convert human-entered text to terms. Program-generated values, like dates, keywords, etc., should be consistently program-generated. +3. In a query form, fields which are general text should use the query parser. All others, such as date ranges, keywords, etc. are better added directly through the query API. A field with a limit set of values, that can be specified with a pull-down menu should not be added to a query string which is subsequently parsed, but rather added as a TermQuery clause. ## Terms @@ -101,17 +102,23 @@ You can search any field by typing the field name followed by a colon ":" and th As an example, let's assume a Lucene index contains two fields, title and text and text is the default field. If you want to find the document entitled "The Right Way" which contains the text "don't go this way", you can enter: +```console title:"The Right Way" AND text:go +``` or +```console title:"The Right Way" AND go +``` Since text is the default field, the field indicator is not required. Note: The field is only valid for the term that it directly precedes, so the query +```console title:The Right Way +``` Will only find "The" in the title field. It will find "Right" and "Way" in the default field (in this case the text field). @@ -129,15 +136,21 @@ To perform a multiple character wildcard search use the "*" symbol. The single character wildcard search looks for terms that match that with the single character replaced. For example, to search for "text" or "test" you can use the search: +```console te?t +``` Multiple character wildcard searches looks for 0 or more characters. For example, to search for test, tests or tester, you can use the search: +```console test* +``` You can also use the wildcard searches in the middle of a term. +```console te*t +``` Note: You cannot use a * or ? symbol as the first character of a search. @@ -145,19 +158,25 @@ Note: You cannot use a * or ? symbol as the first character of a search. Lucene supports regular expression searches matching a pattern between forward slashes "/". The syntax may change across releases, but the current supported syntax is documented in the [RegExp](xref:Lucene.Net.Util.Automaton.RegExp) class. For example to find documents containing "moat" or "boat": +```console /[mb]oat/ +``` ### Fuzzy Searches Lucene supports fuzzy searches based on Damerau-Levenshtein Distance. To do a fuzzy search use the tilde, "~", symbol at the end of a Single word Term. For example to search for a term similar in spelling to "roam" use the fuzzy search: +```console roam~ +``` This search will find terms like foam and roams. An additional (optional) parameter can specify the maximum number of edits allowed. The value is between 0 and 2, For example: +```console roam~1 +``` The default that is used if the parameter is not given is 2 edit distances. @@ -167,17 +186,23 @@ Previously, a floating point value was allowed here. This syntax is considered d Lucene supports finding words are a within a specific distance away. To do a proximity search use the tilde, "~", symbol at the end of a Phrase. For example to search for a "apache" and "jakarta" within 10 words of each other in a document use the search: +```console "jakarta apache"~10 +``` ### Range Searches Range Queries allow one to match documents whose field(s) values are between the lower and upper bound specified by the Range Query. Range Queries can be inclusive or exclusive of the upper and lower bounds. Sorting is done lexicographically. +```console mod_date:[20020101 TO 20030101] +``` This will find documents whose mod_date fields have values between 20020101 and 20030101, inclusive. Note that Range Queries are not reserved for date fields. You could also use range queries with non-date fields: +```console title:{Aida TO Carmen} +``` This will find all documents whose titles are between Aida and Carmen, but not including Aida and Carmen. @@ -189,15 +214,21 @@ Lucene provides the relevance level of matching documents based on the terms fou Boosting allows you to control the relevance of a document by boosting its term. For example, if you are searching for +```console jakarta apache +``` and you want the term "jakarta" to be more relevant boost it using the ^ symbol along with the boost factor next to the term. You would type: +```console jakarta^4 apache +``` This will make documents with the term jakarta appear more relevant. You can also boost Phrase Terms as in the example: +```console "jakarta apache"^4 "Apache Lucene" +``` By default, the boost factor is 1. Although the boost factor must be positive, it can be less than 1 (e.g. 0.2) @@ -211,11 +242,15 @@ The OR operator is the default conjunction operator. This means that if there is To search for documents that contain either "jakarta apache" or just "jakarta" use the query: +```console "jakarta apache" jakarta +``` or +```console "jakarta apache" OR jakarta +``` ### AND @@ -223,15 +258,19 @@ The AND operator matches documents where both terms exist anywhere in the text o To search for documents that contain "jakarta apache" and "Apache Lucene" use the query: +```console "jakarta apache" AND "Apache Lucene" +``` -### + +### + The "+" or required operator requires that the term after the "+" symbol exist somewhere in a the field of a single document. To search for documents that must contain "jakarta" and may contain "lucene" use the query: +```console +jakarta lucene +``` ### NOT @@ -239,19 +278,25 @@ The NOT operator excludes documents that contain the term after NOT. This is equ To search for documents that contain "jakarta apache" but not "Apache Lucene" use the query: +```console "jakarta apache" NOT "Apache Lucene" +``` Note: The NOT operator cannot be used with just one term. For example, the following search will return no results: +```console NOT "jakarta apache" +``` ### - The "-" or prohibit operator excludes documents that contain the term after the "-" symbol. -To search for documents that contain "jakarta apache" but not "Apache Lucene" use the query: +To search for documents that contain "jakarta apache" but not "Apache Lucene" use the query: +```console "jakarta apache" -"Apache Lucene" +``` ## Grouping @@ -259,7 +304,9 @@ Lucene supports using parentheses to group clauses to form sub queries. This can To search for either "jakarta" or "apache" and "website" use the query: +```console (jakarta OR apache) AND website +``` This eliminates any confusion and makes sure you that website must exist and either term jakarta or apache may exist. @@ -269,14 +316,20 @@ Lucene supports using parentheses to group multiple clauses to a single field. To search for a title that contains both the word "return" and the phrase "pink panther" use the query: +```console title:(+return +"pink panther") +``` ## Escaping Special Characters Lucene supports escaping special characters that are part of the query syntax. The current list special characters are +```console + - && || ! ( ) { } [ ] ^ " ~ * ? : \ / +``` To escape these character use the \ before the character. For example to search for (1+1):2 use the query: -\(1\+1\)\:2 \ No newline at end of file +```console +\(1\+1\)\:2 +``` \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Core/Builders/package.md b/src/Lucene.Net.QueryParser/Flexible/Core/Builders/package.md index c294e9e..0608b9e 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Core/Builders/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Core/Builders/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Core.Builders summary: *content --- @@ -25,4 +25,4 @@ Necessary classes to implement query builders. ## Query Parser Builders - The package <tt>org.apache.lucene.queryParser.builders</tt> contains the interface that builders must implement, it also contain a utility <xref:Lucene.Net.QueryParsers.Flexible.Core.Builders.QueryTreeBuilder>, which walks the tree and call the Builder for each node in the tree. Builder normally convert QueryNode Object into a Lucene Query Object, and normally it's a one-to-one mapping class. But other builders implementations can by written to convert QueryNode objects to other non luc [...] \ No newline at end of file +The namespace <tt>Lucene.Net.QueryParsers.Core.Builders</tt> contains the interface that builders must implement, it also contain a utility <xref:Lucene.Net.QueryParsers.Flexible.Core.Builders.QueryTreeBuilder>, which walks the tree and call the Builder for each node in the tree. Builder normally convert QueryNode Object into a Lucene Query Object, and normally it's a one-to-one mapping class. But other builders implementations can by written to convert QueryNode objects to other non-Luc [...] \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Core/Config/package.md b/src/Lucene.Net.QueryParser/Flexible/Core/Config/package.md index d0179d8..861a939 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Core/Config/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Core/Config/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Core.Config summary: *content --- @@ -25,8 +25,8 @@ Base classes used to configure the query processing. ## Query Configuration Interfaces - The package <tt>org.apache.lucene.queryparser.flexible.config</tt> contains query configuration handler abstract class that all config handlers should extend. +The namespace <tt>Lucene.Net.QueryParsers.Flexible.Core.Config</tt> contains query configuration handler abstract class that all config handlers should extend. - See <xref:Lucene.Net.QueryParsers.Flexible.Standard.Config.StandardQueryConfigHandler> for a reference implementation. +See <xref:Lucene.Net.QueryParsers.Flexible.Standard.Config.StandardQueryConfigHandler> for a reference implementation. - The <xref:Lucene.Net.QueryParsers.Flexible.Core.Config.QueryConfigHandler> and <xref:Lucene.Net.QueryParsers.Flexible.Core.Config.FieldConfig> are used in the processors to access config information in a flexible and independent way. See <xref:Lucene.Net.QueryParsers.Flexible.Standard.Processors.TermRangeQueryNodeProcessor> for a reference implementation. +The <xref:Lucene.Net.QueryParsers.Flexible.Core.Config.QueryConfigHandler> and <xref:Lucene.Net.QueryParsers.Flexible.Core.Config.FieldConfig> are used in the processors to access config information in a flexible and independent way. See <xref:Lucene.Net.QueryParsers.Flexible.Standard.Processors.TermRangeQueryNodeProcessor> for a reference implementation. diff --git a/src/Lucene.Net.QueryParser/Flexible/Core/Messages/package.md b/src/Lucene.Net.QueryParser/Flexible/Core/Messages/package.md index 8f87a15..8f315a0 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Core/Messages/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Core/Messages/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Core.Messages summary: *content --- @@ -26,4 +26,4 @@ Messages usually used by query parser implementations. ## Query Parser Messages -Messages for the Flexible Query Parser, they use <tt>org.apache.lucene.messages.NLS</tt> API. \ No newline at end of file +Messages for the Flexible Query Parser use [Lucene.Net.QueryParsers.Flexible.Messages.NLS](xref:Lucene.Net.QueryParsers.Flexible.Messages.NLS) API. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Core/Nodes/package.md b/src/Lucene.Net.QueryParser/Flexible/Core/Nodes/package.md index 0ee330f..7c7a6e1 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Core/Nodes/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Core/Nodes/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Core.Nodes summary: *content --- @@ -25,18 +25,45 @@ Query nodes commonly used by query parser implementations. ## Query Nodes - The package <tt>org.apache.lucene.queryParser.nodes</tt> contains all the basic query nodes. The interface that represents a query node is <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode>. +The namespace <tt>Lucene.Net.QueryParsers.Flexible.Core.Nodes</tt> contains all the basic query nodes. The interface that represents a query node is <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode>. + +<xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode>s are used by the text parser to create a syntax tree. These nodes are designed to be used by UI or other text parsers. The default Lucene text parser is <xref:Lucene.Net.QueryParsers.Flexible.Standard.Parser.StandardSyntaxParser>, it implements Lucene's standard syntax. + +<xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> interface should be implemented by all query nodes, the class <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> implements <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> and is extended by all current query node implementations. + +A query node tree can be printed to the a stream, and it generates a pseudo XML representation with all the nodes. - <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode>s are used by the text parser to create a syntax tree. These nodes are designed to be used by UI or other text parsers. The default Lucene text parser is <xref:Lucene.Net.QueryParsers.Flexible.Standard.Parser.StandardSyntaxParser>, it implements Lucene's standard syntax. +A query node tree can also generate a query string that can be parsed back by the original text parser, at this point only the standard lucene syntax is supported. - <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> interface should be implemented by all query nodes, the class <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNodeImpl> implements <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> and is extended by all current query node implementations. +Grouping nodes: - A query node tree can be printed to the a stream, and it generates a pseudo XML representation with all the nodes. +* AndQueryNode - used for AND operator +* AnyQueryNode - used for ANY operator +* OrQueryNode - used for OR operator +* BooleanQueryNode - used when no operator is specified +* ModifierQueryNode - used for modifier operator +* GroupQueryNode - used for parenthesis +* BoostQueryNode - used for boost operator +* SlopQueryNode - phrase slop +* FuzzyQueryNode - fuzzy node +* TermRangeQueryNode - used for parametric field:`[low_value TO high_value]` +* ProximityQueryNode - used for proximity search +* NumericRangeQueryNode - used for numeric range search +* TokenizedPhraseQueryNode - used by tokenizers/lemmatizers/analyzers for phrases/autophrases - A query node tree can also generate a query string that can be parsed back by the original text parser, at this point only the standard lucene syntax is supported. + Leaf Nodes: - Grouping nodes: * AndQueryNode - used for AND operator * AnyQueryNode - used for ANY operator * OrQueryNode - used for OR operator * BooleanQueryNode - used when no operator is specified * ModifierQueryNode - used for modifier operator * GroupQueryNode - used for parenthesis * BoostQueryNode - used for boost operator * SlopQueryNode - phrase slop * FuzzyQueryNode - fuzzy node * TermRangeQueryNode - used for parametric field:[low_value TO high_value] * ProximityQueryNode - used for proxi [...] +* FieldQueryNode - field/value node +* NumericQueryNode - used for numeric search +* PathQueryNode - <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> object used with path-like queries +* OpaqueQueryNode - Used as for part of the query that can be parsed by other parsers. schema/value +* PrefixWildcardQueryNode - non-phrase wildcard query +* QuotedFieldQUeryNode - regular phrase node +* WildcardQueryNode - non-phrase wildcard query - Leaf Nodes: * FieldQueryNode - field/value node * NumericQueryNode - used for numeric search * PathQueryNode - <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> object used with path-like queries * OpaqueQueryNode - Used as for part of the query that can be parsed by other parsers. schema/value * PrefixWildcardQueryNode - non-phrase wildcard query * QuotedFieldQUeryNode - regular phrase node * WildcardQueryNode - non-phrase wildcard query + Utility Nodes: - Utility Nodes: * DeletedQueryNode - used by processors on optimizations * MatchAllDocsQueryNode - used by processors on optimizations * MatchNoDocsQueryNode - used by processors on optimizations * NoTokenFoundQueryNode - used by tokenizers/lemmatizers/analyzers \ No newline at end of file +* DeletedQueryNode - used by processors on optimizations +* MatchAllDocsQueryNode - used by processors on optimizations +* MatchNoDocsQueryNode - used by processors on optimizations +* NoTokenFoundQueryNode - used by tokenizers/lemmatizers/analyzers \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Core/Parser/package.md b/src/Lucene.Net.QueryParser/Flexible/Core/Parser/package.md index 92b55d0..3717d12 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Core/Parser/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Core/Parser/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Core.Parser summary: *content --- @@ -25,4 +25,7 @@ Necessary interfaces to implement text parsers. ## Parser - The package <tt>org.apache.lucene.queryparser.flexible.parser</tt> contains interfaces that should be implemented by the parsers. Parsers produce QueryNode Trees from a string object. These package still needs some work to add support to for multiple parsers. Features that should be supported for the future, related with the parser: - QueryNode tree should be able convertible to any parser syntax. - The query syntax should support calling other parsers. - QueryNode tree created by multi [...] \ No newline at end of file +The namespace <tt>Lucene.Net.QueryParsers.Flexible.Core.Parser</tt> contains interfaces that should be implemented by the parsers. Parsers produce IQueryNode Trees from a string object. This package still needs some work to add support to for multiple parsers. Features that should be supported for the future, related with the parser: + +- QueryNode tree should be able convertible to any parser syntax. +- The query syntax should support calling other parsers. - QueryNode tree created by multiple parsers. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Core/Processors/package.md b/src/Lucene.Net.QueryParser/Flexible/Core/Processors/package.md index 1d45eff..c7429f7 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Core/Processors/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Core/Processors/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Core.Processors summary: *content --- @@ -25,14 +25,14 @@ Interfaces and implementations used by query node processors ## Query Node Processors - The package <tt>org.apache.lucene.queryParser.processors</tt> contains interfaces that should be implemented by every query node processor. +The namespace <tt>Lucene.Net.QueryParsers.Flexible.Core.Processors</tt> contains interfaces that should be implemented by every query node processor. - The interface that every query node processor should implement is <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.QueryNodeProcessor>. +The interface that every query node processor should implement is <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.IQueryNodeProcessor>. - A query node processor should be used to process a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> tree. <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> trees can be programmatically created or generated by a text parser. See <xref:Lucene.Net.QueryParsers.Flexible.Core.Parser> for more details about text parsers. +A query node processor should be used to process a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> tree. <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> trees can be programmatically created or generated by a text parser. See <xref:Lucene.Net.QueryParsers.Flexible.Core.Parser> for more details about text parsers. - A query node processor should be used to process a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> tree. <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> trees can be programmatically created or generated by a text parser. See <xref:Lucene.Net.QueryParsers.Flexible.Core.Parser> for more details about text parsers. +A query node processor should be used to process a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> tree. <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> trees can be programmatically created or generated by a text parser. See <xref:Lucene.Net.QueryParsers.Flexible.Core.Parser> for more details about text parsers. - A pipeline of processors can be assembled using <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.QueryNodeProcessorPipeline>. +A pipeline of processors can be assembled using <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.QueryNodeProcessorPipeline>. - Implementors may want to extend <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.QueryNodeProcessorImpl>, which simplifies the implementation, because it walks automatically the <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode>. See <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.QueryNodeProcessorImpl> for more details. \ No newline at end of file +Implementors may want to extend <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.QueryNodeProcessor>, which simplifies the implementation, because it walks automatically the <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode>. See <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.QueryNodeProcessor> for more details. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Core/Util/package.md b/src/Lucene.Net.QueryParser/Flexible/Core/Util/package.md index b6464d7..877cac4 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Core/Util/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Core/Util/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Core.Util summary: *content --- @@ -24,4 +24,4 @@ Utility classes to used with the Query Parser. ## Utility classes to used with the Query Parser - This package contains utility classes used with the query parsers. \ No newline at end of file +This namespace contains utility classes used with the query parsers. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Core/package.md b/src/Lucene.Net.QueryParser/Flexible/Core/package.md index 95f2b40..3672604 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Core/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Core/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Core summary: *content --- @@ -25,18 +25,18 @@ Core classes of the flexible query parser framework. ## Flexible Query Parser - This package contains the necessary classes to implement a query parser. +This namespace contains the necessary classes to implement a query parser. - A query parser is divided in at least 2 phases, text parsing and query building, and one optional phase called query processing. +A query parser is divided in at least 2 phases, text parsing and query building, and one optional phase called query processing. ### First Phase: Text Parsing - The text parsing phase is performed by a text parser, which implements <xref:Lucene.Net.QueryParsers.Flexible.Core.Parser.SyntaxParser> interface. A text parser is responsible to get a query string and convert it to a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> tree, which is an object structure that represents the elements defined in the query string. +The text parsing phase is performed by a text parser, which implements <xref:Lucene.Net.QueryParsers.Flexible.Core.Parser.ISyntaxParser> interface. A text parser is responsible to get a query string and convert it to a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> tree, which is an object structure that represents the elements defined in the query string. ### Second (optional) Phase: Query Processing - The query processing phase is performed by a query processor, which implements <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.QueryNodeProcessor>. A query processor is responsible to perform any processing on a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> tree. This phase is optional and is used only if an extra processing, validation, query expansion, etc needs to be performed in a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> tree. The <xref:Luce [...] +The query processing phase is performed by a query processor, which implements <xref:Lucene.Net.QueryParsers.Flexible.Core.Processors.QueryNodeProcessor>. A query processor is responsible to perform any processing on a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> tree. This phase is optional and is used only if an extra processing, validation, query expansion, etc needs to be performed in a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> tree. The <xref:Luc [...] ### Third Phase: Query Building - The query building phase is performed by a query builder, which implements <xref:Lucene.Net.QueryParsers.Flexible.Core.Builders.QueryBuilder>. A query builder is responsible to convert a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> tree into an arbitrary object, which is usually used to be executed against a search index. \ No newline at end of file +The query building phase is performed by a query builder, which implements <xref:Lucene.Net.QueryParsers.Flexible.Core.Builders.IQueryBuilder{Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode}>. A query builder is responsible to convert a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> tree into an arbitrary object, which is usually used to be executed against a search index. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Messages/package.md b/src/Lucene.Net.QueryParser/Flexible/Messages/package.md index 60a1fe4..876bbda 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Messages/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Messages/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Messages summary: *content --- @@ -25,12 +25,61 @@ For Native Language Support (NLS), system of software internationalization. ## NLS message API - This utility API, adds support for NLS messages in the apache code. It is currently used by the lucene "New Flexible Query PArser". +This utility API, adds support for NLS messages in the apache code. It is currently used by the lucene "New Flexible Query PArser". + +Features: + +1. Message reference in the code, using static Strings +2. Message resource validation at class load time, for easier debugging +3. Allows for message IDs to be re-factored using code re-factor tools +4. Allows for reference count on messages, just like code +5. Lazy loading of Message Strings +6. Normal loading Message Strings + + +Prerequisite for these examples: Add a resource file named `MessagesTestBundle.resx` and add messages for each of the public static string fields except for `Q0005E_Message_Not_In_Bundle`. + +Lazy loading of Message Strings + +```cs +public class MessagesTest : NLS +{ + private static readonly string BundleName = typeof(MessagesTest).FullName; + + private MessagesTest() + { + // should never be instantiated + } + + static MessagesTest() + { + InitializeMessages(BundleName, typeof(MessagesTest)); + } + + // static string must match the strings in the property files. + public static string Q0001E_Invalid_Syntax; + public static string Q0004E_Invalid_Syntax_Escape_Unicode_Truncation; + + // this message is missing from the properties file + public static string Q0005E_Message_Not_In_Bundle; +} + +// Create a message reference +IMessage invalidSyntax = new Message(MessagesTest.Q0001E_Invalid_Syntax, "XXX"); + +// Do other stuff in the code... +// when is time to display the message to the user or log the message on a file +// the message is loaded from the correct bundle - Features: 1. Message reference in the code, using static Strings 2. Message resource validation at class load time, for easier debugging 3. Allows for message IDs to be re-factored using eclipse or other code re-factor tools 4. Allows for reference count on messages, just like code 5. Lazy loading of Message Strings 6. Normal loading Message Strings +string message1 = invalidSyntax.GetLocalizedMessage(); +string message2 = invalidSyntax.GetLocalizedMessage(new CultureInfo("ja")); +``` - Lazy loading of Message Strings public class MessagesTestBundle extends NLS { private static final String BUNDLE_NAME = MessagesTestBundle.class.getName(); private MessagesTestBundle() { // should never be instantiated } static { // register all string ids with NLS class and initialize static string // values NLS.initializeMessages(BUNDLE_NAME, MessagesTestBundle.class); } // static string must match the strings in the property files. public static String Q0001E_INVALID_SYNTAX; public s [...] +Normal loading of Message Strings - Normal loading of Message Strings String message1 = NLS.getLocalizedMessage(MessagesTestBundle.Q0004E_INVALID_SYNTAX_ESCAPE_UNICODE_TRUNCATION); String message2 = NLS.getLocalizedMessage(MessagesTestBundle.Q0004E_INVALID_SYNTAX_ESCAPE_UNICODE_TRUNCATION, Locale.JAPANESE); +```cs +string message1 = NLS.GetLocalizedMessage(MessagesTest.Q0004E_Invalid_Syntax_Escape_Unicode_Truncation); +string message2 = NLS.GetLocalizedMessage(MessagesTest.Q0004E_Invalid_Syntax_Escape_Unicode_Truncation, new CultureInfo("ja")); +``` - The org.apache.lucene.messages.TestNLS junit contains several other examples. The TestNLS java code is available from the Apache Lucene code repository. \ No newline at end of file +The `Lucene.Net.QueryParsers.Flexible.Messages.TestNLS` NUnit test contains several other examples. The TestNLS C# code is available from the Apache Lucene.NET code repository. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Standard/Builders/package.md b/src/Lucene.Net.QueryParser/Flexible/Standard/Builders/package.md index 108d1d6..2042af5 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Standard/Builders/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Standard/Builders/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Standard.Builders summary: *content --- @@ -25,6 +25,6 @@ Standard Lucene Query Node Builders. ## Standard Lucene Query Node Builders - The package org.apache.lucene.queryparser.flexible.standard.builders contains all the builders needed to build a Lucene Query object from a query node tree. These builders expect the query node tree was already processed by the <xref:Lucene.Net.QueryParsers.Flexible.Standard.Processors.StandardQueryNodeProcessorPipeline>. +The namespace <tt>Lucene.Net.QueryParsers.Flexible.Standard.Builders</tt> contains all the builders needed to build a Lucene Query object from a query node tree. These builders expect the query node tree was already processed by the <xref:Lucene.Net.QueryParsers.Flexible.Standard.Processors.StandardQueryNodeProcessorPipeline>. - <xref:Lucene.Net.QueryParsers.Flexible.Standard.Builders.StandardQueryTreeBuilder> is a builder that already contains a defined map that maps each QueryNode object with its respective builder. \ No newline at end of file +<xref:Lucene.Net.QueryParsers.Flexible.Standard.Builders.StandardQueryTreeBuilder> is a builder that already contains a defined map that maps each QueryNode object with its respective builder. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Standard/Config/package.md b/src/Lucene.Net.QueryParser/Flexible/Standard/Config/package.md index af925d2..bd8b463 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Standard/Config/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Standard/Config/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Standard.Config summary: *content --- @@ -25,6 +25,6 @@ Standard Lucene Query Configuration. ## Standard Lucene Query Configuration - The package org.apache.lucene.queryparser.flexible.standard.config contains the Lucene query configuration handler (StandardQueryConfigHandler). This configuration handler reproduces almost everything that could be set on the old query parser. +The namespace <tt>Lucene.Net.QueryParsers.Flexible.Standard.Config</tt> contains the Lucene query configuration handler (StandardQueryConfigHandler). This configuration handler reproduces almost everything that could be set on the old query parser. - StandardQueryConfigHandler is the class that should be used to configure the StandardQueryNodeProcessorPipeline. \ No newline at end of file +StandardQueryConfigHandler is the class that should be used to configure the StandardQueryNodeProcessorPipeline. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Standard/Nodes/package.md b/src/Lucene.Net.QueryParser/Flexible/Standard/Nodes/package.md index 1d29ff7..8a9e8d8 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Standard/Nodes/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Standard/Nodes/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Standard.Nodes summary: *content --- @@ -25,4 +25,4 @@ Standard Lucene Query Nodes. ## Standard Lucene Query Nodes - The package org.apache.lucene.queryparser.flexible.standard.nodes contains QueryNode classes that are used specifically for Lucene query node tree. Any other generic QueryNode is defined under org.apache.lucene.queryParser.nodes. \ No newline at end of file +The namespace <tt>Lucene.Net.QueryParsers.Flexible.Standard.Nodes</tt> contains QueryNode classes that are used specifically for Lucene query node tree. Any other generic QueryNode is defined under <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes>. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Standard/Parser/package.md b/src/Lucene.Net.QueryParser/Flexible/Standard/Parser/package.md index 0a1b41d..61c1da8 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Standard/Parser/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Standard/Parser/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Standard.Parser summary: *content --- @@ -25,6 +25,6 @@ Lucene Query Parser. ## Lucene Query Parser - The package org.apache.lucene.queryparser.flexible.standard.parser contains the query parser. +The namespace <tt>Lucene.Net.QueryParsers.Flexible.Standard.Parser</tt> contains the query parser. - This text parser only performs the syntax validation and creates an QueryNode tree from a query string. \ No newline at end of file +This text parser only performs the syntax validation and creates an IQueryNode tree from a query string. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Standard/Processors/package.md b/src/Lucene.Net.QueryParser/Flexible/Standard/Processors/package.md index 93983aa..c50ffa4 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Standard/Processors/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Standard/Processors/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Standard.Processors summary: *content --- @@ -25,6 +25,6 @@ Lucene Query Node Processors. ## Lucene Query Node Processors - The package org.apache.lucene.queryparser.flexible.standard.processors contains every processor needed to assembly a pipeline that modifies the query node tree according to the actual Lucene queries. +The namespace <tt>Lucene.Net.QueryParsers.Flexible.Standard.Processors</tt> contains every processor needed to assembly a pipeline that modifies the query node tree according to the actual Lucene queries. - These processors are already assembled correctly in the StandardQueryNodeProcessorPipeline. \ No newline at end of file +These processors are already assembled correctly in the StandardQueryNodeProcessorPipeline. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Flexible/Standard/package.md b/src/Lucene.Net.QueryParser/Flexible/Standard/package.md index 645df6c..1ef8303 100644 --- a/src/Lucene.Net.QueryParser/Flexible/Standard/package.md +++ b/src/Lucene.Net.QueryParser/Flexible/Standard/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Flexible.Standard summary: *content --- @@ -21,12 +21,12 @@ summary: *content --> -Implementation of the {@linkplain org.apache.lucene.queryparser.classic Lucene classic query parser} using the flexible query parser frameworks +Implementation of the [Lucene classic query parser](xref:Lucene.Net.QueryParsers.Classic) using the flexible query parser frameworks ## Lucene Flexible Query Parser Implementation - The old Lucene query parser used to have only one class that performed all the parsing operations. In the new query parser structure, the parsing was divided in 3 steps: parsing (syntax), processing (semantic) and building. +The old Lucene query parser used to have only one class that performed all the parsing operations. In the new query parser structure, the parsing was divided in 3 steps: parsing (syntax), processing (semantic) and building. - The classes contained in the package org.apache.lucene.queryParser.standard are used to reproduce the same behavior as the old query parser. +The classes contained in the namespace <xref:Lucene.Net.QueryParsers.Flexible.Standard> are used to reproduce the same behavior as the old query parser. - Check <xref:Lucene.Net.QueryParsers.Flexible.Standard.StandardQueryParser> to quick start using the Lucene query parser. \ No newline at end of file +Check <xref:Lucene.Net.QueryParsers.Flexible.Standard.StandardQueryParser> to quick start using the Lucene query parser. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Simple/package.md b/src/Lucene.Net.QueryParser/Simple/package.md index 4739b1d..0e74bca 100644 --- a/src/Lucene.Net.QueryParser/Simple/package.md +++ b/src/Lucene.Net.QueryParser/Simple/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Simple summary: *content --- @@ -20,4 +20,4 @@ summary: *content limitations under the License. --> - A simple query parser for human-entered queries. \ No newline at end of file +A simple query parser for human-entered queries. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/Xml/package.md b/src/Lucene.Net.QueryParser/Xml/package.md index 66dcaf2..b996c6c 100644 --- a/src/Lucene.Net.QueryParser/Xml/package.md +++ b/src/Lucene.Net.QueryParser/Xml/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParsers.Xml summary: *content --- @@ -20,4 +20,4 @@ summary: *content limitations under the License. --> -Parser that produces Lucene Query objects from XML streams. \ No newline at end of file +Parser that produces Lucene Query objects from XML documents. \ No newline at end of file diff --git a/src/Lucene.Net.QueryParser/overview.md b/src/Lucene.Net.QueryParser/overview.md index 2576230..ad4d3c7 100644 --- a/src/Lucene.Net.QueryParser/overview.md +++ b/src/Lucene.Net.QueryParser/overview.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.QueryParser title: Lucene.Net.QueryParser summary: *content @@ -21,47 +21,50 @@ summary: *content limitations under the License. --> - Apache Lucene QueryParsers. +Apache Lucene QueryParsers. - This module provides a number of queryparsers: +This module provides a number of queryparsers: -* [Classic](#classic) +* [Classic](#classicxreflucenenetqueryparsersclassic) -* [Analyzing](#analyzing) +* [Analyzing](#analyzingxreflucenenetqueryparsersanalyzing) -* [Complex Phrase](#complexphrase) +* [Complex Phrase](#complex-phrasexreflucenenetqueryparserscomplexphrase) -* [Extendable](#extendable) +* [Extendable](#extendablexreflucenenetqueryparsersext) * [Flexible](#flexible) * [Surround](#surround) -* [XML](#xml) +* [XML](#xmlxreflucenenetqueryparsersxml) * * * -## [Classic]() +## [Classic](xref:Lucene.Net.QueryParsers.Classic) + +A Simple Lucene QueryParser implemented with JavaCC. - A Simple Lucene QueryParser implemented with JavaCC. +## [Analyzing](xref:Lucene.Net.QueryParsers.Analyzing) -## [Analyzing]() +QueryParser that passes Fuzzy-, Prefix-, Range-, and WildcardQuerys through the given analyzer. - QueryParser that passes Fuzzy-, Prefix-, Range-, and WildcardQuerys through the given analyzer. +## [Complex Phrase](xref:Lucene.Net.QueryParsers.ComplexPhrase) -## [Complex Phrase]() +QueryParser which permits complex phrase query syntax eg "(john jon jonathan~) peters*" - QueryParser which permits complex phrase query syntax eg "(john jon jonathan~) peters*" +## [Extendable](xref:Lucene.Net.QueryParsers.Ext) -## [Extendable]() +Extendable QueryParser provides a simple and flexible extension mechanism by overloading query field names. - Extendable QueryParser provides a simple and flexible extension mechanism by overloading query field names. +## Flexible -## [Flexible]() +This project contains the new Lucene query parser implementation, which matches the syntax of the core QueryParser but offers a more modular architecture to enable customization. - This project contains the new Lucene query parser implementation, which matches the syntax of the core QueryParser but offers a more modular architecture to enable customization. +It's currently divided in 2 main namespaces: - It's currently divided in 2 main packages: * <xref:Lucene.Net.QueryParsers.Flexible.Core>: it contains the query parser API classes, which should be extended by query parser implementations. * <xref:Lucene.Net.QueryParsers.Flexible.Standard>: it contains the current Lucene query parser implementation using the new query parser API. +* <xref:Lucene.Net.QueryParsers.Flexible.Core>: it contains the query parser API classes, which should be extended by query parser implementations. +* <xref:Lucene.Net.QueryParsers.Flexible.Standard>: it contains the current Lucene query parser implementation using the new query parser API. ### Features @@ -82,7 +85,7 @@ summary: *content 6. Standard Builders - convert QueryNode's into several lucene representations. Supported conversion is using a 2.4 compatible logic -7. QueryNode tree's can be converted to a lucene 2.4 syntax string, using toQueryString +7. QueryNode tree's can be converted to a lucene 2.4 syntax string, using ToQueryString() ### Design @@ -102,8 +105,8 @@ summary: *content <dt>QueryParser</dt> <dd> This layer is the text parsing layer which simply transforms the -query text string into a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> tree. Every text parser -must implement the interface <xref:Lucene.Net.QueryParsers.Flexible.Core.Parser.SyntaxParser>. +query text string into a <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> tree. Every text parser +must implement the interface <xref:Lucene.Net.QueryParsers.Flexible.Core.Parser.ISyntaxParser>. Lucene default implementations implements it using JavaCC. </dd> @@ -117,15 +120,15 @@ terms. <dt>QueryBuilder</dt> <dd> -The third layer is a configurable map of builders, which map <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.QueryNode> types to its specific -builder that will transform the QueryNode into Lucene Query object. +The third layer is a configurable map of builders, which map <xref:Lucene.Net.QueryParsers.Flexible.Core.Nodes.IQueryNode> types to its specific +builder that will transform the IQueryNode into Lucene Query object. </dd> </dl> - Furthermore, the query parser uses flexible configuration objects. It also uses message classes that allow to attach resource bundles. This makes it possible to translate messages, which is an important feature of a query parser. +Furthermore, the query parser uses flexible configuration objects. It also uses message classes that allow to attach resource bundles. This makes it possible to translate messages, which is an important feature of a query parser. - This design allows to develop different query syntaxes very quickly. +This design allows to develop different query syntaxes very quickly. ### StandardQueryParser and QueryParserWrapper @@ -140,16 +143,25 @@ you don't need to worry about dealing with those. <xref:Lucene.Net.QueryParsers.Flexible.Standard.StandardQueryParser> usage: - StandardQueryParser qpHelper = new StandardQueryParser(); - StandardQueryConfigHandler config = qpHelper.getQueryConfigHandler(); - config.setAllowLeadingWildcard(true); - config.setAnalyzer(new WhitespaceAnalyzer()); - Query query = qpHelper.parse("apache AND lucene", "defaultField"); -## [Surround]() +```cs +const LuceneVersion matchVersion = LuceneVersion.LUCENE_48; +StandardQueryParser qpHelper = new StandardQueryParser(); +QueryConfigHandler config = qpHelper.QueryConfigHandler; +config.Set(ConfigurationKeys.ALLOW_LEADING_WILDCARD, true); +config.Set(ConfigurationKeys.ANALYZER, new WhitespaceAnalyzer(matchVersion)); +Query query = qpHelper.Parse("apache AND lucene", "defaultField"); +``` + +## Surround + +A QueryParser that supports the Span family of queries as well as pre and infix notation. + +It's divided in 2 main namespaces: - A QueryParser that supports the Span family of queries as well as pre and infix notation. +* <xref:Lucene.Net.QueryParsers.Surround.Parser> +* <xref:Lucene.Net.QueryParsers.Surround.Query> -## [XML]() +## [XML](xref:Lucene.Net.QueryParsers.Xml) -A QueryParser that produces Lucene Query objects from XML streams. \ No newline at end of file +A QueryParser that produces Lucene Query objects from XML documents. \ No newline at end of file diff --git a/src/Lucene.Net.Replicator/Http/package.md b/src/Lucene.Net.Replicator/Http/package.md index 4504188..fd9797b 100644 --- a/src/Lucene.Net.Replicator/Http/package.md +++ b/src/Lucene.Net.Replicator/Http/package.md @@ -1,4 +1,4 @@ ---- +--- uid: Lucene.Net.Replicator.Http summary: *content --- @@ -21,4 +21,4 @@ summary: *content limitations under the License. --> -# HTTP replication implementation +HTTP replication implementation diff --git a/src/Lucene.Net.Replicator/Http/replicator.md b/src/Lucene.Net.Replicator/Http/replicator.md deleted file mode 100644 index 0de6006..0000000 --- a/src/Lucene.Net.Replicator/Http/replicator.md +++ /dev/null @@ -1,162 +0,0 @@ -# Using the ReplicatorService - -Because therer are a number of different hosting frameworks to choose from on .NET, and that they don't implement common -interfaceses or base classes for requests and responses, the ReplicatorService instead provides abstractions so that it can -be integrated easily into any of these framworks. - -So to integrate the replicator into any existing hosting framworks, the IReplicationRequest and IReplicationResponse interfaces -must be implemented for a choosen framwork. - -## An AspNetCore Implementation - -Below is an example of how these wrappers can be implemented for the AspNetCore framwork. -The example only convers the absolutely minimum needed in order for it to become functional within AspNetCore. - -It does not go as far as to implement custom middleware, action results for controllers or anything else, while this -would be a natural to do, such implementations extends beyond the scope of this document. - -#### AspNetCore Request Wrapper - -The first thing to do is to wrap the AspNetCore Request object in a class that implements the IReplicationRequest interface. -This is very straight forward. - -```csharp -// Wrapper class for the Microsoft.AspNetCore.Http.HttpRequest -public class AspNetCoreReplicationRequest : IReplicationRequest -{ - private readonly HttpRequest request; - - // Inject the actual request object in the constructor. - public AspNetCoreReplicationRequest(HttpRequest request) - => this.request = request; - - // Provide the full path relative to the host. - // In the common case in AspNetCore this should just return the full path, so PathBase + Path are concatenated and returned. - // - // The path expected by the ReplicatorService is {context}/{shard}/{action} where: - // - action may be Obtain, Release or Update - // - context is the same context that is provided to the ReplicatorService constructor and defaults to '/replicate' - public string Path - => request.PathBase + request.Path; - - // Return values for parameters used by the ReplicatorService - // The ReplicatorService will call this with: - // - version: The index revision - // - sessionid: The ID of the session - // - source: The source index for the files - // - filename: The file name - // - // In this implementation a exception is thrown in the case that parameters are provided multiple times. - public string QueryParam(string name) - => request.Query[name].SingleOrDefault(); -} -``` - -#### AspNetCore Response Wrapper - -Secondly the AspNetCore Response object is wrapped in a class that implements the IReplicationResponse interface. -This is also very straight forward. - -```csharp -// Wrapper class for the Microsoft.AspNetCore.Http.HttpResponse -public class AspNetCoreReplicationResponse : IReplicationResponse -{ - private readonly HttpResponse response; - - // Inject the actual response object in the constructor. - public AspNetCoreReplicationResponse(HttpResponse response) - => this.response = response; - - // Getter and Setter for the http Status code, in case of failure the ReplicatorService will set this - // Property. - public int StatusCode - { - get => response.StatusCode; - set => response.StatusCode = value; - } - - // Return a stream where the ReplicatorService can write to for the response. - // Depending on the action either the file or the sesssion token will be written to this stream. - public Stream Body => response.Body; - - // Called when the ReplicatorService is done writing data to the response. - // Here it is mapped to the flush method on the "body" stream on the response. - public void Flush() => response.Body.Flush(); -} -``` - -#### AspNetCore Utility Extension Method - -This part is not nessesary, however by providing a extension method as a overload to the ReplicatorService Perform method -that instead takes the AspNetCore HttpRequest and HttpResponse response objects, it's easier to call the ReplicatorService -from either AspNetCore Mvc controllers, inside of middleare or for the absolute minimal solution directly in the delegate of -a IApplicationBuilder.Run method. - -```csharp -public static class AspNetCoreReplicationServiceExtentions -{ - // Optionally, provide a extension method for calling the perform method directly using the specific request - // and response objects from AspNetCore - public static void Perform(this ReplicationService self, HttpRequest request, HttpResponse response) - => self.Perform( - new AspNetCoreReplicationRequest(request), - new AspNetCoreReplicationResponse(response)); -} -``` - -## Using the Implementation - -Now the implementation can be used wihin AspNetCore in order to service Lucene Replicator requests over http. - -In order to enable replication of indexes, the IndewWriter that writes the index should be created with a `SnapshotDeletionPolicy`. - -```csharp -IndexWriterConfig config = new IndexWriterConfig(...ver..., new StandardAnalyzer(...ver...)); -config.IndexDeletionPolicy = new SnapshotDeletionPolicy(config.IndexDeletionPolicy); -IndexWriter writer = new IndexWriter(FSDirectory.Open("..."), config); -``` - -For the absolute minimal solution we can wire the ReplicatorService up on the server side as: - -```csharp -LocalReplicator replicator = new LocalReplicator(); -ReplicatorService service = new ReplicationService(new Dictionary<string, IReplicator>{ - ["shard_name"] = replicator -}, "/api/replicate"); - -app.Map("/api/replicate", builder => { - builder.Run(async context => { - await Task.Yield(); - service.Perform(context.Request, context.Response); - }); -}); -``` - -Now in order to publish a Revision call the publish method in the LocalReplicator: - -```csharp -IndexWriter writer = ...; -LocalReplicator replicator = ...; -replicator.Publish(new IndexRevision(writer)); -``` - -On the client side create a new HttpReplicator and start replicating, e.g.: - -```csharp -IReplicator replicator = new HttpReplicator("http://{host}:{port}/api/replicate/shard_name";); -ReplicationClient client = new ReplicationClient( - replicator, - new IndexReplicationHandler( - FSDirectory.Open(...directory...), - () => ...onUpdate...), - new PerSessionDirectoryFactory(...temp-working-directory...)); - -//Now either start the Update Thread or do manual pulls periodically. -client.UpdateNow(); //Manual Pull -client.StartUpdateThread(1000, "Replicator Thread"); //Pull automatically every second if there is any changes. -``` - -From here it would be natural to use a SearchManager over the directory in order to get Searchers updated outomatically. -But this cannot be created before the first actual replication as the SearchManager will fail because there is no index. - -We can use the onUpdate handler to perform the first initialization in this case. \ No newline at end of file diff --git a/src/Lucene.Net.Replicator/package.md b/src/Lucene.Net.Replicator/package.md index 5d72da3..6db5ab5 100644 --- a/src/Lucene.Net.Replicator/package.md +++ b/src/Lucene.Net.Replicator/package.md @@ -1,50 +1,35 @@ ---- +--- uid: Lucene.Net.Replicator summary: *content --- -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> - # Files replication framework The -[Replicator](Replicator.html) allows replicating files between a server and client(s). Producers publish -[revisions](Revision.html) and consumers update to the latest revision available. -[ReplicationClient](ReplicationClient.html) is a helper utility for performing the update operation. It can +[Replicator](xref:Lucene.Net.Replicator.IReplicator) allows replicating files between a server and client(s). Producers publish +[revisions](xref:Lucene.Net.Replicator.IRevision) and consumers update to the latest revision available. +[ReplicationClient](xref:Lucene.Net.Replicator.ReplicationClient) is a helper utility for performing the update operation. It can be invoked either -[manually](ReplicationClient.html#updateNow()) or periodically by -[starting an update thread](ReplicationClient.html#startUpdateThread(long, java.lang.String)). -[HttpReplicator](http/HttpReplicator.html) can be used to replicate revisions by consumers that reside on +[manually](xref:Lucene.Net.Replicator.ReplicationClient#Lucene_Net_Replicator_ReplicationClient_UpdateNow) or periodically by +[starting an update thread](xref:Lucene.Net.Replicator.ReplicationClient#Lucene_Net_Replicator_ReplicationClient_StartUpdateThread_System_Int64_System_String_). +[HttpReplicator](xref:Lucene.Net.Replicator.Http.HttpReplicator) can be used to replicate revisions by consumers that reside on a different node than the producer. The replication framework supports replicating any type of files, with built-in support for a single search index as well as an index and taxonomy pair. For a single index, the application should publish an -[IndexRevision](IndexRevision.html) and set -[IndexReplicationHandler](IndexReplicationHandler.html) on the client. For an index and taxonomy pair, the -application should publish an [IndexAndTaxonomyRevision](IndexAndTaxonomyRevision.html) and set -[IndexAndTaxonomyReplicationHandler](IndexAndTaxonomyReplicationHandler.html) on the client. +[IndexRevision](xref:Lucene.Net.Replicator.IndexRevision) and set +[IndexReplicationHandler](xref:Lucene.Net.Replicator.IndexReplicationHandler) on the client. For an index and taxonomy pair, the +application should publish an [IndexAndTaxonomyRevision](xref:Lucene.Net.Replicator.IndexAndTaxonomyRevision) and set +[IndexAndTaxonomyReplicationHandler](xref:Lucene.Net.Replicator.IndexAndTaxonomyReplicationHandler) on the client. When the replication client detects that there is a newer revision available, it copies the files of the revision and -then invokes the handler to complete the operation (e.g. copy the files to the index directory, fsync them, reopen an +then invokes the handler to complete the operation (e.g. copy the files to the index directory, sync them, reopen an index reader etc.). By default, only files that do not exist in the handler's -[current revision files](ReplicationClient.ReplicationHandler.html#currentRevisionFiles()) are copied, +[current revision files](xref:Lucene.Net.Replicator.IReplicationHandler.html#Lucene_Net_Replicator_IReplicationHandler_CurrentRevisionFiles) are copied, however this can be overridden by extending the client. +<!-- Old Code Sample - not sure whether this is useful +```cs An example usage of the Replicator: // ++++++++++++++ SERVER SIDE ++++++++++++++ // @@ -66,4 +51,167 @@ ReplicationClient client = new ReplicationClient(replicator, handler, factory); client.updateNow(); // or, periodically -client.startUpdateThread(100); // check for update every 100 milliseconds \ No newline at end of file +client.startUpdateThread(100); // check for update every 100 milliseconds +``` +--> + +# Using the ReplicatorService + +Because there are a number of different hosting frameworks to choose from on .NET and they don't implement common +abstractions for requests and responses, the ReplicatorService provides abstractions so that it can +be integrated easily into any of these frameworks. + +To integrate the replicator into an existing hosting framework, the <xref:Lucene.Net.Replicator.Http.Abstractions.IReplicationRequest> and <xref:Lucene.Net.Replicator.Http.Abstractions.IReplicationResponse> interfaces must be implemented for the chosen framework. + +## An ASP.NET Core Implementation + +Below is an example of how these wrappers can be implemented for the ASP.NET Core framework. +The example only covers the absolute minimum needed in order for it to become functional within ASP.NET Core. + +It does not go as far as to implement custom middleware, action results for controllers or anything else, while this +would be a natural to do, such implementations extends beyond the scope of this document. + +#### ASP.NET Core Request Wrapper + +The first thing to do is to wrap the ASP.NET Core Request object in a class that implements the <xref:Lucene.Net.Replicator.Http.Abstractions.IReplicationRequest> interface. +This is very straight forward. + +```cs +// Wrapper class for the Microsoft.AspNetCore.Http.HttpRequest +public class AspNetCoreReplicationRequest : IReplicationRequest +{ + private readonly HttpRequest request; + + // Inject the actual request object in the constructor. + public AspNetCoreReplicationRequest(HttpRequest request) + => this.request = request; + + // Provide the full path relative to the host. + // In the common case in AspNetCore this should just return the full path, so PathBase + Path are concatenated and returned. + // + // The path expected by the ReplicatorService is {context}/{shard}/{action} where: + // - action may be Obtain, Release or Update + // - context is the same context that is provided to the ReplicatorService constructor and defaults to '/replicate' + public string Path + => request.PathBase + request.Path; + + // Return values for parameters used by the ReplicatorService + // The ReplicatorService will call this with: + // - version: The index revision + // - sessionid: The ID of the session + // - source: The source index for the files + // - filename: The file name + // + // In this implementation a exception is thrown in the case that parameters are provided multiple times. + public string QueryParam(string name) + => request.Query[name].SingleOrDefault(); +} +``` + +#### ASP.NET Core Response Wrapper + +Secondly the ASP.NET Core Response object is wrapped in a class that implements the <xref:Lucene.Net.Replicator.Http.Abstractions.IReplicationResponse> interface. +This is also very straight forward. + +```cs +// Wrapper class for the Microsoft.AspNetCore.Http.HttpResponse +public class AspNetCoreReplicationResponse : IReplicationResponse +{ + private readonly HttpResponse response; + + // Inject the actual response object in the constructor. + public AspNetCoreReplicationResponse(HttpResponse response) + => this.response = response; + + // Getter and Setter for the http Status code, in case of failure the ReplicatorService will set this + // Property. + public int StatusCode + { + get => response.StatusCode; + set => response.StatusCode = value; + } + + // Return a stream where the ReplicatorService can write to for the response. + // Depending on the action either the file or the sesssion token will be written to this stream. + public Stream Body => response.Body; + + // Called when the ReplicatorService is done writing data to the response. + // Here it is mapped to the flush method on the "body" stream on the response. + public void Flush() => response.Body.Flush(); +} +``` + +#### ASP.NET Core Utility Extension Method + +This part is not nessesary, however by providing a extension method as a overload to the ReplicatorService Perform method +that instead takes the ASP.NET Core HttpRequest and HttpResponse response objects, it's easier to call the ReplicatorService +from either ASP.NET Core MVC controllers, inside of middleare or for the absolute minimal solution directly in the delegate parameter of a IApplicationBuilder.Run() method. + +```cs +public static class AspNetCoreReplicationServiceExtentions +{ + // Optionally, provide a extension method for calling the perform method directly using the specific request + // and response objects from AspNetCore + public static void Perform(this ReplicationService self, HttpRequest request, HttpResponse response) + => self.Perform( + new AspNetCoreReplicationRequest(request), + new AspNetCoreReplicationResponse(response)); +} +``` + +## Using the Implementation + +Now the implementation can be used within ASP.NET Core in order to service Lucene.NET Replicator requests over HTTP. + +In order to enable replication of indexes, the <xref:Lucene.Net.Index.IndexWriter> that writes the index should be created with a <xref:Lucene.Net.Index.SnapshotDeletionPolicy>. + +```cs +IndexWriterConfig config = new IndexWriterConfig(...ver..., new StandardAnalyzer(...ver...)); +config.IndexDeletionPolicy = new SnapshotDeletionPolicy(config.IndexDeletionPolicy); +IndexWriter writer = new IndexWriter(FSDirectory.Open("..."), config); +``` + +For the absolute minimal solution we can wire the <xref:Lucene.Net.Replicator.Http.ReplicationService> up on the server side as: + +```cs +LocalReplicator replicator = new LocalReplicator(); +ReplicatorService service = new ReplicationService(new Dictionary<string, IReplicator>{ + ["shard_name"] = replicator +}, "/api/replicate"); + +app.Map("/api/replicate", builder => { + builder.Run(async context => { + await Task.Yield(); + service.Perform(context.Request, context.Response); + }); +}); +``` + +Now in order to publish a [Revision](xref:Lucene.Net.Replicator.IRevision) call the [Publish()](xref:Lucene.Net.Replicator.LocalReplicator#Lucene_Net_Replicator_LocalReplicator_Publish_Lucene_Net_Replicator_IRevision_) method in the <xref:Lucene.Net.Replicator.LocalReplicator>: + +```cs +IndexWriter writer = ...; +LocalReplicator replicator = ...; +replicator.Publish(new IndexRevision(writer)); +``` + +On the client side create a new <xref:Lucene.Net.Replicator.Http.HttpReplicator> and start replicating, e.g.: + +```cs +IReplicator replicator = new HttpReplicator("http://{host}:{port}/api/replicate/shard_name";); +ReplicationClient client = new ReplicationClient( + replicator, + new IndexReplicationHandler( + FSDirectory.Open(...directory...), + () => ...onUpdate...), + new PerSessionDirectoryFactory(...temp-working-directory...)); + +//Now either start the Update Thread or do manual pulls periodically. +client.UpdateNow(); //Manual Pull +client.StartUpdateThread(1000, "Replicator Thread"); //Pull automatically every second if there is any changes. +``` + +From here it would be natural to use a <xref:Lucene.Net.Search.SearcherManager> over the directory in order to get Searchers updated automatically. +But this cannot be created before the first actual replication as the <xref:Lucene.Net.Search.SearcherManager> will fail because there is no index. + +We can use the onUpdate handler to perform the first initialization in this case. diff --git a/websites/apidocs/index.md b/websites/apidocs/index.md index 3b9e45f..a72fdd3 100644 --- a/websites/apidocs/index.md +++ b/websites/apidocs/index.md @@ -60,7 +60,7 @@ on some of the conceptual or inner details of Lucene: - <xref:Lucene.Net.Misc> - Index tools and other miscellaneous code - <xref:Lucene.Net.Queries> - Filters and Queries that add to core Lucene - <xref:Lucene.Net.QueryParser> - Text to Query parsers and parsing framework -- <xref:Lucene.Net.Replicator> Files replication utility +- <xref:Lucene.Net.Replicator> - Files replication utility - <xref:Lucene.Net.Sandbox> - Various third party contributions and new ideas - [Lucene.Net.Spatial](xref:Lucene.Net.Spatial) - Geospatial search - <xref:Lucene.Net.Suggest> - Auto-suggest and Spell-checking support
