This is an automated email from the ASF dual-hosted git repository. dweiss pushed a commit to branch jira/solr-13105-toMerge in repository https://gitbox.apache.org/repos/asf/solr.git
commit 4ab0647e35fa4a9393c90a95ef2252721b2dd669 Author: Joel Bernstein <[email protected]> AuthorDate: Wed Oct 16 10:03:31 2019 -0400 SOLR-13105: The Visual Guide to Streaming Expressions and Math Expressions --- .../solr-ref-guide/src/computational-geometry.adoc | 8 ++++-- solr/solr-ref-guide/src/curve-fitting.adoc | 2 +- solr/solr-ref-guide/src/regression.adoc | 13 +++++++--- solr/solr-ref-guide/src/statistics.adoc | 7 ++++++ solr/solr-ref-guide/src/streaming-expressions.adoc | 29 ++++++++++++++++++++++ solr/solr-ref-guide/src/vector-math.adoc | 9 +++++++ solr/solr-ref-guide/src/visualization.adoc | 2 -- 7 files changed, 61 insertions(+), 9 deletions(-) diff --git a/solr/solr-ref-guide/src/computational-geometry.adoc b/solr/solr-ref-guide/src/computational-geometry.adoc index 50d7ac6..05740b7 100644 --- a/solr/solr-ref-guide/src/computational-geometry.adoc +++ b/solr/solr-ref-guide/src/computational-geometry.adoc @@ -19,6 +19,10 @@ This section of the math expressions user guide covers computational geometry functions. +<<Convex Hull, Convex Hull>> - +<<Visualization, Visualization>> - +<<Enclosing Disk, Enclosing Disk>> + == Convex Hull A convex hull is the smallest convex set of points that encloses a data set. Math expressions has support for computing @@ -32,8 +36,8 @@ set of 2D points. Border visualizations can be useful for understanding where da in relation to the border. In the examples below the `convexHull` function is used -to visualize a border for a set of latitude and longitude points of rat sightings in the NYC311 -complaints database. An investigation of the border around the rat sightings can be done +to visualize a boarder for a set of latitude and longitude points of rat sightings in the nyc311 +complaints database. An investigation of the boarder around the rat sightings can be done to better understand how rats may be entering or exiting the specific region. ==== Scatter Plot diff --git a/solr/solr-ref-guide/src/curve-fitting.adoc b/solr/solr-ref-guide/src/curve-fitting.adoc index 966e888..60f0990 100644 --- a/solr/solr-ref-guide/src/curve-fitting.adoc +++ b/solr/solr-ref-guide/src/curve-fitting.adoc @@ -23,7 +23,7 @@ These functions support constructing a curve through bivariate non-linear data. The `polyfit` function is a general purpose curve fitter used to model the non-linear relationship between two random variables. -The `polyfit` function is passed x- and y-axes and fits a smooth curve to the data. +The `polyfit` function is passed *x* and *y* axes and fits a smooth curve to the data. If only a single array is provided it is treated as the y-axis and a sequence is generated for the x-axis. A third parameter can be added that specifies the degree of the polynomial. If the degree is not provided a 3 degree polynomial is used by default. The higher diff --git a/solr/solr-ref-guide/src/regression.adoc b/solr/solr-ref-guide/src/regression.adoc index 4e3dcad..eb8b6e8 100644 --- a/solr/solr-ref-guide/src/regression.adoc +++ b/solr/solr-ref-guide/src/regression.adoc @@ -18,6 +18,11 @@ The math expressions library supports simple and multivariate linear regression. +<<Simple Linear Regression, Simple Linear Regression>> - +<<Regression Plot, Regression Plot>> - +<<Residual Plot, Residual Plot>> - +<<Multivariate Linear Regression, Multivariate Linear Regression>> + == Simple Linear Regression The `regress` function is used to build a linear regression model @@ -131,7 +136,7 @@ let(a=random(logs, q="*:*", rows="5000", fl="filesize_d, response_d"), When this expression is sent to the `/stream` handler it responds with: -[source,json] +[source,text] ---- { "result-set": { @@ -192,7 +197,7 @@ let(a=random(logs, q="*:*", rows="500", fl="filesize_d, response_d"), When this expression is sent to the `/stream` handler it responds with: -[source,json] +[source,text] ---- { "result-set": { @@ -376,7 +381,7 @@ let(a=random(logs, q="*:*", rows="5000", fl="filesize_d, load_d, response_d"), When this expression is sent to the `/stream` handler it responds with: -[source,json] +[source,text] ---- { "result-set": { @@ -428,7 +433,7 @@ let(a=random(logs, q="*:*", rows="5000", fl="filesize_d, load_d, response_d"), When this expression is sent to the `/stream` handler it responds with: -[source,json] +[source,text] ---- { "result-set": { diff --git a/solr/solr-ref-guide/src/statistics.adoc b/solr/solr-ref-guide/src/statistics.adoc index d391e4b..759d995 100644 --- a/solr/solr-ref-guide/src/statistics.adoc +++ b/solr/solr-ref-guide/src/statistics.adoc @@ -19,6 +19,13 @@ This section of the user guide covers the core statistical functions available in math expressions. + +<<descriptive-statistics, Descriptive Statistics>> - <<Histograms and Frequency Tables, Histograms>> - +<<Frequency Tables, Frequency Tables>> - <<Percentiles, Percentiles>> - <<Quantile Plots, Quantile Plots>> - +<<Correlation and Covariance, Correlation and Covariance>> - <<Statistical Inference Tests, Inference Tests>> - +<<Transformations, Transformations>> - <<Z-scores, Z-scores>> + + == Descriptive Statistics The `describe` function returns descriptive statistics for a diff --git a/solr/solr-ref-guide/src/streaming-expressions.adoc b/solr/solr-ref-guide/src/streaming-expressions.adoc index 1ce3638..20c29f9 100644 --- a/solr/solr-ref-guide/src/streaming-expressions.adoc +++ b/solr/solr-ref-guide/src/streaming-expressions.adoc @@ -30,6 +30,7 @@ At a high level there a four main capabilities that will be explored in the docu * *Visualizing* result sets, aggregations and statistical models of the data. +>>>>>>> SOLR-13105: The Visual Guide to Streaming Expressions and Math Expressions == Stream Language Basics @@ -91,10 +92,38 @@ Stream sources originate streams. There are rich set of searching, sampling and A full reference to all available source expressions is available in <<stream-source-reference.adoc#stream-source-reference,Stream Source Reference>>. +== Elements of the Lanaguage + +=== Stream Sources + +Stream sources originate streams. There are rich set of searching, sampling and aggregation stream sources to choose from. + +A full reference to all available source expressions is available in <<stream-source-reference.adoc#stream-source-reference,Stream Source Reference>>. + + +=== Stream Decorators + +=== Request Routing + +Streaming Expressions respect the <<distributed-requests.adoc#shards-preference-parameter,shards.preference parameter>> for any call to Solr. + +The value of `shards.preference` that is used to route requests is determined in the following order. The first option available is used. +- Provided as a parameter in the streaming expression (e.g., `search(...., shards.preference="replica.type:PULL")`) +- Provided in the URL Params of the streaming expression (e.g., `http://solr_url:8983/solr/stream?expr=....&shards.preference=replica.type:PULL`) +- Set as a default in the Cluster properties. + +=== Adding Custom Expressions + +Creating your own custom expressions can be easily done by implementing the {solr-javadocs}/solrj/org/apache/solr/client/solrj/io/stream/expr/Expressible.html[Expressible] interface. To add a custom expression to the +list of known mappings for the `/stream` and `/graph` handlers, you just need to declare it as a plugin in `solrconfig.xml` via: +>>>>>>> SOLR-13105: The Visual Guide to Streaming Expressions and Math Expressions + === Stream Decorators +<<<<<<< HEAD Stream decorators wrap stream sources and other stream decorators to transform a stream. +>>>>>>> SOLR-13105: The Visual Guide to Streaming Expressions and Math Expressions A full reference to all available decorator expressions is available in <<stream-decorator-reference.adoc#stream-decorator-reference,Stream Decorator Reference>>. diff --git a/solr/solr-ref-guide/src/vector-math.adoc b/solr/solr-ref-guide/src/vector-math.adoc index 2c3ee66..5721c45 100644 --- a/solr/solr-ref-guide/src/vector-math.adoc +++ b/solr/solr-ref-guide/src/vector-math.adoc @@ -18,6 +18,15 @@ This section covers vector math and vector manipulation functions. +<<Arrays, Arrays>> - +<<Visualization, Visualization>> - +<<Array Operations, Array Operations>> - +<<Getting Values By Index, valueAt>> - +<<Sequences, Sequences>> - +<<Vector Sorting, Sorting>> - +<<Vector Summarizations and Norms, Summarization>> - +<<Scalar Vector Math, Vector Math>> + == Arrays Arrays can be created with the `array` function. diff --git a/solr/solr-ref-guide/src/visualization.adoc b/solr/solr-ref-guide/src/visualization.adoc index 1cc02ea..fe5c68c 100644 --- a/solr/solr-ref-guide/src/visualization.adoc +++ b/solr/solr-ref-guide/src/visualization.adoc @@ -142,5 +142,3 @@ image::images/math-expressions/noise-autocorrelation.png[] === Fourier Transform image::images/math-expressions/signal-fft.png[] - -
