[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072812#comment-15072812
]
Dennis Gove commented on SOLR-8458:
---
As I see it there are 2 pieces here, both related but separate.
First, adding support for parameter substitution in an expression. This would
be handled with the changes I discussed above to StreamFactory, StreamParser,
and the addition of a new type StreamExpressionSubstitutionParameter. Note that
this doesn't necessarily care how the expressions come in.
And second, adding support for parameter substitution in StreamHandler and in
an http request. I like the syntax Joel uses in the description. What this
would mean is that StreamHandler would see http params like "expr", "left" and
"right", would know that these are expressions (can call into StreamFactory to
check if something is a valid expression), and would pass them off
independently to be parsed and then together to be pieced together.
This approach modularizes the implementation such that how an expression with
substitution comes in via http is independent to how it is handled within the
Streaming API.
For example, the following comes into StreamHandler
{code}
http://localhost:8983/col/stream?expr=merge($left, $right,
...)=jaz=search(...)=search(...)=bar
{code}
The StreamHandler will see five parameters, expr, baz, left, right, and foo. It
would then determine that expr, left, and right are valid expressions and pass
them off to be parsed into three expression objects. It would then pass all
three into the factory to be combined into a single Stream object. The factory
would then iterate (recursively?) until there aren't any more instances of a
StreamExpressionSubstitutionParameter at any level (considering the possibility
of infinite loops, of course). At this point it'd then just be passed off to
create a Stream object as any other expression would be.
Another possibility would be to parse out the substitution expressions and then
register them in the factory for use during Stream object creation. This would
negate the need to do that pre-processing of the N substitution expression and
would give a place to register "pre-compiled" expressions. I'm not a huge fan
of this approach as it would add more state to the factory and I'm not a huge
fan of the state it already contains.
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072754#comment-15072754
]
Joel Bernstein commented on SOLR-8458:
--
I'm fine either as a pre-processing step or including the substitution as part
of the parsing.
If we stick with the pre-processing step I think we'll need to process the
nested vars. One approach to this would be to continue looping over the
expression string and call the substitution method on each loop until there are
no more $vars.
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072767#comment-15072767
]
Cao Manh Dat commented on SOLR-8458:
That a good idea.
About substitution as part of the parsing. Should the syntax look like?
We cant use {&} key words here
{merge($left, $right, ...)=search(...)=search(...)}
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072763#comment-15072763
]
Joel Bernstein commented on SOLR-8458:
--
[~dpgove], you have good ideas here though. +1 if you want to take this on.
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072820#comment-15072820
]
Yonik Seeley commented on SOLR-8458:
Regarding pre-processing (i.e. ignorant of syntax), Solr already has parameter
substitution across entire query requests:
http://yonik.com/solr-query-parameter-substitution/
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072875#comment-15072875
]
Yonik Seeley commented on SOLR-8458:
Yeah, looks like it is already working:
Putting this into my browser:
{code}
http://localhost:8983/solr/techproducts/stream?expr=${foo}=hello
{code}
I get:
{code}
{"EXCEPTION":"'hello' is not a proper expression clause","EOF":true}]}}
{code}
If you try it with a GET in curl, just turn off curl's globbing:
{code}
curl -g 'http://localhost:8983/solr/techproducts/stream?expr=${foo}=hello'
{code}
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072874#comment-15072874
]
Dennis Gove commented on SOLR-8458:
---
This is great news. I'm all for continuing to make use of this feature. Thanks!
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072848#comment-15072848
]
Joel Bernstein commented on SOLR-8458:
--
Thanks for bringing this up.
In reading the blog post it sounds like we already have param substitution for
streaming expression and just didn't know it. I didn't realize this
functionality was so broad.
So, any parameter regardless of whether it's part of a local param will
automatically have parameter substitutions applied?
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072871#comment-15072871
]
Dennis Gove commented on SOLR-8458:
---
I agree. There's no reason to reinvent and I'm always a fan of keeping things
consistent. If preprocessing substitution is already implemented for all
incoming requests then we should absolutely make use of it.
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072863#comment-15072863
]
Joel Bernstein commented on SOLR-8458:
--
I like the syntax as well:
{code}
http://localhost:8983/col/stream?expr=merge(${left}, ${right},
...)=jaz=search(...)=search(...)=bar
{code}
If this is already working then we can change the ticket to *Add Streaming
Expressions tests for param substitution*.
[~dpgove], we can always decide to work on the pre-compiled expressions at a
later time. But I think that will take quite a bit of thought.
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072865#comment-15072865
]
Joel Bernstein commented on SOLR-8458:
--
[~caomanhdat], would you like to work on a first test case based on Yonik's
blog? http://yonik.com/solr-query-parameter-substitution/
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072936#comment-15072936
]
Joel Bernstein commented on SOLR-8458:
--
Ok, great.
I'll change the ticket to Add Streaming Expression tests with param
substitution.
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072398#comment-15072398
]
Cao Manh Dat commented on SOLR-8458:
It quite a good idea. But I also confuse about the syntax of substitution in
Streaming Expressions
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-8458) Parameter substitution for Streaming Expressions
[
https://issues.apache.org/jira/browse/SOLR-8458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15072395#comment-15072395
]
Dennis Gove commented on SOLR-8458:
---
What if we were to make substitution parameters first class citizens similar to
named parameters? During the parsing in ExpressionParser we could create
instances of StreamExpressionSubstitutionParameters which exist as first class
citizens of an StreamExpression object. This would allow us to send (in the
example in the description) "expr", "left", and "right" through the
ExpressionParser. Then, a simple method can be added to the StreamFactory which
accepts a main expression and a map of names => expressions. It could then
iterate over parameters of the main expression doing replacements until there
are no more instances of StreamExpressionSubstitutionParameter in the main
expression. Some checks for infinite loops would have to be added but those are
relatively simple.
This approach would allow the logic to exist outside of the StreamHandler which
I think would be beneficial for the SQL Handler.
It might also allow for some type of prepared statements with "pre-compiled"
pieces (similar to what one might see in a DBMS). For example, this might be
beneficial in a situation where some very expensive part of the expression is
static which you want to perform different rollups or joins or whatever with.
An optimizer could hang onto the static results in a RepeatableStream (doesn't
exist yet) and substitute that into some other expression.
> Parameter substitution for Streaming Expressions
>
>
> Key: SOLR-8458
> URL: https://issues.apache.org/jira/browse/SOLR-8458
> Project: Solr
> Issue Type: Improvement
>Reporter: Joel Bernstein
>Priority: Minor
> Attachments: SOLR-8458.patch
>
>
> As Streaming Expressions become more complicated it would be nice to support
> parameter substitution. For example:
> {code}
> http://localhost:8983/col/stream?expr=merge($left, $right,
> ...)=search(...)=search(...)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
