Thanks Alex and Jorg. Frankly speaking I didn't think of details of implementing such external DSL in ES. I am just from a user's perspective thinking of what's the best user experience.
Jorg raises a good question that how to distinguish data type given the nature of schemaless in ES, the DSL may have to have some keywords(e.g int, long, date) to tell data type explicitly if it's important, otherwise all data are treated as string. For other questions I can not answer because some of features like facet I have not even used. I know it's not straightforward to do such thing, but I do think if JSON DSL can solve these problems then the external DSL can also do because it's more flexible like a programming language. I have been playing with some NoSql databases as well as search engine like ES, an impression I got is every software has its own search DSL which are not user friendly. Though I think it might be impossible for this area to agree on a standard similar to SQL given these software focus on different parts and usually vary a lot, I think using a SQL style DSL without join and sub-select can make user's life much easier. On Friday, December 20, 2013 4:32:30 AM UTC-8, Jörg Prante wrote: > > Just some food for thought: > > In the DSL you give, there are some subtle issues which makes it hard for > implementing a parser and correct query execution. > > For example, the word "New feature": is it just a phrase, or is it two > terms? Is a span query for the two terms also valid? > > Another issue is well-formed input data typing. Is "ES" in the query an > input of type "string"? And is "3.1" is double value or also a string type? > Can types always be inferenced? What about date parsing, will date types be > assigned automatically? Maybe by looking up the ES mapping (well, ES > parsers do that already) ? Or, how to decide if "true" is the string "true" > or the boolean constant? ES catches this but I mean the general case of an > external DSL is dealing with JSON data type flaws. You are forced to > re-implement the ES parser for all these nasty pitfalls. > > How to declare facets and filters in an external DSL? Or multi phrase and > multi wildcard searching? It's not straightforward if there is simply no > context information how to execute such things. How do you parse and > translate wildcards mixed/nested with phrases such as "scien* 'week* > magazin*'"? I was surprised how many folks are trained to use wildcards > excessively. It is only possible to replace such "bad queries" with > heuristics that can be executed on ES with high performance, with ranked > results etc. > > I have written a CQL parser > http://docs.oasis-open.org/search-ws/searchRetrieve/v1.0/os/part5-cql/searchRetrieve-v1.0-os-part5-cql.pdfto > generate ES DSL, but for Java API only. If anyone is interested in > adding a CQL parser as a REST action, I could offer it as a plugin. It is > of course not perfect, I'm not very satisfied with the result. > > My experience so far is, at least for CQL, because it is a weak typed > query language (it does not have a notion of input data types) , that > external query languages must really be able to match the power of > Lucene/ES features, or you get into trouble implementing simplifications, > fallbacks, and shortcuts all the way. > > So my favorite is still ES DSL, and for establishing simple searches, > there are special featured query types that are designed for simplified > free form input. E.g. ES DSL query type "query_string" understands the > Lucene syntax, or the "match" query, and now we also have the > "simple_query" https://github.com/elasticsearch/elasticsearch/pull/4402 > > I'm interested in OpenSearch for ES http://www.opensearch.org/ so if > anyone is working on this, it would be nice to know. > > Jörg > > > On Thu, Dec 19, 2013 at 8:05 PM, xin zhang <[email protected]<javascript:> > > wrote: >> >> >> project = ES AND issuetype = "New Feature" AND fixVersion = 3.1 ORDER BY >> created DESC, cf[10514] DESC >> >> >> -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0a5a07e4-666b-4498-a781-c48882fae2c6%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
