Hi John, Unfortunately this feature only works with the options that can be set as part of the format plugin right now. I think we should definitely move the options for interpreting files, like the JSON options you mentioned to the format plugin/select with options scope, rather than making users set them at a system or session level.
Do you think it is useful to keep these available at the system/session level as well? Steven Phillips has mentioned before that he has concerns about these types of options that can affect query results that are not associated directly with a table or file. It would be worth opening a discussion about a community consensus on what behavior makes the most sense, and if we need to maintain backwards compatibility. I am okay with breaking compatibility for consistency, but I would like us to have a few releases where we warn people setting the soon to be deprecated version at the system/session and direct them to docs on the new expected workflow, setting the option in the format plugin or using select with options. I have opened a JIRA for this task: https://issues.apache.org/jira/browse/DRILL-4206 On Wed, Dec 16, 2015 at 11:38 AM, John Omernik <[email protected]> wrote: > This is great! > > I tried using the select with options on a json table, where I get the > issues with different types on number fields. I tried this and got an > error. I think I am following the docs correctly, any thoughts here? (note > I've tried it with json.read_numbers_as_double, > store.json.read_numbers_as_double and get the same error. I get a different > error, below with just read_numbers_as_double. Anythoughts would be > helpful!! > > > select * from table(dfs.dev.`jsontable`(json.read_numbers_as_double => > true)) limit 10; > > Error: PARSE ERROR: Encountered "=>" at line 1, column 67. > > Was expecting one of: > > ")" ... > > "ORDER" ... > > "LIMIT" ... > > "OFFSET" ... > > "FETCH" ... > > "," ... > > "UNION" ... > > "INTERSECT" ... > > "EXCEPT" ... > > "NOT" ... > > "IN" ... > > "BETWEEN" ... > > "LIKE" ... > > "SIMILAR" ... > > "=" ... > > ">" ... > > "<" ... > > "<=" ... > > ">=" ... > > "<>" ... > > "+" ... > > "-" ... > > "*" ... > > "/" ... > > "||" ... > > "AND" ... > > "OR" ... > > "IS" ... > > "MEMBER" ... > > "SUBMULTISET" ... > > "MULTISET" ... > > "[" ... > > "." ... > > "(" ... > > > > > while parsing SQL query: > > select * from table(dfs.dev.`jsontable`(json.read_numbers_as_double => > true)) limit 10 > > ^ > > > > [Error Id: 9b07ad9b-0f4a-440f-8c4e-a380a7e73f73 on node4:31010] > (state=,code=0) > > > select * from table(dfs.dev.`jsondata`(read_numbers_as_double => true)) > limit 10; > > Error: VALIDATION ERROR: From line 1, column 29 to line 1, column 69: No > match found for function signature jsondata(read_numbers_as_double => > <BOOLEAN>) > > > > [Error Id: 708ed594-a54a-49a1-808e-a21000ae2ca3 on node4:31010] > (state=,code=0) > > > > > > On Tue, Dec 15, 2015 at 4:52 PM, Venki Korukanti < > [email protected]> > wrote: > > > On behalf of Apache Drill community, I am happy to announce the release > of > > Apache Drill 1.4.0. > > > > This release of Drill fixes many issues and introduces a number of > > enhancements, > > including the following ones: > > > > - Partition pruning improvements to reduce the planning time (DRILL-3765 > > <https://issues.apache.org/jira/browse/DRILL-3765>). > > - Select with options. More about this feature here [1]. > > - ValueVector related code is extracted from 'exec/java-exec' module > into a > > separate module 'exec/vector'. See [2]. There is no change for Drill end > > user experience, but it is an opportunity for developers to use the > Drill's > > in-memory columnar representation in their own projects. > > > > The source and binary artifacts are available at [3] > > Review a complete list of fixes and enhancements at [4] > > > > Thanks to everyone in the community who contributed in this release. > > > > [1] > > > > > https://drill.apache.org/docs/plugin-configuration-basics/#using-the-formats-attributes-as-table-function-parameters > > [2] https://issues.apache.org/jira/browse/DRILL-3987 > > [3] http://drill.apache.org/download/ > > [4] http://drill.apache.org/docs/apache-drill-1-4-0-release-notes/ > > > > Thanks > > Venki > > >
