Paul Rogers created DRILL-6169:

             Summary: Table functions do not allow list arguments
                 Key: DRILL-6169
             Project: Apache Drill
          Issue Type: Bug
    Affects Versions: 1.12.0
            Reporter: Paul Rogers

Consider DRILL-6167 which describes a regex storage plugin. The plugin 
definition wants to take a list of columns. This can be expressed in the 
storage plugin config as a list:

public class RegexFormatConfig implements FormatPluginConfig {
  public List<String> fields = new ArrayList<>();

The list can be defined in JSON using the Drill web console. All works fine.

Drill also provides table functions that allow us to specify format plugin 
config options at query time. Because the function is defined as SQL, there 
appears to be no way to express an argument which is a list.

Let's try to define the list in sql:

   SELECT * FROM table(cp.`regex/simple.log2`
      (type => 'regex',
       regex => '(\\d\\d\\d\\d)-(\\d\\d)-(\\d\\d) .*',
       fields => ('a', 'b', 'c')))

The syntax above is legal SQL (see below), but is not wired up correctly to map 
to a list argument. When run:

  From line 1, column 24 to line 4, column 27:
  No match found for function signature
    regex/simple.log2(type => <CHARACTER>,
      regex => <CHARACTER>,
      fields => <RecordType(CHAR(1) EXPR$0, CHAR(1) EXPR$1, CHAR(1) EXPR$2)>)

This error seems to say that:

* The syntax is OK. (Check {{parser.jj}}, if you work down the rules starting 
from {{TableFunctionCall}}, you'll see that {{ParenthesizedQueryOrCommaList}} 
defines the above list as valid syntax for a function argument.
* The type of the argument appears to be a {{RecordType}}.
* No rules are in place to realize that, for a plugin config table function, 
the {{RecordType}} can be translated to a String list.

The only workaround appears to be to redefine the string list to be a single 
string. In this case, rather than a list of fields, require a string with a 
comma-separated list of files which is parsed by the plugin at run time.

If Drill provides lists in the config, and provides table functions to set the 
properties in a config, then it should provide a way to set lists.

This message was sent by Atlassian JIRA

Reply via email to