[
https://issues.apache.org/jira/browse/DRILL-1656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14202463#comment-14202463
]
Hao Zhu commented on DRILL-1656:
--------------------------------
Hi Jacques,
Per globStatus Doc:
https://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/FileSystem.html
It supports:
{code}
A filename pattern is composed of regular characters and special pattern
matching characters, which are:
?
Matches any single character.
*
Matches zero or more characters.
[abc]
Matches a single character from character set {a,b,c}.
[a-b]
Matches a single character from the character range {a...b}. Note that
character a must be lexicographically less than or equal to character b.
[^a]
Matches a single character that is not from character set or range {a}. Note
that the ^ character must occur immediately to the right of the opening bracket.
\c
Removes (escapes) any special meaning of character c.
{ab,cd}
Matches a string from the string set {ab, cd}
{ab,c{de,fh}}
Matches a string from the string set {ab, cde, cfh}
{code}
Per tests, many of them are supported in "hadoop fs -ls" but not supported in
drill.
{code}
0: jdbc:drill:> select * from `ab*/201409*`
. . . . . . . > ;
Query failed: Failure while parsing sql. Unsupported selection path:
/mapr/demo.mapr.com/data/ab*/201409* [9a1f79a3-8485-45f5-9af4-442e4f5985da]
Node details: maprdemo:31011/31012
net.hydromatic.optiq.tools.ValidationException:
java.lang.IllegalStateException: Unsupported selection path:
/mapr/demo.mapr.com/data/ab*/201409*
at
net.hydromatic.optiq.prepare.PlannerImpl.validate(PlannerImpl.java:175)
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:143)
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:123)
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:132)
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:425)
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:219)
at
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Unsupported selection path:
/mapr/demo.mapr.com/data/ab*/201409*
at
com.google.common.base.Preconditions.checkState(Preconditions.java:149)
at
org.apache.drill.exec.store.dfs.FileSelection.create(FileSelection.java:145)
at
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.create(WorkspaceSchemaFactory.java:122)
at
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.create(WorkspaceSchemaFactory.java:54)
at
org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96)
at
org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90)
at
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:260)
at
net.hydromatic.optiq.jdbc.SimpleOptiqSchema.getTable(SimpleOptiqSchema.java:75)
at
net.hydromatic.optiq.prepare.OptiqCatalogReader.getTableFrom(OptiqCatalogReader.java:87)
at
net.hydromatic.optiq.prepare.OptiqCatalogReader.getTable(OptiqCatalogReader.java:70)
at
net.hydromatic.optiq.prepare.OptiqCatalogReader.getTable(OptiqCatalogReader.java:42)
at
org.eigenbase.sql.validate.EmptyScope.getTableNamespace(EmptyScope.java:67)
at
org.eigenbase.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:75)
at
org.eigenbase.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:85)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:779)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:768)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:2599)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:2807)
at
org.eigenbase.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at
org.eigenbase.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:85)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:779)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:768)
at org.eigenbase.sql.SqlSelect.validate(SqlSelect.java:208)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:742)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:458)
at
net.hydromatic.optiq.prepare.PlannerImpl.validate(PlannerImpl.java:173)
... 9 more
Error: exception while executing query: Failure while trying to get next result
batch. (state=,code=0)
[root@maprdemo]# hadoop fs -ls /mapr/demo.mapr.com/data/ab*/201409*
Found 2 items
-rw-r--r-- 1 mapr mapr 6 2014-11-06 16:04
/mapr/demo.mapr.com/data/abc/20140901/a.csv
-rw-r--r-- 1 mapr mapr 6 2014-11-06 16:04
/mapr/demo.mapr.com/data/abc/20140901/b.csv
Found 1 items
-rw-r--r-- 1 mapr mapr 6 2014-11-06 16:04
/mapr/demo.mapr.com/data/abc/20140902/c.csv
Found 1 items
-rw-r--r-- 1 root root 6 2014-11-06 18:20
/mapr/demo.mapr.com/data/abc/20140903/d.csv
{code}
Another example:
{code}
0: jdbc:drill:> select * from `ab?/201409*`;
Query failed: Failure while parsing sql. Unsupported selection path:
/mapr/demo.mapr.com/data/ab?/201409* [960beaa3-19dd-4bd2-9dba-3f0b95f9e301]
Node details: maprdemo:31011/31012
net.hydromatic.optiq.tools.ValidationException:
java.lang.IllegalStateException: Unsupported selection path:
/mapr/demo.mapr.com/data/ab?/201409*
at
net.hydromatic.optiq.prepare.PlannerImpl.validate(PlannerImpl.java:175)
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:143)
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:123)
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:132)
at org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:425)
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:219)
at
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:250)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Unsupported selection path:
/mapr/demo.mapr.com/data/ab?/201409*
at
com.google.common.base.Preconditions.checkState(Preconditions.java:149)
at
org.apache.drill.exec.store.dfs.FileSelection.create(FileSelection.java:145)
at
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.create(WorkspaceSchemaFactory.java:122)
at
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory.create(WorkspaceSchemaFactory.java:54)
at
org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.getNewEntry(ExpandingConcurrentMap.java:96)
at
org.apache.drill.exec.planner.sql.ExpandingConcurrentMap.get(ExpandingConcurrentMap.java:90)
at
org.apache.drill.exec.store.dfs.WorkspaceSchemaFactory$WorkspaceSchema.getTable(WorkspaceSchemaFactory.java:260)
at
net.hydromatic.optiq.jdbc.SimpleOptiqSchema.getTable(SimpleOptiqSchema.java:75)
at
net.hydromatic.optiq.prepare.OptiqCatalogReader.getTableFrom(OptiqCatalogReader.java:87)
at
net.hydromatic.optiq.prepare.OptiqCatalogReader.getTable(OptiqCatalogReader.java:70)
at
net.hydromatic.optiq.prepare.OptiqCatalogReader.getTable(OptiqCatalogReader.java:42)
at
org.eigenbase.sql.validate.EmptyScope.getTableNamespace(EmptyScope.java:67)
at
org.eigenbase.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:75)
at
org.eigenbase.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:85)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:779)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:768)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:2599)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:2807)
at
org.eigenbase.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60)
at
org.eigenbase.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:85)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:779)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:768)
at org.eigenbase.sql.SqlSelect.validate(SqlSelect.java:208)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:742)
at
org.eigenbase.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:458)
at
net.hydromatic.optiq.prepare.PlannerImpl.validate(PlannerImpl.java:173)
... 9 more
Error: exception while executing query: Failure while trying to get next result
batch. (state=,code=0)
[root@maprdemo ]# hadoop fs -ls /mapr/demo.mapr.com/data/ab?/201409*
Found 2 items
-rw-r--r-- 1 mapr mapr 6 2014-11-06 16:04
/mapr/demo.mapr.com/data/abc/20140901/a.csv
-rw-r--r-- 1 mapr mapr 6 2014-11-06 16:04
/mapr/demo.mapr.com/data/abc/20140901/b.csv
Found 1 items
-rw-r--r-- 1 mapr mapr 6 2014-11-06 16:04
/mapr/demo.mapr.com/data/abc/20140902/c.csv
Found 1 items
-rw-r--r-- 1 root root 6 2014-11-06 18:20
/mapr/demo.mapr.com/data/abc/20140903/d.csv
{code}
> Multiple wildcards(*) are not supported
> ---------------------------------------
>
> Key: DRILL-1656
> URL: https://issues.apache.org/jira/browse/DRILL-1656
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 0.6.0
> Reporter: Hao Zhu
>
> Drill can not query the directories using multiple wild cards.
> eg:
> {code}
> 0: jdbc:drill:> select * from `abc/201409*`;
> +------------+------------+
> | columns | dir0 |
> +------------+------------+
> | ["4","4","4"] | 20140903 |
> | ["3","3","3"] | 20140902 |
> | ["2","2","2"] | 20140901 |
> | ["1","1","1"] | 20140901 |
> +------------+------------+
> 4 rows selected (0.213 seconds)
> 0: jdbc:drill:> select * from `ab*/201409*`;
> Query failed: Failure while parsing sql. Unsupported selection path:
> /mapr/demo.mapr.com/data/ab*/201409* [0006b50e-ef6b-475d-8488-2fc237d39517]
> Error: exception while executing query: Failure while trying to get next
> result batch. (state=,code=0)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
