[jira] [Updated] (DRILL-2419) UDF that returns string representation of expression type
[ https://issues.apache.org/jira/browse/DRILL-2419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2419: - Fix Version/s: (was: 1.0.0) 1.1.0 UDF that returns string representation of expression type - Key: DRILL-2419 URL: https://issues.apache.org/jira/browse/DRILL-2419 Project: Apache Drill Issue Type: Improvement Components: Functions - Drill Reporter: Victoria Markman Assignee: Mehant Baid Fix For: 1.1.0 Suggested name: typeof (credit goes to Aman) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2487) Schema is ignored when using : between schema and zk on sqlline connection string
[ https://issues.apache.org/jira/browse/DRILL-2487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2487: - Fix Version/s: (was: 1.0.0) 1.1.0 Schema is ignored when using : between schema and zk on sqlline connection string Key: DRILL-2487 URL: https://issues.apache.org/jira/browse/DRILL-2487 Project: Apache Drill Issue Type: Bug Components: Client - CLI Affects Versions: 0.8.0 Reporter: Krystal Assignee: Daniel Barclay (Drill) Priority: Minor Fix For: 1.1.0 git.commit.id=ae2053d2a078a40033a140f2dfaeef802a5e8254 Invoking sqlline using a : between the schema and zk causes sqlline not to connect the specified schema. For example: root@qa-node113:~# /opt/drill/bin/sqlline -u 'jdbc:drill:schema=hive:zk=10.10.100.113:5181' touch: cannot touch `/var/log/drill/sqlline.log': No such file or directory Drill log directory /var/log/drill does not exist or is not writable, defaulting to /opt/drill/log sqlline version 1.1.6 0: jdbc:drill:schema=hive:zk=10.10.100.113:51 show tables; Query failed: RelConversionException: No schema selected. Select a schema using 'USE schema' command If I put a ; between schema and zk, then sqlline connects to the specified schema: root@qa-node113:~# /opt/drill/bin/sqlline -u 'jdbc:drill:schema=hive;zk=10.10.100.113:5181' touch: cannot touch `/var/log/drill/sqlline.log': No such file or directory Drill log directory /var/log/drill does not exist or is not writable, defaulting to /opt/drill/log sqlline version 1.1.6 0: jdbc:drill:schema=hive show tables; +--++ | TABLE_SCHEMA | TABLE_NAME | +--++ | hive.default | t2 | | hive.default | episodes_partitioned | | hive.default | store | | hive.default | store_sales | -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2478) Validating values assigned to SYSTEM/SESSION configuration parameters
[
https://issues.apache.org/jira/browse/DRILL-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2478:
-
Assignee: Chris Westin (was: Parth Chandra)
Validating values assigned to SYSTEM/SESSION configuration parameters
-
Key: DRILL-2478
URL: https://issues.apache.org/jira/browse/DRILL-2478
Project: Apache Drill
Issue Type: Bug
Components: Execution - Flow
Affects Versions: 0.8.0
Environment: {code}
0: jdbc:drill: select * from sys.version;
+++-+-++
| commit_id | commit_message | commit_time | build_email | build_time |
+++-+-++
| f658a3c513ddf7f2d1b0ad7aa1f3f65049a594fe | DRILL-2209 Insert
ProjectOperator with MuxExchange | 09.03.2015 @ 01:49:18 EDT | Unknown |
09.03.2015 @ 04:50:05 EDT |
+++-+-++
1 row selected (0.046 seconds)
{code}
Reporter: Khurram Faraaz
Assignee: Chris Westin
Fix For: 1.0.0
Values that are assigned to configuration parameters of type SYSTEM and
SESSION must be validated. Currently any value can be assigned to some of the
SYSTEM/SESSION type parameters.
Here are two examples where assignment of invalid values to store.format does
not result in any error.
{code}
0: jdbc:drill: alter session set `store.format`='1';
+++
| ok | summary |
+++
| true | store.format updated. |
+++
1 row selected (0.02 seconds)
{code}
{code}
0: jdbc:drill: alter session set `store.format`='foo';
+++
| ok | summary |
+++
| true | store.format updated. |
+++
1 row selected (0.039 seconds)
{code}
In some cases values to some of the configuration parameters are validated,
like in this example, where trying to assign an invalid value to parameter
store.parquet.compression results in an error, which is correct. However,
this kind of validation is not performed for every configuration parameter of
SYSTEM/SESSION type. These values that are assigned to parameters must be
validated, and report errors if incorrect values are assigned by users.
{code}
0: jdbc:drill: alter session set `store.parquet.compression`='anything';
Query failed: ExpressionParsingException: Option store.parquet.compression
must be one of: [snappy, gzip, none]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2241) CTAS fails when writing a repeated list
[
https://issues.apache.org/jira/browse/DRILL-2241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacques Nadeau updated DRILL-2241:
--
Fix Version/s: (was: Future)
0.9.0
CTAS fails when writing a repeated list
---
Key: DRILL-2241
URL: https://issues.apache.org/jira/browse/DRILL-2241
Project: Apache Drill
Issue Type: New Feature
Components: Storage - Parquet
Affects Versions: 0.8.0
Reporter: Abhishek Girish
Assignee: Deneche A. Hakim
Priority: Blocker
Fix For: 0.9.0
Attachments: drillbit_replist.log
Drill can read the following JSON file with a repeated list:
{
a : null
b : [ [B1, B2] ],
}
Writing this to Parquet via a simple CTAS fails.
create table temp as select * from `replist.json`;
Log indicates this to be unsupported (UnsupportedOperationException:
Unsupported type LIST)
Log attached.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2435) Query SCHEMATA using querySubmitter against C++ client return no data
[ https://issues.apache.org/jira/browse/DRILL-2435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2435: - Fix Version/s: (was: 1.0.0) 1.1.0 Query SCHEMATA using querySubmitter against C++ client return no data - Key: DRILL-2435 URL: https://issues.apache.org/jira/browse/DRILL-2435 Project: Apache Drill Issue Type: Bug Components: Client - C++ Affects Versions: 0.8.0 Reporter: Krystal Assignee: Parth Chandra Priority: Minor Fix For: 1.1.0 The following query from querySubmitter against C++ client returns rows with empty column values: CATALOG_NAMESCHEMA_NAME SCHEMA_OWNERTYPEIS_MUTABLE ROW: 1 ROW: 2 ROW: 3 ROW: 4 ROW: 5 ROW: 6 Running the same query from drill, all data is returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2416) Zookeeper in sqlline connection string does not override the entry from drill-override.conf
[ https://issues.apache.org/jira/browse/DRILL-2416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2416: - Fix Version/s: (was: 1.0.0) 1.1.0 Zookeeper in sqlline connection string does not override the entry from drill-override.conf Key: DRILL-2416 URL: https://issues.apache.org/jira/browse/DRILL-2416 Project: Apache Drill Issue Type: Bug Components: Client - CLI Affects Versions: 0.8.0 Reporter: Krystal Assignee: Parth Chandra Fix For: 1.1.0 git.commit.id=f658a3c513ddf7f2d1b0ad7aa1f3f65049a594fe On the sqlline jdbc connection string, I changed the zookeeper ip to point to another cluster; however, sqlline kept connecting to the drillbits specified in drill-override.conf. I updated the drill-override.conf with the other zookeeper information, then I was able to successfully connected to the drillbits on a remote cluster. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2496) Add SSL support to C++ client
[ https://issues.apache.org/jira/browse/DRILL-2496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2496: - Fix Version/s: (was: 1.0.0) 1.1.0 Add SSL support to C++ client - Key: DRILL-2496 URL: https://issues.apache.org/jira/browse/DRILL-2496 Project: Apache Drill Issue Type: Improvement Components: Client - C++ Reporter: Parth Chandra Assignee: Parth Chandra Fix For: 1.1.0 Needed for impersonation where username and password are sent over the wire to the user. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2273) Move new JDBC test support classes to org.apache.drill.jdbc.test/etc.
[ https://issues.apache.org/jira/browse/DRILL-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2273: - Fix Version/s: (was: 1.0.0) 1.1.0 Move new JDBC test support classes to org.apache.drill.jdbc.test/etc. - Key: DRILL-2273 URL: https://issues.apache.org/jira/browse/DRILL-2273 Project: Apache Drill Issue Type: Bug Components: Client - JDBC Reporter: Daniel Barclay (Drill) Assignee: Daniel Barclay (Drill) Priority: Minor Fix For: 1.1.0 JDBC test support classes in package org.apache.drill.jdbc in the src/test/java tree should be elsewhere, presumably in package org.apache.drill.jdbc.test: - CachingConnectionFactory - ConnectionFactory - ConnectionInfo - JdbcTest - MultiConnectionCachingFactory - NonClosableConnectionCachingFactory - SingleConnectionCachingFactory (In the src/test/java tree, package org.apache.drill.jdbc should be reserved for tests of types in package org.apache.drill.jdbc in src/main/java.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2594) Web UI visualized plan may show incorrect plan due to rearranging
[ https://issues.apache.org/jira/browse/DRILL-2594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Westin updated DRILL-2594: Fix Version/s: 1.0.0 Web UI visualized plan may show incorrect plan due to rearranging -- Key: DRILL-2594 URL: https://issues.apache.org/jira/browse/DRILL-2594 Project: Apache Drill Issue Type: Improvement Components: Tools, Build Test Affects Versions: 0.8.0 Reporter: Aman Sinha Assignee: Sudheesh Katkam Fix For: 1.0.0 For large enough query plans (such as TPCH query 8), the visualized plan in the Web UI shows the inputs of a HashJoin as swapped whereas in the actual plan it is the correct order. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1724) Can't use ~ to specify home directory in table path
[ https://issues.apache.org/jira/browse/DRILL-1724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1724: - Assignee: Steven Phillips (was: Mehant Baid) Can't use ~ to specify home directory in table path --- Key: DRILL-1724 URL: https://issues.apache.org/jira/browse/DRILL-1724 Project: Apache Drill Issue Type: New Feature Components: Storage - Text CSV Reporter: Tomer Shiran Assignee: Steven Phillips Priority: Minor Fix For: 1.0.0 Drill throws an error when there's a tilda (~) in the path (which was intended to refer to the user's home directory). This should work (and even if not, the error message should explain what the issue is). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-2064) Internal Float4 and Float8 names are confusing; rename using Float and Double
[ https://issues.apache.org/jira/browse/DRILL-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra reassigned DRILL-2064: Assignee: Parth Chandra Internal Float4 and Float8 names are confusing; rename using Float and Double - Key: DRILL-2064 URL: https://issues.apache.org/jira/browse/DRILL-2064 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Reporter: Daniel Barclay (Drill) Assignee: Parth Chandra Priority: Minor Fix For: 1.1.0 [For/from Jacques:] The internal names (e.g., enumerations, vector subtypes, function names) containing Float4 and Float8 are confusing since they don't correspond to type names float and double in Java, etc. Things currently named using Float4 should be renamed to names using Float; things currently named using Float8 should be renamed using Double. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2064) Internal Float4 and Float8 names are confusing; rename using Float and Double
[ https://issues.apache.org/jira/browse/DRILL-2064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2064: - Fix Version/s: (was: 1.0.0) 1.1.0 Internal Float4 and Float8 names are confusing; rename using Float and Double - Key: DRILL-2064 URL: https://issues.apache.org/jira/browse/DRILL-2064 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Reporter: Daniel Barclay (Drill) Priority: Minor Fix For: 1.1.0 [For/from Jacques:] The internal names (e.g., enumerations, vector subtypes, function names) containing Float4 and Float8 are confusing since they don't correspond to type names float and double in Java, etc. Things currently named using Float4 should be renamed to names using Float; things currently named using Float8 should be renamed using Double. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1966) Fix mock data generation for date, time and decimal types
[ https://issues.apache.org/jira/browse/DRILL-1966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1966: - Fix Version/s: (was: 1.0.0) 1.1.0 Fix mock data generation for date, time and decimal types - Key: DRILL-1966 URL: https://issues.apache.org/jira/browse/DRILL-1966 Project: Apache Drill Issue Type: Test Components: Execution - Data Types Reporter: Jason Altekruse Assignee: Daniel Barclay (Drill) Priority: Minor Fix For: 1.1.0 Currently the mock data generation code is shared among all of the vectors created by a single freemarker template. Thus all of the types that are generated by the FixedValueVectors.java template all use the same method that fills values with alternating min/max byte values. This produces invalid data for the decimal, time and date types. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2457) aggregation function on complex type should give appropriate error
[
https://issues.apache.org/jira/browse/DRILL-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481480#comment-14481480
]
Parth Chandra commented on DRILL-2457:
--
[~cch...@maprtech.com] Can you verify the error message is displayed now. If so
please lower the priority. We can mark this fixed only after DRILL-2430 is done.
aggregation function on complex type should give appropriate error
---
Key: DRILL-2457
URL: https://issues.apache.org/jira/browse/DRILL-2457
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.8.0
Reporter: Chun Chang
Assignee: Parth Chandra
Priority: Critical
Fix For: 0.9.0
#Fri Mar 06 16:51:10 EST 2015
git.commit.id.abbrev=fb293ba
Apply aggregation function on complex data should error out with appropriate
error message instead of returning null.
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select t.soa[1] from
`complex.json` t limit 1;
++
| EXPR$0 |
++
| {in:1,fl:1.12345} |
++
1 row selected (0.169 seconds)
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select avg(t.soa[1]) from
`complex.json` t;
+--+
| |
+--+
+--+
No rows selected (0.164 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2457) aggregation function on complex type should give appropriate error
[
https://issues.apache.org/jira/browse/DRILL-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2457:
-
Assignee: Chun Chang (was: Parth Chandra)
aggregation function on complex type should give appropriate error
---
Key: DRILL-2457
URL: https://issues.apache.org/jira/browse/DRILL-2457
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.8.0
Reporter: Chun Chang
Assignee: Chun Chang
Priority: Critical
Fix For: 0.9.0
#Fri Mar 06 16:51:10 EST 2015
git.commit.id.abbrev=fb293ba
Apply aggregation function on complex data should error out with appropriate
error message instead of returning null.
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select t.soa[1] from
`complex.json` t limit 1;
++
| EXPR$0 |
++
| {in:1,fl:1.12345} |
++
1 row selected (0.169 seconds)
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select avg(t.soa[1]) from
`complex.json` t;
+--+
| |
+--+
+--+
No rows selected (0.164 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-1446) C++ Client. Make Drill Client Logger thread safe
[ https://issues.apache.org/jira/browse/DRILL-1446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1446: - Fix Version/s: (was: 1.0.0) 1.1.0 C++ Client. Make Drill Client Logger thread safe Key: DRILL-1446 URL: https://issues.apache.org/jira/browse/DRILL-1446 Project: Apache Drill Issue Type: Bug Components: Client - C++ Reporter: Xiao Meng Assignee: Parth Chandra Priority: Minor Fix For: 1.1.0 The C++ client uses ostream/ofstream to log which is not thread-safe. If multiple thread writing on the logger file, it causes undefined behaviors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2192) DrillScanRel should differentiate skip-all scan-all scan-some semantics while creating a GroupScan [umbrella]
[ https://issues.apache.org/jira/browse/DRILL-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481493#comment-14481493 ] Parth Chandra commented on DRILL-2192: -- Reassigning to Steven for the two remaining subtasks. DrillScanRel should differentiate skip-all scan-all scan-some semantics while creating a GroupScan [umbrella] - Key: DRILL-2192 URL: https://issues.apache.org/jira/browse/DRILL-2192 Project: Apache Drill Issue Type: Improvement Components: Query Planning Optimization Reporter: Hanifi Gunes Assignee: Steven Phillips Fix For: 1.0.0 DrillScanRel passes a list of columns to be read into GroupScan. Currently the logic here is to scan all of the columns even if planner asks to skip them all. Skipping all of the columns is particularly beneficial for the case of count(star) that is translated to count(constant) where we just need row count but not the actual data. The idea is to distinguish three separate states depending on the output coming from planner as follows: | list of columns from planner | scan semantics | | null | scan-all | | empty list of columns | skip-all | | non-empty list of columns w/o star | scan-some | | list of columns with star | scan-all | As part this umbrella, we should make readers understand skip-all semantics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2192) DrillScanRel should differentiate skip-all scan-all scan-some semantics while creating a GroupScan [umbrella]
[ https://issues.apache.org/jira/browse/DRILL-2192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2192: - Assignee: Steven Phillips (was: Hanifi Gunes) DrillScanRel should differentiate skip-all scan-all scan-some semantics while creating a GroupScan [umbrella] - Key: DRILL-2192 URL: https://issues.apache.org/jira/browse/DRILL-2192 Project: Apache Drill Issue Type: Improvement Components: Query Planning Optimization Reporter: Hanifi Gunes Assignee: Steven Phillips Fix For: 1.0.0 DrillScanRel passes a list of columns to be read into GroupScan. Currently the logic here is to scan all of the columns even if planner asks to skip them all. Skipping all of the columns is particularly beneficial for the case of count(star) that is translated to count(constant) where we just need row count but not the actual data. The idea is to distinguish three separate states depending on the output coming from planner as follows: | list of columns from planner | scan semantics | | null | scan-all | | empty list of columns | skip-all | | non-empty list of columns w/o star | scan-some | | list of columns with star | scan-all | As part this umbrella, we should make readers understand skip-all semantics. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1890) Parquet reader throws an cryptic exception when reading a file with colliding column names accoring to Drill's case-insensitive field matching
[ https://issues.apache.org/jira/browse/DRILL-1890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1890: - Assignee: Steven Phillips (was: Parth Chandra) Parquet reader throws an cryptic exception when reading a file with colliding column names accoring to Drill's case-insensitive field matching -- Key: DRILL-1890 URL: https://issues.apache.org/jira/browse/DRILL-1890 Project: Apache Drill Issue Type: Bug Components: Storage - Parquet Reporter: Jason Altekruse Assignee: Steven Phillips Priority: Minor Fix For: 1.0.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2457) aggregation function on complex type should give appropriate error
[
https://issues.apache.org/jira/browse/DRILL-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481590#comment-14481590
]
Chun Chang commented on DRILL-2457:
---
[~parthc] I will lower the priority to minor.
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select avg(t.soa[1]) from
`complex.json` t;
Query failed: RemoteRpcException: Failure while running fragment., Failure
while materializing expression.
Error in expression at index -1. Error: Missing function implementation:
[$sum0(MAP-REQUIRED)]. Full expression: --UNKNOWN EXPRESSION--.
Error in expression at index -1. Error: Missing function implementation:
[count(MAP-REQUIRED)]. Full expression: --UNKNOWN EXPRESSION--. [
0a36eadd-802a-49df-a2b8-6718e4f2b2c4 on qa-node118.qa.lab:31010 ]
[ 0a36eadd-802a-49df-a2b8-6718e4f2b2c4 on qa-node118.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
aggregation function on complex type should give appropriate error
---
Key: DRILL-2457
URL: https://issues.apache.org/jira/browse/DRILL-2457
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.8.0
Reporter: Chun Chang
Assignee: Chun Chang
Priority: Critical
Fix For: 0.9.0
#Fri Mar 06 16:51:10 EST 2015
git.commit.id.abbrev=fb293ba
Apply aggregation function on complex data should error out with appropriate
error message instead of returning null.
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select t.soa[1] from
`complex.json` t limit 1;
++
| EXPR$0 |
++
| {in:1,fl:1.12345} |
++
1 row selected (0.169 seconds)
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select avg(t.soa[1]) from
`complex.json` t;
+--+
| |
+--+
+--+
No rows selected (0.164 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2409) Drill profile page mishandles statistics from long running queries
[ https://issues.apache.org/jira/browse/DRILL-2409?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Westin updated DRILL-2409: Fix Version/s: (was: 0.9.0) 1.0.0 Drill profile page mishandles statistics from long running queries -- Key: DRILL-2409 URL: https://issues.apache.org/jira/browse/DRILL-2409 Project: Apache Drill Issue Type: Bug Components: Client - HTTP Reporter: Jacques Nadeau Assignee: Sudheesh Katkam Fix For: 1.0.0 We recently ran a 72 hour query that joined several trillion records for a customer. While the query completed successfully, the presentation on the profile page had a number of problems. This included times not being correctly reported (they were truncated) and the Gant timeline being unreadable (since it doesn't scale the axes from seconds). We should correct these. (For durations specifically, we should really be present as 4m 2s, 7h 4m or 7d 4h 4m instead of 07:04:02 since we're talking about durations and not times.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2697) Pause injections should pause indefinitely until signalled
Sudheesh Katkam created DRILL-2697: -- Summary: Pause injections should pause indefinitely until signalled Key: DRILL-2697 URL: https://issues.apache.org/jira/browse/DRILL-2697 Project: Apache Drill Issue Type: Improvement Components: Execution - Flow Affects Versions: 0.9.0 Reporter: Sudheesh Katkam Fix For: 1.1.0 Currently injected pauses make threads sleep for a specified time. This can be an enhanced to stop the thread indefinitely using a CountDownLatch. It is quite similar to how cancellation works. Tasks: (a) Add another message to RPC layer to signal paused remote threads to resume (through ControlHandler) by counting down. Complications if the thread has not reached the pause site yet. (b) Add resume signal (like ctrl-c) to sqlline (further enhancement: another signal to trigger pause from sqlline) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2409) Drill profile page mishandles statistics from long running queries
[ https://issues.apache.org/jira/browse/DRILL-2409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481594#comment-14481594 ] Chris Westin commented on DRILL-2409: - This sounds like a superset of DRILL-2533. Drill profile page mishandles statistics from long running queries -- Key: DRILL-2409 URL: https://issues.apache.org/jira/browse/DRILL-2409 Project: Apache Drill Issue Type: Bug Components: Client - HTTP Reporter: Jacques Nadeau Assignee: Sudheesh Katkam Fix For: 1.0.0 We recently ran a 72 hour query that joined several trillion records for a customer. While the query completed successfully, the presentation on the profile page had a number of problems. This included times not being correctly reported (they were truncated) and the Gant timeline being unreadable (since it doesn't scale the axes from seconds). We should correct these. (For durations specifically, we should really be present as 4m 2s, 7h 4m or 7d 4h 4m instead of 07:04:02 since we're talking about durations and not times.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2688) Use of ORDER BY on right side of Union All results in SqlValidatorException
[
https://issues.apache.org/jira/browse/DRILL-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481864#comment-14481864
]
Sean Hsuan-Yi Chu commented on DRILL-2688:
--
Indeed, it is a bug. Calcite believes order by is associated with outer query
(which violates the SQL standard).
A simple workaround is to use parenthesis to help Calcite. Given the simplicity
of the workaround, this issue is downgraded to minor.
Thanks Khurram and Abhishek!
Use of ORDER BY on right side of Union All results in SqlValidatorException
---
Key: DRILL-2688
URL: https://issues.apache.org/jira/browse/DRILL-2688
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.9.0
Environment: 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580:
Exit early from HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53
EDT
Reporter: Khurram Faraaz
Assignee: Sean Hsuan-Yi Chu
Order by in the right leg of Union All results in SqlValidatorException.
Executing the query individually (the query that is on right side of Union
All) gives expected results and no Exceptions are seen. However, executing
the query as part of the right leg of Union All results in Exception.
Tests were executed on 4 node cluster on CentOS. Both sides of Union All got
input from CSV files.
The query below returns correct results when executed individually. (this is
the same query used on right side of Union All query)
{code}
0: jdbc:drill: select columns[0] from `testWindow.csv` t2 where
t2.columns[0] is not null group by columns[0] order by columns[0];
++
| EXPR$0 |
++
| 10 |
| 100|
| 113|
| 119|
| 2 |
| 50 |
| 55 |
| 57 |
| 61 |
| 67 |
| 89 |
++
11 rows selected (0.265 seconds)
{code}
Note that the Union All query works when there is no order by in the right
leg.
{code}
0: jdbc:drill: select columns[0] from `employee.csv` t1 where
t1.columns[0]=1 union all select columns[0] from `testWindow.csv` t2 where
t2.columns[0] is not null group by columns[0];
++
| EXPR$0 |
++
| 1 |
| 100|
| 10 |
| 2 |
| 50 |
| 55 |
| 67 |
| 113|
| 119|
| 89 |
| 57 |
| 61 |
++
12 rows selected (0.175 seconds)
{code}
In this case we do use an order by on the right side of Union All, which
results in SqlValidatorException. Same query as above, except that we have an
order by clause in the query on the right side of Union All.
{code}
0: jdbc:drill: select columns[0] from `employee.csv` t1 where
t1.columns[0]=1 union all select columns[0] from `testWindow.csv` t2 where
t2.columns[0] is not null group by columns[0] order by columns[0];
Query failed: SqlValidatorException: Column 'columns' not found in any table
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Stack trace from drillbit.log
{code}
2015-04-04 04:31:22,882 [2ae096e5-4085-de95-5642-ae10d320a8f7:foreman] ERROR
o.a.drill.exec.work.foreman.Foreman - Error
c6a60f71-c959-4ae5-ba8c-23b9c7b6bb7e: SqlValidatorException: Column 'columns'
not found in any table
org.apache.drill.exec.planner.sql.QueryInputException: Failure validating
SQL. org.eigenbase.util.EigenbaseContextException: From line 1, column 178 to
line 1, column 184: Column 'columns' not found in any table
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:147)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:773)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:204)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: net.hydromatic.optiq.tools.ValidationException:
org.eigenbase.util.EigenbaseContextException: From line 1, column 178 to line
1, column 184: Column 'columns' not found in any table
at
net.hydromatic.optiq.prepare.PlannerImpl.validate(PlannerImpl.java:176)
~[optiq-core-0.9-drill-r20.jar:na]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:157)
[jira] [Commented] (DRILL-2591) Aggregate in left input to Union All does not work
[
https://issues.apache.org/jira/browse/DRILL-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481875#comment-14481875
]
Sean Hsuan-Yi Chu commented on DRILL-2591:
--
Review done at:
https://reviews.apache.org/r/32590/
Commit #: a53e12336c29b421f1df51da480af9a65d70bb72
Aggregate in left input to Union All does not work
--
Key: DRILL-2591
URL: https://issues.apache.org/jira/browse/DRILL-2591
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.9.0
Environment: {code}
0: jdbc:drill: select * from sys.version;
+++-+-++
| commit_id | commit_message | commit_time | build_email | build_time |
+++-+-++
| 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580: Exit early from
HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53 EDT | Unknown
| 26.03.2015 @ 16:53:21 EDT |
+++-+-++
1 row selected (0.104 seconds)
{code}
Reporter: Khurram Faraaz
Assignee: Sean Hsuan-Yi Chu
Attachments: DRILL-2591.1.patch
If the left input to Union All has an aggregate function, the result is
SQLException. This was seen on a 4 node cluster.
{code}
0: jdbc:drill: select max(key) from `dateData.json` union all select key
from `timeStmpData.json`;
++
| EXPR$0 |
++
Query failed: Query stopped., Schema change detected in the left input of
Union-All. This is not currently supported [
441285d7-e4a5-46c8-ab11-a0332945e3fc on centos-04.qa.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Stack trace from drillbit.log
{code}
2015-03-27 00:29:09,795 [2aeb5baa-5af0-ac70-b49a-53e61c92be51:frag:0:0] ERROR
o.a.drill.exec.work.foreman.Foreman - Error
e3ad43f5-fda6-48e5-9e74-779c69bb3cb2: RemoteRpcException: Failure while
running fragment., Schema change detected in the left input of Union-All.
This is not currently supported [ c2c7add0-651b-44d8-9a7c-3218761098e4 on
centos-04.qa.lab:31010 ]
[ c2c7add0-651b-44d8-9a7c-3218761098e4 on centos-04.qa.lab:31010 ]
org.apache.drill.exec.rpc.RemoteRpcException: Failure while running
fragment., Schema change detected in the left input of Union-All. This is not
currently supported [ c2c7add0-651b-44d8-9a7c-3218761098e4 on
centos-04.qa.lab:31010 ]
[ c2c7add0-651b-44d8-9a7c-3218761098e4 on centos-04.qa.lab:31010 ]
at
org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:163)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:281)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:114)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:110)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:230)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:165)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
2015-03-27 00:29:09,796 [2aeb5baa-5af0-ac70-b49a-53e61c92be51:frag:0:0] WARN
o.a.d.e.w.fragment.FragmentExecutor - Error while initializing or executing
fragment
java.lang.RuntimeException: Error closing fragment context.
at
[jira] [Resolved] (DRILL-2591) Aggregate in left input to Union All does not work
[
https://issues.apache.org/jira/browse/DRILL-2591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Hsuan-Yi Chu resolved DRILL-2591.
--
Resolution: Fixed
Aggregate in left input to Union All does not work
--
Key: DRILL-2591
URL: https://issues.apache.org/jira/browse/DRILL-2591
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.9.0
Environment: {code}
0: jdbc:drill: select * from sys.version;
+++-+-++
| commit_id | commit_message | commit_time | build_email | build_time |
+++-+-++
| 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580: Exit early from
HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53 EDT | Unknown
| 26.03.2015 @ 16:53:21 EDT |
+++-+-++
1 row selected (0.104 seconds)
{code}
Reporter: Khurram Faraaz
Assignee: Sean Hsuan-Yi Chu
Attachments: DRILL-2591.1.patch
If the left input to Union All has an aggregate function, the result is
SQLException. This was seen on a 4 node cluster.
{code}
0: jdbc:drill: select max(key) from `dateData.json` union all select key
from `timeStmpData.json`;
++
| EXPR$0 |
++
Query failed: Query stopped., Schema change detected in the left input of
Union-All. This is not currently supported [
441285d7-e4a5-46c8-ab11-a0332945e3fc on centos-04.qa.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Stack trace from drillbit.log
{code}
2015-03-27 00:29:09,795 [2aeb5baa-5af0-ac70-b49a-53e61c92be51:frag:0:0] ERROR
o.a.drill.exec.work.foreman.Foreman - Error
e3ad43f5-fda6-48e5-9e74-779c69bb3cb2: RemoteRpcException: Failure while
running fragment., Schema change detected in the left input of Union-All.
This is not currently supported [ c2c7add0-651b-44d8-9a7c-3218761098e4 on
centos-04.qa.lab:31010 ]
[ c2c7add0-651b-44d8-9a7c-3218761098e4 on centos-04.qa.lab:31010 ]
org.apache.drill.exec.rpc.RemoteRpcException: Failure while running
fragment., Schema change detected in the left input of Union-All. This is not
currently supported [ c2c7add0-651b-44d8-9a7c-3218761098e4 on
centos-04.qa.lab:31010 ]
[ c2c7add0-651b-44d8-9a7c-3218761098e4 on centos-04.qa.lab:31010 ]
at
org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:163)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:281)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:114)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:110)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:230)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:165)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
2015-03-27 00:29:09,796 [2aeb5baa-5af0-ac70-b49a-53e61c92be51:frag:0:0] WARN
o.a.d.e.w.fragment.FragmentExecutor - Error while initializing or executing
fragment
java.lang.RuntimeException: Error closing fragment context.
at
org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:224)
[jira] [Updated] (DRILL-1924) Unexpected exception in TRUNC function when unsupported data type is passed in
[
https://issues.apache.org/jira/browse/DRILL-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-1924:
-
Assignee: Mehant Baid
Unexpected exception in TRUNC function when unsupported data type is passed in
--
Key: DRILL-1924
URL: https://issues.apache.org/jira/browse/DRILL-1924
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.8.0
Reporter: Victoria Markman
Assignee: Mehant Baid
Priority: Minor
Fix For: 1.0.0
t.json
{code}
{ a1: 0, b1: 0, c1: true, d1 : 2015-01-02}
{ a1: 0, b1: 0, c1: false , d1 : 2015-01-03}
{ a1: 0, b1: 0, c1: false , d1 : 2015-01-04}
{ a1: 1, b1: 1, c1: true , d1 : 2015-01-05}
{ a1: 1, b1: 1, c1: true , d1 : 2015-01-06}
{code}
{code}
0: jdbc:drill:schema=dfs select trunc(c1) from `t.json`;
Query failed: Query stopped., Failure while trying to materialize incoming
schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[castTINYINT(VARCHAR-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
8fa0b28d-da4a-4c73-81df-823033dedc35 on atsqa4-133.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
0: jdbc:drill:schema=dfs select trunc(cast(d1 as timestamp)) from `t.json`;
Query failed: Query stopped., Failure while trying to materialize incoming
schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[trunc(TIMESTAMP-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
2a6783c4-e111-47dc-ba67-a767a111e3a9 on atsqa4-133.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
0: jdbc:drill:schema=dfs select trunc(cast(d1 as date)) from `t.json`;
Query failed: Query stopped., Failure while trying to materialize incoming
schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[trunc(DATE-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
65de17df-5c57-44e0-b5ea-653d0ad6afb9 on atsqa4-133.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
It also works on boolean data type, whatever it means ...
{code}
0: jdbc:drill:schema=dfs select cast(c1 as boolean) from `t.json`;
++
| EXPR$0 |
++
| true |
| false |
| false |
| true |
| true |
++
5 rows selected (0.066 seconds)
{code}
TRUNC function is listed in external documentation as math function.
Please throw an error message when any other data type is passed in.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-1706) date_sub function does not accept string as input in Drill
[
https://issues.apache.org/jira/browse/DRILL-1706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-1706:
-
Fix Version/s: (was: 0.9.0)
1.0.0
date_sub function does not accept string as input in Drill
--
Key: DRILL-1706
URL: https://issues.apache.org/jira/browse/DRILL-1706
Project: Apache Drill
Issue Type: New Feature
Components: Functions - Drill, Functions - Hive
Affects Versions: 0.7.0
Reporter: Hao Zhu
Assignee: Mehant Baid
Fix For: 1.0.0
date_sub function does not accept string as input in Drill, however it does
in Hive.
This different behavior of the function will make customer re-write their
query to use cast as date.
Minimum reproduce :
{code}
0: jdbc:drill:zk=local select date_sub('2014-11-12 16:45:22',15) from
dfs.tmp.`drilltest/test.csv` ;
Query failed: Failure while running fragment., Invalid format: 2014-11-12
16:45:22 is malformed at 14-11-12 16:45:22 [
9a6f18da-eb1e-4d91-879a-8d9d528efd59 on 10.250.0.115:31010 ]
(java.lang.IllegalArgumentException) Invalid format: 2014-11-12 16:45:22
is malformed at 14-11-12 16:45:22
org.joda.time.format.DateTimeFormatter.parseDateTime():873
org.apache.drill.exec.test.generated.ProjectorGen23.doSetup():63
org.apache.drill.exec.test.generated.ProjectorGen23.setup():97
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema():427
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.buildSchema():270
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.buildSchema():80
org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.buildSchema():95
org.apache.drill.exec.work.fragment.FragmentExecutor.run():111
org.apache.drill.exec.work.WorkManager$RunnableWrapper.run():249
...():0
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Comparing to Hive which is good.
{code}
0: jdbc:hive2://n1a:1/default select date_sub('2014-11-12 16:45:22',15)
from passwords limit 1 ;
+-+
| _c0 |
+-+
| 2014-10-28 |
+-+
1 row selected (6.568 seconds)
{code}
Workaround in Drill:
{code}
0: jdbc:drill:zk=local select date_sub(cast('2014-11-12 16:45:22' as
date),15) from dfs.tmp.`drilltest/test.csv` ;
++
| EXPR$0 |
++
| 2014-10-28 |
++
1 row selected (0.082 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2455) Cluster utility
[ https://issues.apache.org/jira/browse/DRILL-2455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Westin updated DRILL-2455: Fix Version/s: Future Cluster utility --- Key: DRILL-2455 URL: https://issues.apache.org/jira/browse/DRILL-2455 Project: Apache Drill Issue Type: New Feature Components: Tools, Build Test Reporter: Sudheesh Katkam Assignee: Sudheesh Katkam Fix For: Future A bash utility to -download and deploy- *manage* Drill cluster will be useful for users and developers. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2691) Source files with Windows line endings
[
https://issues.apache.org/jira/browse/DRILL-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deneche A. Hakim updated DRILL-2691:
Assignee: Jacques Nadeau (was: Deneche A. Hakim)
Source files with Windows line endings
--
Key: DRILL-2691
URL: https://issues.apache.org/jira/browse/DRILL-2691
Project: Apache Drill
Issue Type: Bug
Affects Versions: 0.6.0
Reporter: Deneche A. Hakim
Assignee: Jacques Nadeau
Fix For: 0.9.0
Attachments: DRILL-2691.1.patch.txt
The following files:
{noformat}
common/src/main/java/org/apache/drill/common/util/DrillStringUtils.java
contrib/storage-hbase/src/test/java/org/apache/drill/hbase/TestHBaseCFAsJSONString.java
{noformat}
Have Windows line endings in them. Trying to apply a patch that contains
changes in one of those files will fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2318) Query fails when an ORDER BY clause is used with WITH-CLAUSE
[
https://issues.apache.org/jira/browse/DRILL-2318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481856#comment-14481856
]
Sean Hsuan-Yi Chu commented on DRILL-2318:
--
Resolved in Calcite side:
https://issues.apache.org/jira/browse/CALCITE-662
This issue will be gone after rebasing is done.
Query fails when an ORDER BY clause is used with WITH-CLAUSE
Key: DRILL-2318
URL: https://issues.apache.org/jira/browse/DRILL-2318
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.8.0
Reporter: Abhishek Girish
Assignee: Sean Hsuan-Yi Chu
Fix For: 1.0.0
Attachments: drillbit.log
Adding a WITH clause with a simple CTE causes a query with an ORDER BY to
fail. This happens even when the CTE is unrelated to the main query.
*The following query fails to execute:*
{code:sql}
WITH
x
AS (SELECT ss_sold_date_sk a1
FROM store_sales)
SELECT x.a1
FROM x
ORDER BY
x.a1;
{code}
Error:
Query failed: SqlValidatorException: Table 'x' not found
Log attached.
*The following query executes fine:*
{code:sql}
WITH
x
AS (SELECT ss_sold_date_sk a1
FROM store_sales)
SELECT x.a1
FROM x
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-2637) Schema change reported incorrectly although both the input columns are of same datatype
[
https://issues.apache.org/jira/browse/DRILL-2637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Hsuan-Yi Chu resolved DRILL-2637.
--
Resolution: Fixed
Schema change reported incorrectly although both the input columns are of
same datatype
---
Key: DRILL-2637
URL: https://issues.apache.org/jira/browse/DRILL-2637
Project: Apache Drill
Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Khurram Faraaz
Assignee: Sean Hsuan-Yi Chu
Schema change is reported incorrectly, although the two columns hold data of
same datatype, and those two columns are input to Union All and an aggregate
count is performed on the results returned by Union All.
Tests were run on 4 node cluster.
{code}
0: jdbc:drill: select c1 from (select columns[0] c1 from `testWindow.csv`)
union all (select columns[0] c2 from `testWindow.csv`);
++
| c1 |
++
| 100|
| 10 |
| 2 |
| 50 |
| 55 |
| 67 |
| 113|
| 119|
| 89 |
| 57 |
| 61 |
| 100|
| 10 |
| 2 |
| 50 |
| 55 |
| 67 |
| 113|
| 119|
| 89 |
| 57 |
| 61 |
++
22 rows selected (0.121 seconds)
{code}
{code}
0: jdbc:drill: select count(c1) from (select columns[0] c1 from
`testWindow.csv`) union all (select columns[0] c2 from `testWindow.csv`);
++
| EXPR$0 |
++
Query failed: Query stopped., Schema change detected in the left input of
Union-All. This is not currently supported [
57dd6384-fb23-4ab0-aee9-fb7def390788 on centos-04.qa.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Stack trace from drillbit.log
{code}
2015-03-31 20:10:07,825 [2ae500df-db85-2583-fa7f-b89beb7e5ac0:frag:0:0] ERROR
o.a.drill.exec.work.foreman.Foreman - Error
0b4d9b3a-d8af-4dc9-be47-46c4547a793a: RemoteRpcException: Failure while
running fragment., Schema change detected in the left input of Union-All.
This is not currently supported [ b9555eb8-c009-4e9c-b058-ffae3f015df7 on
centos-04.qa.lab:31010 ]
[ b9555eb8-c009-4e9c-b058-ffae3f015df7 on centos-04.qa.lab:31010 ]
org.apache.drill.exec.rpc.RemoteRpcException: Failure while running
fragment., Schema change detected in the left input of Union-All. This is not
currently supported [ b9555eb8-c009-4e9c-b058-ffae3f015df7 on
centos-04.qa.lab:31010 ]
[ b9555eb8-c009-4e9c-b058-ffae3f015df7 on centos-04.qa.lab:31010 ]
at
org.apache.drill.exec.work.foreman.QueryManager.statusUpdate(QueryManager.java:163)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:281)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:114)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:110)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.internalFail(FragmentExecutor.java:230)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:165)
[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
2015-03-31 20:10:07,825 [2ae500df-db85-2583-fa7f-b89beb7e5ac0:frag:0:0] WARN
o.a.d.e.w.fragment.FragmentExecutor - Error while initializing or executing
fragment
java.lang.RuntimeException: Error closing fragment
[jira] [Updated] (DRILL-2200) Obscure error message from kvgen when the field does not exist in a parquet file
[
https://issues.apache.org/jira/browse/DRILL-2200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2200:
-
Fix Version/s: (was: 1.0.0)
1.1.0
Obscure error message from kvgen when the field does not exist in a parquet
file
Key: DRILL-2200
URL: https://issues.apache.org/jira/browse/DRILL-2200
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Reporter: Rahul Challapalli
Assignee: Mehant Baid
Priority: Minor
Fix For: 1.1.0
Attachments: data.parquet
git.commit.id.abbrev=3d863b5
Json Data Set :
{code}
{
id:1,
map: {
map1:{col1:1, col2:2}
}
}
{code}
Equivalent Parquet File is attached.
The below response properly indicates the reason for failure
{code}
select kvgen(t.map.abcd) from dfs.flatten_operators.`data.json` t;
Query failed: RemoteRpcException: Failure while running fragment., kvgen
function only supports Simple maps as input [
9b441359-9f04-4199-8cca-cdc23aaab517 on qa-node191.qa.lab:31010 ]
[ 9b441359-9f04-4199-8cca-cdc23aaab517 on qa-node191.qa.lab:31010 ]
{code}
However the same query on an equivalent parquet file returns an obscure error
message
{code}
select kvgen(t.map.abcd) from `data.parquet` t;
Query failed: RemoteRpcException: Failure while running fragment., Unable to
find holder type for minorType: LATE [ 9697f4aa-1e8d-4221-8e7f-4d374ffbd916
on qa-node191.qa.lab:31010 ]
[ 9697f4aa-1e8d-4221-8e7f-4d374ffbd916 on qa-node191.qa.lab:31010 ]
{code}
However if we apply kvgen on a first level non-existent field we get back a
proper error message even for a parquet file
{code}
select kvgen(t.map1) from `data.parquet` t;
Query failed: RemoteRpcException: Failure while running fragment., kvgen
function only supports Simple maps as input [
1d1c4ca3-ceda-4320-8ff8-fe08afe32090 on qa-node191.qa.lab:31010 ]
[ 1d1c4ca3-ceda-4320-8ff8-fe08afe32090 on qa-node191.qa.lab:31010 ]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-2226) Create test utilities for checking plans for patterns
[ https://issues.apache.org/jira/browse/DRILL-2226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-2226. Resolution: Fixed Fix Version/s: (was: 1.0.0) 0.8.0 Fixed in ed397862eb9584572aa0fcb684dfc9554b00cf60 Create test utilities for checking plans for patterns - Key: DRILL-2226 URL: https://issues.apache.org/jira/browse/DRILL-2226 Project: Apache Drill Issue Type: Improvement Components: Tools, Build Test Reporter: Jason Altekruse Assignee: Jason Altekruse Fix For: 0.8.0 Attachments: DRILL-2226.patch Regex matching for calcite text format plans, includes expected and excluded pattern matching. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1914) Commands given in SQLLine !help don't work as expected
[ https://issues.apache.org/jira/browse/DRILL-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1914: - Fix Version/s: (was: 1.0.0) 1.1.0 Commands given in SQLLine !help don't work as expected -- Key: DRILL-1914 URL: https://issues.apache.org/jira/browse/DRILL-1914 Project: Apache Drill Issue Type: Wish Components: Client - CLI Affects Versions: 0.6.0 Reporter: irfan Assignee: Daniel Barclay (Drill) Priority: Minor Fix For: 1.1.0 how to use help utility in apache drill. we have tried with !help it's giving list like !sql,!tables ,!run etc. we want to know the usage of reserved keywords like RANK and ROW_NUM. please respond if you have any idea -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (DRILL-2639) Planner bug - RelOptPlanner.CannotPlanException
[
https://issues.apache.org/jira/browse/DRILL-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481872#comment-14481872
]
Sean Hsuan-Yi Chu edited comment on DRILL-2639 at 4/6/15 8:27 PM:
--
DRILL-600 was based on the previous implementation of Union-All. Basically, we
re-did a brand new one. Thanks!!!
was (Author: seanhychu):
DRILL-600 was based on the previous implementation of Union-All. Basically, we
re-did a brand new one.
Planner bug - RelOptPlanner.CannotPlanException
---
Key: DRILL-2639
URL: https://issues.apache.org/jira/browse/DRILL-2639
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.9.0
Environment: | 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580:
Exit early from HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53
EDT | Unknown | 26.03.2015 @ 16:53:21 EDT |
Reporter: Khurram Faraaz
Assignee: Sean Hsuan-Yi Chu
Priority: Critical
Fix For: 0.9.0
Reporting this as a separate JIRA as this issue related to a bug in the
planner. Performing aggregate on the output returned by Union All results in
CannotPlanException. Note that the two inputs to Union All are casted to
integer and hence the inputs from both legs are of the same datatype.
{code}
0: jdbc:drill: select count(c1) from (select cast(columns[0] as int) c1 from
`testWindow.csv`) union all (select cast(columns[0] as int) c2 from
`testWindow.csv`);
Query failed: RelOptPlanner.CannotPlanException: Node
[rel#59393:Subset#4.LOGICAL.ANY([]).[]] could not be implemented; planner
state:
Root: rel#59393:Subset#4.LOGICAL.ANY([]).[]
Original rel:
AbstractConverter(subset=[rel#59393:Subset#4.LOGICAL.ANY([]).[]],
convention=[LOGICAL], DrillDistributionTraitDef=[ANY([])], sort=[[]]):
rowcount = 1.7976931348623157E308, cumulative cost = {inf}, id = 59394
UnionRel(subset=[rel#59392:Subset#4.NONE.ANY([]).[]], all=[true]): rowcount
= 1.7976931348623157E308, cumulative cost = {1.7976931348623157E308 rows,
1.7976931348623157E308 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 59391
AggregateRel(subset=[rel#59388:Subset#2.NONE.ANY([]).[]], group=[{}],
EXPR$0=[COUNT($0)]): rowcount = 1.7976931348623158E307, cumulative cost =
{1.7976931348623158E307 rows, 0.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id =
59387
ProjectRel(subset=[rel#59386:Subset#1.NONE.ANY([]).[]],
c1=[CAST(ITEM($1, 0)):INTEGER]): rowcount = 100.0, cumulative cost = {100.0
rows, 100.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 59385
EnumerableTableAccessRel(subset=[rel#59384:Subset#0.ENUMERABLE.ANY([]).[]],
table=[[dfs, tmp, testWindow.csv]]): rowcount = 100.0, cumulative cost =
{100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 59368
ProjectRel(subset=[rel#59390:Subset#3.NONE.ANY([]).[]], c2=[CAST(ITEM($1,
0)):INTEGER]): rowcount = 100.0, cumulative cost = {100.0 rows, 100.0 cpu,
0.0 io, 0.0 network, 0.0 memory}, id = 59389
EnumerableTableAccessRel(subset=[rel#59384:Subset#0.ENUMERABLE.ANY([]).[]],
table=[[dfs, tmp, testWindow.csv]]): rowcount = 100.0, cumulative cost =
{100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 59368
Sets:
Set#0, type: (DrillRecordRow[*, columns])
rel#59384:Subset#0.ENUMERABLE.ANY([]).[], best=rel#59368,
importance=0.6561
rel#59368:EnumerableTableAccessRel.ENUMERABLE.ANY([]).[](table=[dfs, tmp,
testWindow.csv]), rowcount=100.0, cumulative cost={100.0 rows, 101.0 cpu, 0.0
io, 0.0 network, 0.0 memory}
rel#59408:AbstractConverter.ENUMERABLE.ANY([]).[](child=rel#59407:Subset#0.LOGICAL.ANY([]).[],convention=ENUMERABLE,DrillDistributionTraitDef=ANY([]),sort=[]),
rowcount=1.0, cumulative cost={inf}
rel#59407:Subset#0.LOGICAL.ANY([]).[], best=rel#59415,
importance=0.59049001
rel#59409:AbstractConverter.LOGICAL.ANY([]).[](child=rel#59384:Subset#0.ENUMERABLE.ANY([]).[],convention=LOGICAL,DrillDistributionTraitDef=ANY([]),sort=[]),
rowcount=100.0, cumulative cost={inf}
rel#59415:DrillScanRel.LOGICAL.ANY([]).[](table=[dfs, tmp,
testWindow.csv],groupscan=EasyGroupScan [selectionRoot=/tmp/testWindow.csv,
numFiles=1, columns=[`*`], files=[maprfs:/tmp/testWindow.csv]]),
rowcount=1.0, cumulative cost={1.0 rows, 1.0 cpu, 0.0 io, 0.0 network,
0.0 memory}
Set#1, type: RecordType(INTEGER c1)
rel#59386:Subset#1.NONE.ANY([]).[], best=null,
importance=0.7291
rel#59385:ProjectRel.NONE.ANY([]).[](child=rel#59384:Subset#0.ENUMERABLE.ANY([]).[],c1=CAST(ITEM($1,
0)):INTEGER), rowcount=100.0, cumulative cost={inf}
rel#59404:Subset#1.LOGICAL.ANY([]).[],
[jira] [Updated] (DRILL-2108) Running SQLLine with bad command syntax yields index exception (not usage message)
[ https://issues.apache.org/jira/browse/DRILL-2108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2108: - Fix Version/s: (was: 1.0.0) Future Running SQLLine with bad command syntax yields index exception (not usage message) -- Key: DRILL-2108 URL: https://issues.apache.org/jira/browse/DRILL-2108 Project: Apache Drill Issue Type: Bug Components: Client - CLI Reporter: Daniel Barclay (Drill) Assignee: Daniel Barclay (Drill) Priority: Minor Fix For: Future Running .../sqlline -u yields an ArrayIndexOutOfBoundsException and a Java stack trace (rather than reporting the command syntax error and giving a usage message). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2259) Projecting a nested repeated map results in IOOBE when the data has schema changes
[
https://issues.apache.org/jira/browse/DRILL-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2259:
-
Fix Version/s: (was: 1.0.0)
1.1.0
Projecting a nested repeated map results in IOOBE when the data has schema
changes
--
Key: DRILL-2259
URL: https://issues.apache.org/jira/browse/DRILL-2259
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Reporter: Rahul Challapalli
Assignee: Hanifi Gunes
Fix For: 1.1.0
Attachments: error.log, file1.json, file2.json, file3.json
git.commit.id.abbrev=6676f2d
The data set contains 3 files with schema changes among them. By schema
changes I refer to adding/removing fields among records/files
The below query which does a simple project on a field which does not have a
schema change and is present in all the records
{code}
select d.type type, d.map.rm[1].rptd from `schema_changes` d;
+++
|type| EXPR$1 |
+++
| web| [{a:bar},{c:1},{d:4.5}] |
| store | [{a:bar},{c:-1},{d:4.5}] |
Query failed: RemoteRpcException: Failure while running fragment., index: -4,
length: 4 (expected: range(0, 16384)) [ 289ca604-0ece-45ff-94ec-35c11264d7a8
on qa-node190.qa.lab:31010 ]
[ 289ca604-0ece-45ff-94ec-35c11264d7a8 on qa-node190.qa.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
The below query succeeds when we explicitly read from any one file
{code}
select d.type type, d.map.rm[1].rptd from `schema_changes/file3.json` d;
+++
|type| EXPR$1 |
+++
| web| [{a:bar},{c:1},{d:4.5}] |
| store | [{a:bar},{c:-1},{d:4.5}] |
+++
2 rows selected (0.064 seconds)
{code}
I attached the data set and the error from the logs
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2456) regexp_replace using hex codes fails on larger JSON data sets
[ https://issues.apache.org/jira/browse/DRILL-2456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2456: - Fix Version/s: (was: 0.9.0) 1.0.0 regexp_replace using hex codes fails on larger JSON data sets - Key: DRILL-2456 URL: https://issues.apache.org/jira/browse/DRILL-2456 Project: Apache Drill Issue Type: Bug Components: Functions - Drill Affects Versions: 0.7.0 Environment: Drill 0.7 MapR 4.0.1 CentOS Reporter: Andries Engelbrecht Assignee: Mehant Baid Fix For: 1.0.0 Attachments: drillbit.log This query works with only 1 file select regexp_replace(`text`, '[^\x20-\xad]', '°'), count(id) from dfs.twitter.`/feed/2015/03/13/17/FlumeData.1426267859699.json` group by `text` order by count(id) desc limit 10; This one fails with multiple files select regexp_replace(`text`, '[^\x20-\xad]', '°'), count(id) from dfs.twitter.`/feed/2015/03/13` group by `text` order by count(id) desc limit 10; Query failed: Query failed: Failure while trying to start remote fragment, Encountered an illegal char on line 1, column 31: '' [ 43ff1aa4-4a71-455d-b817-ec5eb8d179bb on twitternode:31010 ] Using text in regexp_replace does work for same dataset. This query works fine on full data set. select regexp_replace(`text`, '[^ -~¡-ÿ]', '°'), count(id) from dfs.twitter.`/feed/2015/03/13` group by `text` order by count(id) desc limit 10; Attached snippet drillbit.log for error -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2282) Eliminate spaces, special characters from names in function templates
[ https://issues.apache.org/jira/browse/DRILL-2282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2282: - Fix Version/s: (was: 0.9.0) 1.1.0 Eliminate spaces, special characters from names in function templates - Key: DRILL-2282 URL: https://issues.apache.org/jira/browse/DRILL-2282 Project: Apache Drill Issue Type: Bug Components: Functions - Drill Reporter: Mehant Baid Assignee: Mehant Baid Fix For: 1.1.0 Attachments: DRILL-2282.patch Having spaces in the name of the functions causes issues while deserializing such expressions when we try to read the plan fragment. As part of this JIRA would like to clean up all the templates to not include special characters in their names. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2457) aggregation function on complex type should give appropriate error
[
https://issues.apache.org/jira/browse/DRILL-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chang updated DRILL-2457:
--
Priority: Minor (was: Critical)
aggregation function on complex type should give appropriate error
---
Key: DRILL-2457
URL: https://issues.apache.org/jira/browse/DRILL-2457
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.8.0
Reporter: Chun Chang
Assignee: Chun Chang
Priority: Minor
Fix For: 0.9.0
#Fri Mar 06 16:51:10 EST 2015
git.commit.id.abbrev=fb293ba
Apply aggregation function on complex data should error out with appropriate
error message instead of returning null.
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select t.soa[1] from
`complex.json` t limit 1;
++
| EXPR$0 |
++
| {in:1,fl:1.12345} |
++
1 row selected (0.169 seconds)
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select avg(t.soa[1]) from
`complex.json` t;
+--+
| |
+--+
+--+
No rows selected (0.164 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2697) Pause injections should pause indefinitely until signalled
[ https://issues.apache.org/jira/browse/DRILL-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481630#comment-14481630 ] Chris Westin commented on DRILL-2697: - Note this should speed up any tests that require pauses for synchronizing execution lifecycle with test steps, because they wouldn't have to wait for an arbitrary period of time, but can instead progress when ready. If we find such tests are common, and are starting to take a lot of time, we may want to prioritize this. Pause injections should pause indefinitely until signalled -- Key: DRILL-2697 URL: https://issues.apache.org/jira/browse/DRILL-2697 Project: Apache Drill Issue Type: Improvement Components: Execution - Flow Affects Versions: 0.9.0 Reporter: Sudheesh Katkam Fix For: 1.1.0 Currently injected pauses make threads sleep for a specified time. This can be an enhanced to stop the thread indefinitely using a CountDownLatch. It is quite similar to how cancellation works. Tasks: (a) Add another message to RPC layer to signal paused remote threads to resume (through ControlHandler) by counting down. Complications if the thread has not reached the pause site yet. (b) Add resume signal (like ctrl-c) to sqlline (further enhancement: another signal to trigger pause from sqlline) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2688) Use of ORDER BY on right side of Union All results in SqlValidatorException
[
https://issues.apache.org/jira/browse/DRILL-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Hsuan-Yi Chu updated DRILL-2688:
-
Priority: Minor (was: Major)
Use of ORDER BY on right side of Union All results in SqlValidatorException
---
Key: DRILL-2688
URL: https://issues.apache.org/jira/browse/DRILL-2688
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.9.0
Environment: 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580:
Exit early from HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53
EDT
Reporter: Khurram Faraaz
Assignee: Sean Hsuan-Yi Chu
Priority: Minor
Fix For: 1.0.0
Order by in the right leg of Union All results in SqlValidatorException.
Executing the query individually (the query that is on right side of Union
All) gives expected results and no Exceptions are seen. However, executing
the query as part of the right leg of Union All results in Exception.
Tests were executed on 4 node cluster on CentOS. Both sides of Union All got
input from CSV files.
The query below returns correct results when executed individually. (this is
the same query used on right side of Union All query)
{code}
0: jdbc:drill: select columns[0] from `testWindow.csv` t2 where
t2.columns[0] is not null group by columns[0] order by columns[0];
++
| EXPR$0 |
++
| 10 |
| 100|
| 113|
| 119|
| 2 |
| 50 |
| 55 |
| 57 |
| 61 |
| 67 |
| 89 |
++
11 rows selected (0.265 seconds)
{code}
Note that the Union All query works when there is no order by in the right
leg.
{code}
0: jdbc:drill: select columns[0] from `employee.csv` t1 where
t1.columns[0]=1 union all select columns[0] from `testWindow.csv` t2 where
t2.columns[0] is not null group by columns[0];
++
| EXPR$0 |
++
| 1 |
| 100|
| 10 |
| 2 |
| 50 |
| 55 |
| 67 |
| 113|
| 119|
| 89 |
| 57 |
| 61 |
++
12 rows selected (0.175 seconds)
{code}
In this case we do use an order by on the right side of Union All, which
results in SqlValidatorException. Same query as above, except that we have an
order by clause in the query on the right side of Union All.
{code}
0: jdbc:drill: select columns[0] from `employee.csv` t1 where
t1.columns[0]=1 union all select columns[0] from `testWindow.csv` t2 where
t2.columns[0] is not null group by columns[0] order by columns[0];
Query failed: SqlValidatorException: Column 'columns' not found in any table
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Stack trace from drillbit.log
{code}
2015-04-04 04:31:22,882 [2ae096e5-4085-de95-5642-ae10d320a8f7:foreman] ERROR
o.a.drill.exec.work.foreman.Foreman - Error
c6a60f71-c959-4ae5-ba8c-23b9c7b6bb7e: SqlValidatorException: Column 'columns'
not found in any table
org.apache.drill.exec.planner.sql.QueryInputException: Failure validating
SQL. org.eigenbase.util.EigenbaseContextException: From line 1, column 178 to
line 1, column 184: Column 'columns' not found in any table
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:147)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:773)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:204)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: net.hydromatic.optiq.tools.ValidationException:
org.eigenbase.util.EigenbaseContextException: From line 1, column 178 to line
1, column 184: Column 'columns' not found in any table
at
net.hydromatic.optiq.prepare.PlannerImpl.validate(PlannerImpl.java:176)
~[optiq-core-0.9-drill-r20.jar:na]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:157)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:133)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
[jira] [Updated] (DRILL-2688) Use of ORDER BY on right side of Union All results in SqlValidatorException
[
https://issues.apache.org/jira/browse/DRILL-2688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Hsuan-Yi Chu updated DRILL-2688:
-
Fix Version/s: 1.0.0
Use of ORDER BY on right side of Union All results in SqlValidatorException
---
Key: DRILL-2688
URL: https://issues.apache.org/jira/browse/DRILL-2688
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.9.0
Environment: 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580:
Exit early from HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53
EDT
Reporter: Khurram Faraaz
Assignee: Sean Hsuan-Yi Chu
Priority: Minor
Fix For: 1.0.0
Order by in the right leg of Union All results in SqlValidatorException.
Executing the query individually (the query that is on right side of Union
All) gives expected results and no Exceptions are seen. However, executing
the query as part of the right leg of Union All results in Exception.
Tests were executed on 4 node cluster on CentOS. Both sides of Union All got
input from CSV files.
The query below returns correct results when executed individually. (this is
the same query used on right side of Union All query)
{code}
0: jdbc:drill: select columns[0] from `testWindow.csv` t2 where
t2.columns[0] is not null group by columns[0] order by columns[0];
++
| EXPR$0 |
++
| 10 |
| 100|
| 113|
| 119|
| 2 |
| 50 |
| 55 |
| 57 |
| 61 |
| 67 |
| 89 |
++
11 rows selected (0.265 seconds)
{code}
Note that the Union All query works when there is no order by in the right
leg.
{code}
0: jdbc:drill: select columns[0] from `employee.csv` t1 where
t1.columns[0]=1 union all select columns[0] from `testWindow.csv` t2 where
t2.columns[0] is not null group by columns[0];
++
| EXPR$0 |
++
| 1 |
| 100|
| 10 |
| 2 |
| 50 |
| 55 |
| 67 |
| 113|
| 119|
| 89 |
| 57 |
| 61 |
++
12 rows selected (0.175 seconds)
{code}
In this case we do use an order by on the right side of Union All, which
results in SqlValidatorException. Same query as above, except that we have an
order by clause in the query on the right side of Union All.
{code}
0: jdbc:drill: select columns[0] from `employee.csv` t1 where
t1.columns[0]=1 union all select columns[0] from `testWindow.csv` t2 where
t2.columns[0] is not null group by columns[0] order by columns[0];
Query failed: SqlValidatorException: Column 'columns' not found in any table
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Stack trace from drillbit.log
{code}
2015-04-04 04:31:22,882 [2ae096e5-4085-de95-5642-ae10d320a8f7:foreman] ERROR
o.a.drill.exec.work.foreman.Foreman - Error
c6a60f71-c959-4ae5-ba8c-23b9c7b6bb7e: SqlValidatorException: Column 'columns'
not found in any table
org.apache.drill.exec.planner.sql.QueryInputException: Failure validating
SQL. org.eigenbase.util.EigenbaseContextException: From line 1, column 178 to
line 1, column 184: Column 'columns' not found in any table
at
org.apache.drill.exec.planner.sql.DrillSqlWorker.getPlan(DrillSqlWorker.java:147)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.work.foreman.Foreman.runSQL(Foreman.java:773)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at org.apache.drill.exec.work.foreman.Foreman.run(Foreman.java:204)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_75]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_75]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_75]
Caused by: net.hydromatic.optiq.tools.ValidationException:
org.eigenbase.util.EigenbaseContextException: From line 1, column 178 to line
1, column 184: Column 'columns' not found in any table
at
net.hydromatic.optiq.prepare.PlannerImpl.validate(PlannerImpl.java:176)
~[optiq-core-0.9-drill-r20.jar:na]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.validateNode(DefaultSqlHandler.java:157)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.planner.sql.handlers.DefaultSqlHandler.getPlan(DefaultSqlHandler.java:133)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
[jira] [Commented] (DRILL-2639) Planner bug - RelOptPlanner.CannotPlanException
[
https://issues.apache.org/jira/browse/DRILL-2639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481882#comment-14481882
]
Sean Hsuan-Yi Chu commented on DRILL-2639:
--
Under review at: https://reviews.apache.org/r/32886/
Planner bug - RelOptPlanner.CannotPlanException
---
Key: DRILL-2639
URL: https://issues.apache.org/jira/browse/DRILL-2639
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.9.0
Environment: | 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580:
Exit early from HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53
EDT | Unknown | 26.03.2015 @ 16:53:21 EDT |
Reporter: Khurram Faraaz
Assignee: Sean Hsuan-Yi Chu
Priority: Critical
Fix For: 0.9.0
Reporting this as a separate JIRA as this issue related to a bug in the
planner. Performing aggregate on the output returned by Union All results in
CannotPlanException. Note that the two inputs to Union All are casted to
integer and hence the inputs from both legs are of the same datatype.
{code}
0: jdbc:drill: select count(c1) from (select cast(columns[0] as int) c1 from
`testWindow.csv`) union all (select cast(columns[0] as int) c2 from
`testWindow.csv`);
Query failed: RelOptPlanner.CannotPlanException: Node
[rel#59393:Subset#4.LOGICAL.ANY([]).[]] could not be implemented; planner
state:
Root: rel#59393:Subset#4.LOGICAL.ANY([]).[]
Original rel:
AbstractConverter(subset=[rel#59393:Subset#4.LOGICAL.ANY([]).[]],
convention=[LOGICAL], DrillDistributionTraitDef=[ANY([])], sort=[[]]):
rowcount = 1.7976931348623157E308, cumulative cost = {inf}, id = 59394
UnionRel(subset=[rel#59392:Subset#4.NONE.ANY([]).[]], all=[true]): rowcount
= 1.7976931348623157E308, cumulative cost = {1.7976931348623157E308 rows,
1.7976931348623157E308 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 59391
AggregateRel(subset=[rel#59388:Subset#2.NONE.ANY([]).[]], group=[{}],
EXPR$0=[COUNT($0)]): rowcount = 1.7976931348623158E307, cumulative cost =
{1.7976931348623158E307 rows, 0.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id =
59387
ProjectRel(subset=[rel#59386:Subset#1.NONE.ANY([]).[]],
c1=[CAST(ITEM($1, 0)):INTEGER]): rowcount = 100.0, cumulative cost = {100.0
rows, 100.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 59385
EnumerableTableAccessRel(subset=[rel#59384:Subset#0.ENUMERABLE.ANY([]).[]],
table=[[dfs, tmp, testWindow.csv]]): rowcount = 100.0, cumulative cost =
{100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 59368
ProjectRel(subset=[rel#59390:Subset#3.NONE.ANY([]).[]], c2=[CAST(ITEM($1,
0)):INTEGER]): rowcount = 100.0, cumulative cost = {100.0 rows, 100.0 cpu,
0.0 io, 0.0 network, 0.0 memory}, id = 59389
EnumerableTableAccessRel(subset=[rel#59384:Subset#0.ENUMERABLE.ANY([]).[]],
table=[[dfs, tmp, testWindow.csv]]): rowcount = 100.0, cumulative cost =
{100.0 rows, 101.0 cpu, 0.0 io, 0.0 network, 0.0 memory}, id = 59368
Sets:
Set#0, type: (DrillRecordRow[*, columns])
rel#59384:Subset#0.ENUMERABLE.ANY([]).[], best=rel#59368,
importance=0.6561
rel#59368:EnumerableTableAccessRel.ENUMERABLE.ANY([]).[](table=[dfs, tmp,
testWindow.csv]), rowcount=100.0, cumulative cost={100.0 rows, 101.0 cpu, 0.0
io, 0.0 network, 0.0 memory}
rel#59408:AbstractConverter.ENUMERABLE.ANY([]).[](child=rel#59407:Subset#0.LOGICAL.ANY([]).[],convention=ENUMERABLE,DrillDistributionTraitDef=ANY([]),sort=[]),
rowcount=1.0, cumulative cost={inf}
rel#59407:Subset#0.LOGICAL.ANY([]).[], best=rel#59415,
importance=0.59049001
rel#59409:AbstractConverter.LOGICAL.ANY([]).[](child=rel#59384:Subset#0.ENUMERABLE.ANY([]).[],convention=LOGICAL,DrillDistributionTraitDef=ANY([]),sort=[]),
rowcount=100.0, cumulative cost={inf}
rel#59415:DrillScanRel.LOGICAL.ANY([]).[](table=[dfs, tmp,
testWindow.csv],groupscan=EasyGroupScan [selectionRoot=/tmp/testWindow.csv,
numFiles=1, columns=[`*`], files=[maprfs:/tmp/testWindow.csv]]),
rowcount=1.0, cumulative cost={1.0 rows, 1.0 cpu, 0.0 io, 0.0 network,
0.0 memory}
Set#1, type: RecordType(INTEGER c1)
rel#59386:Subset#1.NONE.ANY([]).[], best=null,
importance=0.7291
rel#59385:ProjectRel.NONE.ANY([]).[](child=rel#59384:Subset#0.ENUMERABLE.ANY([]).[],c1=CAST(ITEM($1,
0)):INTEGER), rowcount=100.0, cumulative cost={inf}
rel#59404:Subset#1.LOGICAL.ANY([]).[], best=rel#59413,
importance=0.36455
rel#59405:AbstractConverter.LOGICAL.ANY([]).[](child=rel#59386:Subset#1.NONE.ANY([]).[],convention=LOGICAL,DrillDistributionTraitDef=ANY([]),sort=[]),
rowcount=1.7976931348623157E308,
[jira] [Updated] (DRILL-2561) Profile UI: Metrics displayed incorrectly for failed query
[ https://issues.apache.org/jira/browse/DRILL-2561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Westin updated DRILL-2561: Fix Version/s: (was: 0.9.0) 1.0.0 Profile UI: Metrics displayed incorrectly for failed query -- Key: DRILL-2561 URL: https://issues.apache.org/jira/browse/DRILL-2561 Project: Apache Drill Issue Type: Bug Components: Client - HTTP Affects Versions: 0.9.0 Reporter: Krystal Assignee: Sudheesh Katkam Fix For: 1.0.0 git.commit.id=8493713cafe6e5d1f56f2dffc9d8bea294a6e013 I have a query that failed to execute. The profile UI for this query displayed wrong metrics in columns. Here is the url for that profile: http://10.10.100.115:8047/profiles/2aed1b79-17a0-312d-42a5-161a1c2c66a4 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2653) All additions to storage plugin configuration gets wiped out if info is not correct
[ https://issues.apache.org/jira/browse/DRILL-2653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Westin updated DRILL-2653: Fix Version/s: (was: 0.9.0) 1.0.0 All additions to storage plugin configuration gets wiped out if info is not correct Key: DRILL-2653 URL: https://issues.apache.org/jira/browse/DRILL-2653 Project: Apache Drill Issue Type: Bug Components: Client - HTTP Affects Versions: 0.9.0 Reporter: Krystal Assignee: Sudheesh Katkam Fix For: 1.0.0 Whenever a user enters an invalid information into a storage plugin configuration, the entire data gets wiped out. This can be frustrating to users if alot of information has been entered. The data should be retained on the screen to allow users correct the mistakes rather than having to write all the information over again. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1478) The order of query results for the selected fields seems to be different from sqlline vs Web UI
[
https://issues.apache.org/jira/browse/DRILL-1478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Westin updated DRILL-1478:
Fix Version/s: (was: 0.9.0)
1.0.0
The order of query results for the selected fields seems to be different from
sqlline vs Web UI
---
Key: DRILL-1478
URL: https://issues.apache.org/jira/browse/DRILL-1478
Project: Apache Drill
Issue Type: Bug
Components: Client - HTTP
Environment: I executed a query with aggregation, it seems the order
results is different from sqlline vs Web UI.
Reporter: B Anil Kumar
Assignee: Sudheesh Katkam
Fix For: 1.0.0
Here the order means selected columns order. For example.
{noformat}
0: jdbc:drill:zk=localhost:2181 select state,city,avg(pop) from
mongo.test.`zips` zipcodes group by state, city limit 5;
++++
| state|city| EXPR$2 |
++++
| MA | AGAWAM | 15338.0|
| MA | CUSHMAN| 36963.0|
| MA | BARRE | 4546.0 |
| MA | BELCHERTOWN | 10579.0|
| MA | BLANDFORD | 1240.0 |
++++
{noformat}
The above is as expected. But where as for the same above query in Web UI
{noformat}
EXPR$2 state city
1,240 MA BLANDFORD
4,546 MA BARRE
10,579MA BELCHERTOWN
15,338MA AGAWAM
36,963MA CUSHMAN
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2457) aggregation function on complex type should give appropriate error
[
https://issues.apache.org/jira/browse/DRILL-2457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chun Chang updated DRILL-2457:
--
Assignee: Parth Chandra (was: Chun Chang)
aggregation function on complex type should give appropriate error
---
Key: DRILL-2457
URL: https://issues.apache.org/jira/browse/DRILL-2457
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.8.0
Reporter: Chun Chang
Assignee: Parth Chandra
Priority: Minor
Fix For: 0.9.0
#Fri Mar 06 16:51:10 EST 2015
git.commit.id.abbrev=fb293ba
Apply aggregation function on complex data should error out with appropriate
error message instead of returning null.
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select t.soa[1] from
`complex.json` t limit 1;
++
| EXPR$0 |
++
| {in:1,fl:1.12345} |
++
1 row selected (0.169 seconds)
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select avg(t.soa[1]) from
`complex.json` t;
+--+
| |
+--+
+--+
No rows selected (0.164 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2401) Value 0 for monthOfYear must be in the range [1,12]
[
https://issues.apache.org/jira/browse/DRILL-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14481898#comment-14481898
]
Parth Chandra commented on DRILL-2401:
--
[~cch...@maprtech.com] Can you try to reproduce this.
Value 0 for monthOfYear must be in the range [1,12]
---
Key: DRILL-2401
URL: https://issues.apache.org/jira/browse/DRILL-2401
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.7.0
Environment: Sandbox, SE cluster
Reporter: Sudhakar Thota
Assignee: Chun Chang
Priority: Minor
Fix For: 1.0.0
Attachments: month-00-drill-MapR-DB.rtf
0: jdbc:drill:zk=10.10.15.10:5181,10.10.15.11 select cast(row_key as
varchar(20)) as order_id, cast(t.CF1.`clerk` as varchar(20)) as order_clerk,
cast(t.CF1.orderstatus as varchar(20)) as ord_stat,cast(t.CF1.orderdate as
date) ordrdt from m7thotas.orders t where cast(t.CF1.orderdate as date)
BETWEEN cast('1996-01-10' as date) AND cast('1996-01-11' as date) and
cast(t.CF1.`clerk` as varchar(20))cast('Clerk#00334' as varchar(20));
.
.
.
| 3592807| Clerk#00358 | O | 1996-01-11 |
| 6969024| Clerk#00185 | O | 1996-01-10 |
| 6973990| Clerk#00491 | O | 1996-01-10 |
| 6975170| Clerk#00493 | O | 1996-01-10 |
| 6975462| Clerk#00352 | O | 1996-01-11 |
| 6978503| Clerk#00482 | O | 1996-01-11 |
| 6979654| Clerk#00389 | O | 1996-01-10 |
| 5344355| Clerk#00653 | O | 1996-01-11 |
| 5351969| Clerk#00237 | O | 1996-01-10 |
| 5352003| Clerk#00756 | O | 1996-01-10 |
| 3608519| Clerk#00384 | O | 1996-01-10 |
Query failed: Query failed: Failure while running fragment., Value 0 for
monthOfYear must be in the range [1,12] [
2b41018b-b35e-42a9-bd4e-0307e582b718 on se-node11.se.lab:31010 ]
[ 2b41018b-b35e-42a9-bd4e-0307e582b718 on se-node11.se.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-1626) sqlline echoes statements with unnecessary wrapping
[ https://issues.apache.org/jira/browse/DRILL-1626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1626: - Fix Version/s: (was: 1.0.0) 1.1.0 sqlline echoes statements with unnecessary wrapping --- Key: DRILL-1626 URL: https://issues.apache.org/jira/browse/DRILL-1626 Project: Apache Drill Issue Type: Bug Components: Client - CLI Environment: sqlline running under Eclipse on OSX Reporter: Chris Westin Assignee: Mehant Baid Priority: Minor Fix For: 1.1.0 I type a command in sqlline, and when it is echoed, it is unnecessarily wrapped (making it hard to read, as well as casting uncertainty on what I was doing). Example: 0: jdbc:drill:zk=local create table donuts_parquet as select * from `donuts.json`; create table donuts_parquet as select * from `donuts.jso n`; ++---+ | Fragment | Number of records written | ++---+ | 0_0| 5 | That's sqlline adding a newline after jso and before n in the echoed command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2588) Profile UI: First Start field contains incorrect data
[ https://issues.apache.org/jira/browse/DRILL-2588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Westin updated DRILL-2588: Fix Version/s: (was: 0.9.0) 1.0.0 Profile UI: First Start field contains incorrect data - Key: DRILL-2588 URL: https://issues.apache.org/jira/browse/DRILL-2588 Project: Apache Drill Issue Type: Bug Components: Client - HTTP Affects Versions: 0.9.0 Reporter: Krystal Assignee: Sudheesh Katkam Fix For: 1.0.0 Some data under the Start Time column for a long running query contain incorrect data. Below is the data for major fragment 2: Major Fragment: 02-xx-xx Minor FragmentHostStart End Total Time Max Records Max Batches Peak Memory State 02-00-xx ucs-node4.perf.lab 2.224 7:597:5757,346,551 7,012 2MB FINISHED 02-01-xx ucs-node8.perf.lab 2.210 8:068:0461,095,926 7,470 2MB FINISHED 02-02-xx ucs-node9.perf.lab 2.223 8:138:1161,329,266 7,496 2MB FINISHED 02-03-xx ucs-node3.perf.lab 2.250 8:088:0661,112,691 7,472 2MB FINISHED 02-04-xx ucs-node7.perf.lab 2.235 8:048:0161,173,642 7,478 2MB FINISHED 02-05-xx ucs-node5.perf.lab 43.518 7:528:0869,778,300 8,526 2MB FINISHED 02-06-xx ucs-node11.perf.lab 2.258 7:357:3357,324,207 7,009 2MB FINISHED 02-07-xx ucs-node2.perf.lab 2.265 7:597:5761,165,117 7,477 2MB FINISHED 02-08-xx ucs-node10.perf.lab 2.276 7:016:5954,521,464 6,667 2MB FINISHED 02-09-xx ucs-node6.perf.lab 2.250 7:437:4155,190,738 6,748 2MB FINISHED Below is the summary for major fragment 2 as reported in the UI: Major FragmentMinor Fragments Reporting First Start Last Start First End Last Endtmintavgtmaxmemmax 02-xx-xx 10 / 10 43.518 2.276 7:018:136:597:528:11 2MB The value for `First Start` should be 2.210. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2551) Varbinary literals are not supported in Drill logical expressions
[ https://issues.apache.org/jira/browse/DRILL-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2551: - Fix Version/s: (was: 1.0.0) Future Varbinary literals are not supported in Drill logical expressions - Key: DRILL-2551 URL: https://issues.apache.org/jira/browse/DRILL-2551 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Affects Versions: 0.8.0 Reporter: Jason Altekruse Assignee: Jason Altekruse Fix For: Future Drill does not support the varbinary literal type at the logical expression level. We need to define the Type in ValueExpressions, RexToDrill and add corresponding methods to ExprVisitor and its implementations. There may be other areas that are needed to make the definition complete within Drill. I will check the patches from adding other types and see what else was changed when we added them. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2257) Filter on nested repeated types on top of data with schema changes fails
[
https://issues.apache.org/jira/browse/DRILL-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2257:
-
Fix Version/s: (was: 1.0.0)
1.1.0
Filter on nested repeated types on top of data with schema changes fails
Key: DRILL-2257
URL: https://issues.apache.org/jira/browse/DRILL-2257
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Reporter: Rahul Challapalli
Assignee: Hanifi Gunes
Fix For: 1.1.0
Attachments: error.log, file1.json, file2.json, file3.json
git.commit.id.abbrev=6676f2d
The data set contains 3 files and 2 records within each file. We
added/removed fields between these files. However the below query uses fields
which are present in all the files.
{code}
select d.uid from `schema_changes` d where d.map.rm[0].rptd[0].a = 'foo';
++
|uid |
++
| 1 |
Query failed: RemoteRpcException: Failure while running fragment., index: -4,
length: 4 (expected: range(0, 16384)) [ bebebc31-0ce7-4b77-9984-f5e548790bfd
on qa-node190.qa.lab:31010 ]
[ bebebc31-0ce7-4b77-9984-f5e548790bfd on qa-node190.qa.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Running the same query on each of the individual files succeeds and returns
the data correctly.
Let me know if you need anything else
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2698) Add allocation policy for (direct memory) limits per-query-per-node
Chris Westin created DRILL-2698: --- Summary: Add allocation policy for (direct memory) limits per-query-per-node Key: DRILL-2698 URL: https://issues.apache.org/jira/browse/DRILL-2698 Project: Apache Drill Issue Type: Bug Components: Execution - Flow Affects Versions: 0.8.0 Reporter: Chris Westin Assignee: Chris Westin Fix For: 1.0.0 We want the allocator to support a policy that limits memory allocation per-query-per-node. Current policy is to divide the available direct memory by the number of fragments running on a node, and limit each fragment to that. This new policy would count fragments that belong to the same query and are running on the same node together. So, if query A has 3 fragments running on a node, and query B has 5 fragments running on the same node, and there are no other queries with fragments on that node, then there are two queries running on the node, and their allocation limit is the available direct memory divided by two. (Under the original policy, which divides the available memory by the number of fragments, A would be allowed 3/8 of the memory, and B would be allowed 5/8 of the memory.) Note that this policy should be a user-settable boot-time option; the original policy would still also be available as an option. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2401) Value 0 for monthOfYear must be in the range [1,12]
[
https://issues.apache.org/jira/browse/DRILL-2401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2401:
-
Assignee: Chun Chang (was: Daniel Barclay (Drill))
Value 0 for monthOfYear must be in the range [1,12]
---
Key: DRILL-2401
URL: https://issues.apache.org/jira/browse/DRILL-2401
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.7.0
Environment: Sandbox, SE cluster
Reporter: Sudhakar Thota
Assignee: Chun Chang
Priority: Minor
Fix For: 1.0.0
Attachments: month-00-drill-MapR-DB.rtf
0: jdbc:drill:zk=10.10.15.10:5181,10.10.15.11 select cast(row_key as
varchar(20)) as order_id, cast(t.CF1.`clerk` as varchar(20)) as order_clerk,
cast(t.CF1.orderstatus as varchar(20)) as ord_stat,cast(t.CF1.orderdate as
date) ordrdt from m7thotas.orders t where cast(t.CF1.orderdate as date)
BETWEEN cast('1996-01-10' as date) AND cast('1996-01-11' as date) and
cast(t.CF1.`clerk` as varchar(20))cast('Clerk#00334' as varchar(20));
.
.
.
| 3592807| Clerk#00358 | O | 1996-01-11 |
| 6969024| Clerk#00185 | O | 1996-01-10 |
| 6973990| Clerk#00491 | O | 1996-01-10 |
| 6975170| Clerk#00493 | O | 1996-01-10 |
| 6975462| Clerk#00352 | O | 1996-01-11 |
| 6978503| Clerk#00482 | O | 1996-01-11 |
| 6979654| Clerk#00389 | O | 1996-01-10 |
| 5344355| Clerk#00653 | O | 1996-01-11 |
| 5351969| Clerk#00237 | O | 1996-01-10 |
| 5352003| Clerk#00756 | O | 1996-01-10 |
| 3608519| Clerk#00384 | O | 1996-01-10 |
Query failed: Query failed: Failure while running fragment., Value 0 for
monthOfYear must be in the range [1,12] [
2b41018b-b35e-42a9-bd4e-0307e582b718 on se-node11.se.lab:31010 ]
[ 2b41018b-b35e-42a9-bd4e-0307e582b718 on se-node11.se.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2699) Collect all cleanup errors before reporting a failure to the client
Deneche A. Hakim created DRILL-2699: --- Summary: Collect all cleanup errors before reporting a failure to the client Key: DRILL-2699 URL: https://issues.apache.org/jira/browse/DRILL-2699 Project: Apache Drill Issue Type: Improvement Components: Execution - Flow Affects Versions: 0.8.0 Reporter: Deneche A. Hakim Assignee: Chris Westin Fix For: 0.9.0 If a query fails, the fragments and foreman should make sure to collect all failures and report them back to the client. Some known places where this isn't respected: - If a fragment fails, it will report the failure to the foreman before cleaning up. Any failure that happens in the cleanup process will be dropped by the foreman. - If multiple fragments fail, the Foreman will only report to the user the first failure it received and close immediately. All other failures will be dropped. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2700) Missing/deleted drillbit.log files on cluster are not reported
Khurram Faraaz created DRILL-2700:
-
Summary: Missing/deleted drillbit.log files on cluster are not
reported
Key: DRILL-2700
URL: https://issues.apache.org/jira/browse/DRILL-2700
Project: Apache Drill
Issue Type: Bug
Components: Tools, Build Test
Affects Versions: 0.9.0
Environment: 4 node cluster on CentOS
Reporter: Khurram Faraaz
Assignee: Steven Phillips
Missing drillbit.log files on cluster, under /opt/mapr/drill/drill-0.9.0/logs/
are not reported as missing. Instead, user is able to issue and execute SQL
queries from sqlline prompt. Failing queries are also executed and
errors/Exceptions are reported on sqlline prompt, but there is no corresponding
drillbit.log file where details must be written to in the case of
passing/failing queries.
I have manually deleted the drillbit.log (users can accidentally delete
drillbit.log), and continue to execute SQL from sqlline prompt, and in the case
of failing SQL there will be no drillbit.log file since it was accidentally
deleted, and hence stacktrace and other diagnostic information is lost.
{code}
[root@centos-01 bin]# clush -g khurram service mapr-warden stop
10.10.100.201: stopping WARDEN
10.10.100.202: stopping WARDEN
10.10.100.203: stopping WARDEN
10.10.100.204: stopping WARDEN
10.10.100.202: looking to stop mapr-core processes not started by warden
10.10.100.203: looking to stop mapr-core processes not started by warden
10.10.100.204: looking to stop mapr-core processes not started by warden
10.10.100.201: looking to stop mapr-core processes not started by warden
[root@centos-01 bin]# clush -g khurram service mapr-warden start
10.10.100.204: Starting WARDEN, logging to /opt/mapr/logs/warden.log.
10.10.100.203: Starting WARDEN, logging to /opt/mapr/logs/warden.log.
10.10.100.202: Starting WARDEN, logging to /opt/mapr/logs/warden.log.
10.10.100.201: Starting WARDEN, logging to /opt/mapr/logs/warden.log.
10.10.100.204: .
10.10.100.203: .
10.10.100.204: For diagnostics look at /opt/mapr/logs/ for
createsystemvolumes.log, warden.log and configured services log files
10.10.100.202: .
10.10.100.201: .
10.10.100.203: For diagnostics look at /opt/mapr/logs/ for
createsystemvolumes.log, warden.log and configured services log files
10.10.100.202: For diagnostics look at /opt/mapr/logs/ for
createsystemvolumes.log, warden.log and configured services log files
10.10.100.201: For diagnostics look at /opt/mapr/logs/ for
createsystemvolumes.log, warden.log and configured services log files
[root@centos-01 bin]# clush -g khurram ls -lrt
/opt/mapr/drill/drill-0.9.0/logs/drillbit.log
10.10.100.201: -rw-r--r-- 1 mapr mapr 17522 Apr 6 20:49
/opt/mapr/drill/drill-0.9.0/logs/drillbit.log
10.10.100.204: -rw-r--r-- 1 mapr mapr 17522 Apr 6 20:49
/opt/mapr/drill/drill-0.9.0/logs/drillbit.log
10.10.100.202: -rw-r--r-- 1 mapr mapr 17522 Apr 6 20:49
/opt/mapr/drill/drill-0.9.0/logs/drillbit.log
10.10.100.203: -rw-r--r-- 1 mapr mapr 17522 Apr 6 20:49
/opt/mapr/drill/drill-0.9.0/logs/drillbit.log
Manually remove drillbit.log files from cluster nodes
[root@centos-01 bin]# clush -g khurram rm -rf
/opt/mapr/drill/drill-0.9.0/logs/drillbit.log
[root@centos-01 bin]# clush -g khurram ls -lrt
/opt/mapr/drill/drill-0.9.0/logs/drillbit.log
10.10.100.201: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log:
No such file or directory
clush: 10.10.100.201: exited with exit code 2
10.10.100.204: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log:
No such file or directory
clush: 10.10.100.204: exited with exit code 2
10.10.100.203: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log:
No such file or directory
clush: 10.10.100.203: exited with exit code 2
10.10.100.202: ls: cannot access /opt/mapr/drill/drill-0.9.0/logs/drillbit.log:
No such file or directory
clush: 10.10.100.202: exited with exit code 2
[root@centos-01 bin]# ./sqlline -u jdbc:drill:
Drill log directory: /opt/mapr/drill/drill-0.9.0/logs
sqlline version 1.1.6
0: jdbc:drill: use sys;
+++
| ok | summary |
+++
| true | Default schema changed to 'sys' |
+++
1 row selected (1.199 seconds)
0: jdbc:drill: select * from sys.version;
+++-+-++
| commit_id | commit_message | commit_time | build_email | build_time |
+++-+-++
| 9d92b8e319f2d46e8659d903d355450e15946533 | DRILL-2580: Exit early from
HashJoinBatch if build side is empty | 26.03.2015 @ 16:13:53 EDT | Unknown
| 26.03.2015 @ 16:59:31 EDT |
+++-+-++
1 row selected (0.608 seconds)
0: jdbc:drill: select * from sys.drillbits;
[jira] [Updated] (DRILL-2039) use forks-per-CPU units in forkCount
[
https://issues.apache.org/jira/browse/DRILL-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hanifi Gunes updated DRILL-2039:
Assignee: Parth Chandra (was: Hanifi Gunes)
use forks-per-CPU units in forkCount
Key: DRILL-2039
URL: https://issues.apache.org/jira/browse/DRILL-2039
Project: Apache Drill
Issue Type: Improvement
Components: Tools, Build Test
Reporter: Daniel Barclay (Drill)
Assignee: Parth Chandra
Priority: Minor
Fix For: 0.9.0
Attachments: DRILL-2039.1.patch.txt
Can Surefire's fork count be set using units of forks per CPU instead of the
absolute number of forks (i.e., {{forkCount1C/forkCount}} instead of
{{forkCount4/forkcount}})?
That would let developers with more powerful machines still run tests in
parallel while letting others avoid or reduce test timeout errors without the
risk of forgetting to re-edit pom.xml after every switch to a different git
branch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2039) use forks-per-CPU units in forkCount
[
https://issues.apache.org/jira/browse/DRILL-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hanifi Gunes updated DRILL-2039:
Attachment: DRILL-2039.1.patch.txt
Use 1 fork per logical core
use forks-per-CPU units in forkCount
Key: DRILL-2039
URL: https://issues.apache.org/jira/browse/DRILL-2039
Project: Apache Drill
Issue Type: Improvement
Components: Tools, Build Test
Reporter: Daniel Barclay (Drill)
Assignee: Hanifi Gunes
Priority: Minor
Fix For: 0.9.0
Attachments: DRILL-2039.1.patch.txt
Can Surefire's fork count be set using units of forks per CPU instead of the
absolute number of forks (i.e., {{forkCount1C/forkCount}} instead of
{{forkCount4/forkcount}})?
That would let developers with more powerful machines still run tests in
parallel while letting others avoid or reduce test timeout errors without the
risk of forgetting to re-edit pom.xml after every switch to a different git
branch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-2060) Support constant folding in expressions in Filter
[
https://issues.apache.org/jira/browse/DRILL-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Altekruse resolved DRILL-2060.
Resolution: Fixed
Fixed in 8d70709ddaddc7111517f4bed883fc9908ef94de
Support constant folding in expressions in Filter
-
Key: DRILL-2060
URL: https://issues.apache.org/jira/browse/DRILL-2060
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow, Query Planning Optimization
Reporter: Aditya Kishore
Assignee: Jason Altekruse
Fix For: 0.9.0
Attachments: DRILL-2060-15-mar-15.patch,
DRILL-2060-const-folding-feb-6.diff,
DRILL-2060-part1-const-folding-feb-27.patch,
DRILL-2060-part2-const-folding-feb-27.patch, DRILL-2060.patch
If one side of a filter condition is a constant expression, we should be able
to evaluate the expression in the planning phase and replace the expression
with the constant value.
For example, the following filter
{{Where shipdate date '1997-01-01' + interval '1' year}}
can be re-written as
{{Where shipdate date '1998-01-01'}}
Or
{{Where age + 5 42}}
can be re-written as
{{Where age 37}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2383) add exception and pause injections for testing drillbit stability
[
https://issues.apache.org/jira/browse/DRILL-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sudheesh Katkam updated DRILL-2383:
---
Attachment: DRILL-2383.3.patch.txt
Addressed review comments and rebased on current master.
add exception and pause injections for testing drillbit stability
-
Key: DRILL-2383
URL: https://issues.apache.org/jira/browse/DRILL-2383
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow
Reporter: Chris Westin
Assignee: Sudheesh Katkam
Fix For: 0.9.0
Attachments: DRILL-2383.1.patch.txt, DRILL-2383.3.patch.txt
Use the exception injection mechanism to add exception injections to test a
variety of distributed failure scenarios.
Here are some scenarios we've worked out before:
1. Cancellation:
TC1: cancel before any result set is returned
TC2: cancel in the middle of fetching result set
TC3: cancel after all result set are produced but not all are fetched
TC4: cancel after everything is completed and fetched
As test setup, we need:
- query dataset large enough to be sent to different drillbits, e.g., TPCH
100
- queries that force multiple drillbits to work on them; e.g., count ...
group by
2. Completed (in each case check all drillbits are still up and running):
TC1: success
TC2: failed query - before query is executed - while sql parsing
TC3: failed query - before query is executed - while sending fragments to
other drillbits for execution
TC4: failed query - during query execution
It is currently not possible to create a scenario in which a query may hang.
To check all drillbits up and running and in a clean state, run:
-select count(*) from sys.drillbits;-
{code}
select count(*) from sys.memory;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (DRILL-2060) Support constant folding in expressions in Filter
[
https://issues.apache.org/jira/browse/DRILL-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Altekruse reopened DRILL-2060:
Assignee: Jason Altekruse (was: Jacques Nadeau)
Messed up the workflow, should have resolved instead of closed
Support constant folding in expressions in Filter
-
Key: DRILL-2060
URL: https://issues.apache.org/jira/browse/DRILL-2060
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow, Query Planning Optimization
Reporter: Aditya Kishore
Assignee: Jason Altekruse
Fix For: 0.9.0
Attachments: DRILL-2060-15-mar-15.patch,
DRILL-2060-const-folding-feb-6.diff,
DRILL-2060-part1-const-folding-feb-27.patch,
DRILL-2060-part2-const-folding-feb-27.patch, DRILL-2060.patch
If one side of a filter condition is a constant expression, we should be able
to evaluate the expression in the planning phase and replace the expression
with the constant value.
For example, the following filter
{{Where shipdate date '1997-01-01' + interval '1' year}}
can be re-written as
{{Where shipdate date '1998-01-01'}}
Or
{{Where age + 5 42}}
can be re-written as
{{Where age 37}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2704) Schema change not reported in Union All query
Khurram Faraaz created DRILL-2704:
-
Summary: Schema change not reported in Union All query
Key: DRILL-2704
URL: https://issues.apache.org/jira/browse/DRILL-2704
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Affects Versions: 0.9.0
Environment: 4 node cluster on CentOS
Reporter: Khurram Faraaz
Assignee: Jinfeng Ni
Schema change is not reported in Union All query. Inputs to Union All read from
CSV file.
{code}
0: jdbc:drill: select columns[0] from `f1.csv` where columns[0] = 13 union all
select columns[0] from `f2.csv` where columns[0] = 10;
Query failed: RemoteRpcException: Failure while running fragment., 9.99 [
a80303f5-781c-4288-875e-90ad60c8caac on centos-02.qa.lab:31010 ]
[ a80303f5-781c-4288-875e-90ad60c8caac on centos-02.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
0: jdbc:drill: select * from `f1.csv`;
++
| columns |
++
| [1] |
| [2] |
| [55] |
| [77] |
| [-100] |
| [13] |
| [17] |
| [19] |
| [2] |
| [1000] |
| [99] |
| [9.99] |
| [12345.987] |
| [Hello World] |
| [this is a test string] |
| [foobar] |
| [] |
| [new string] |
| [34534534] |
| [345345] |
| [2321312] |
| [67867867] |
| [112319089] |
| [-14695967] |
| [100]|
| [99] |
| [7] |
| [5] |
| [13] |
| [] |
| [] |
| [] |
| [0] |
| [1] |
| [-1] |
| null |
| null |
| null |
| null |
++
39 rows selected (0.16 seconds)
0: jdbc:drill: select * from `f2.csv`;
++
| columns |
++
| [1] |
| [0] |
| [-1] |
| [99] |
| [1000] |
| [10] |
| [-13]|
| [-17]|
| [19] |
| [17] |
| [13] |
| [] |
| [no way] |
| [good quality] |
| [] |
| [done!] |
| [Yay!!!] |
| null |
++
18 rows selected (0.128 seconds)
Casting columns[0] to varchar gives correct results
0: jdbc:drill: select cast(columns[0] as varchar(100)) from `f1.csv` where
columns[0] = '13' union all select cast(columns[0] as varchar(100)) from
`f2.csv` where columns[0] = '10';
++
| EXPR$0 |
++
| 13 |
| 13 |
| 10 |
++
3 rows selected (0.474 seconds)
Stack trace from drillbit.log for failing query
2015-04-06 23:31:43,674 [2adce8a0-24c8-58ad-2f46-30c42a33ef2a:frag:0:0] ERROR
o.a.d.e.w.f.AbstractStatusReporter - Error
a4ec50e8-88ea-4474-9a36-b2250f34e340: Failure while running fragment.
java.lang.NumberFormatException: 9.99
at
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.nfeI(StringFunctionHelpers.java:96)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.expr.fn.impl.StringFunctionHelpers.varCharToInt(StringFunctionHelpers.java:121)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.test.generated.FiltererGen37.doEval(FilterTemplate2.java:42)
~[na:na]
at
org.apache.drill.exec.test.generated.FiltererGen37.filterBatchNoSV(FilterTemplate2.java:98)
~[na:na]
at
org.apache.drill.exec.test.generated.FiltererGen37.filterBatch(FilterTemplate2.java:71)
~[na:na]
at
org.apache.drill.exec.physical.impl.filter.FilterRecordBatch.doWork(FilterRecordBatch.java:82)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:93)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.svremover.RemovingRecordBatch.innerNext(RemovingRecordBatch.java:96)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
[jira] [Closed] (DRILL-2681) add sql functions docs (string, convert_to/from)
[ https://issues.apache.org/jira/browse/DRILL-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn closed DRILL-2681. add sql functions docs (string, convert_to/from) Key: DRILL-2681 URL: https://issues.apache.org/jira/browse/DRILL-2681 Project: Apache Drill Issue Type: Task Components: Documentation Reporter: Kristine Hahn Assignee: Kristine Hahn Includes SQL scalar functions, maprdb plugin doc fixes, time zone limitations section, Query HBase doc fixes, fix broken links in contribution guidelines and contribution ideas, prelim. overview of config options -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1686) Formalize and document sqlline logging configuration
[ https://issues.apache.org/jira/browse/DRILL-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1686: - Fix Version/s: (was: 0.9.0) 1.1.0 Formalize and document sqlline logging configuration Key: DRILL-1686 URL: https://issues.apache.org/jira/browse/DRILL-1686 Project: Apache Drill Issue Type: Bug Components: Client - CLI Affects Versions: 0.7.0 Reporter: Jacques Nadeau Assignee: Daniel Barclay (Drill) Priority: Minor Fix For: 1.1.0 Drill bit install is creating zero size sqlline.log and does not log any failures from sqlline cli. All error/info/debug messages are logged to drillbit.log Need a solution to log sqlline errors to sqlline.log -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1686) Formalize and document sqlline logging configuration
[ https://issues.apache.org/jira/browse/DRILL-1686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1686: - Assignee: Daniel Barclay (Drill) (was: Mehant Baid) Formalize and document sqlline logging configuration Key: DRILL-1686 URL: https://issues.apache.org/jira/browse/DRILL-1686 Project: Apache Drill Issue Type: Bug Components: Client - CLI Affects Versions: 0.7.0 Reporter: Jacques Nadeau Assignee: Daniel Barclay (Drill) Priority: Minor Fix For: 1.1.0 Drill bit install is creating zero size sqlline.log and does not log any failures from sqlline cli. All error/info/debug messages are logged to drillbit.log Need a solution to log sqlline errors to sqlline.log -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (DRILL-2508) Add new column to sys.options table that exposes whether or not the current system value is the default
[ https://issues.apache.org/jira/browse/DRILL-2508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394684#comment-14394684 ] Sudheesh Katkam edited comment on DRILL-2508 at 4/6/15 9:43 PM: Updated patch, use status instead of boolean. Statuses can be: BOOT, DEFAULT, CHANGED was (Author: sudheeshkatkam): Updated patch with status instead of boolean. Statuses can be: BOOT, DEFAULT, CHANGED Add new column to sys.options table that exposes whether or not the current system value is the default --- Key: DRILL-2508 URL: https://issues.apache.org/jira/browse/DRILL-2508 Project: Apache Drill Issue Type: Improvement Components: Storage - Other Reporter: Victoria Markman Assignee: Sudheesh Katkam Fix For: 0.9.0 Attachments: DRILL-2508.1.patch.txt, DRILL-2508.2.patch.txt Need to be able to see system parameters that I changed. There is an enhancement already to reset them to default values: drill-1065 I don't necessarily want to do that, I just want to see only things that I changed : default value vs. my change. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2702) Support more advanced constant expression folding
Jason Altekruse created DRILL-2702: -- Summary: Support more advanced constant expression folding Key: DRILL-2702 URL: https://issues.apache.org/jira/browse/DRILL-2702 Project: Apache Drill Issue Type: Improvement Components: Query Planning Optimization Reporter: Jason Altekruse Assignee: Jinfeng Ni A variety of constant expressions are now folded at planning time with the work completed for DRILL-2060. One of the examples listed in the issue is not currently supported. Supporting an expression re-write like this will require more work, this issue is here to track that enhancement request. These types of rewrites are particularly useful for Drill because the speed of Drill is often limited by the types of filter pushdowns supported by the underlying storage system. For system that support range scans, we can rewrite inequalities to range scans in simple cases. Currently we cannot re-write more complicated inequalities like this one below, this query would prompt a full table scan with a filter, but the transformation shown can turn it into a range scan. Where age + 5 42 can be re-written as Where age 37 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2681) add sql functions docs (string, convert_to/from)
[ https://issues.apache.org/jira/browse/DRILL-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn resolved DRILL-2681. -- Resolution: Fixed add sql functions docs (string, convert_to/from) Key: DRILL-2681 URL: https://issues.apache.org/jira/browse/DRILL-2681 Project: Apache Drill Issue Type: Task Components: Documentation Reporter: Kristine Hahn Assignee: Kristine Hahn Includes SQL scalar functions, maprdb plugin doc fixes, time zone limitations section, Query HBase doc fixes, fix broken links in contribution guidelines and contribution ideas, prelim. overview of config options -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-1840) Cloning the source repository as documented in the wiki
[
https://issues.apache.org/jira/browse/DRILL-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kristine Hahn resolved DRILL-1840.
--
Resolution: Fixed
Sorry for the delay. Thanks for letting us know about the broken links.
We fixed the links in the docs at the new site http://drill.apache.org/docs/.
We're moving away from the wiki docs.
The corrected links are in these sections:
* http://drill.apache.org/docs/apache-drill-contribution-guidelines/
* http://drill.apache.org/docs/apache-drill-contribution-ideas/
Thanks!
Cloning the source repository as documented in the wiki
---
Key: DRILL-1840
URL: https://issues.apache.org/jira/browse/DRILL-1840
Project: Apache Drill
Issue Type: Improvement
Components: Documentation
Environment: wiki
Reporter: Karl Heinz Marbaise
Assignee: Kristine Hahn
Fix For: Future
On the following page
https://cwiki.apache.org/confluence/display/DRILL/Compiling+Drill+from+Source
it is mentioned to clone the following repository:
{{git clone https://git-wip-us.apache.org/repos/asf/incubator-drill.git}}
which is the meantime not possible cause the repository name has changed
based on the migration from incubator to tlp...
{{git clone https://git-wip-us.apache.org/repos/asf/drill.git}}
Unfortunately i haven't the karma to change the wiki nor to login into the
wicki so i couldn' fix this issue..
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-1687) Remove references to deprecated dense Decimal representation
[ https://issues.apache.org/jira/browse/DRILL-1687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1687: - Fix Version/s: (was: 0.9.0) 1.1.0 Remove references to deprecated dense Decimal representation Key: DRILL-1687 URL: https://issues.apache.org/jira/browse/DRILL-1687 Project: Apache Drill Issue Type: Improvement Components: Execution - Data Types Reporter: Jason Altekruse Assignee: Mehant Baid Priority: Minor Fix For: 1.1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-2060) Support constant folding in expressions in Filter
[
https://issues.apache.org/jira/browse/DRILL-2060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Altekruse closed DRILL-2060.
--
Resolution: Fixed
Fixed in 8d70709ddaddc7111517f4bed883fc9908ef94de
Support constant folding in expressions in Filter
-
Key: DRILL-2060
URL: https://issues.apache.org/jira/browse/DRILL-2060
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow, Query Planning Optimization
Reporter: Aditya Kishore
Assignee: Jacques Nadeau
Fix For: 0.9.0
Attachments: DRILL-2060-15-mar-15.patch,
DRILL-2060-const-folding-feb-6.diff,
DRILL-2060-part1-const-folding-feb-27.patch,
DRILL-2060-part2-const-folding-feb-27.patch, DRILL-2060.patch
If one side of a filter condition is a constant expression, we should be able
to evaluate the expression in the planning phase and replace the expression
with the constant value.
For example, the following filter
{{Where shipdate date '1997-01-01' + interval '1' year}}
can be re-written as
{{Where shipdate date '1998-01-01'}}
Or
{{Where age + 5 42}}
can be re-written as
{{Where age 37}}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-2448) Remove outdated code to ignore type resolution with varchar vs varbinary now that implicit casting subsumes it
[ https://issues.apache.org/jira/browse/DRILL-2448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse resolved DRILL-2448. Resolution: Fixed Fix Version/s: (was: 1.0.0) 0.9.0 Fixed in a49b99ff1a679a0dfb696686d62449edbb231f37 Remove outdated code to ignore type resolution with varchar vs varbinary now that implicit casting subsumes it -- Key: DRILL-2448 URL: https://issues.apache.org/jira/browse/DRILL-2448 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Affects Versions: 0.7.0 Reporter: Jason Altekruse Assignee: Jason Altekruse Priority: Critical Fix For: 0.9.0 Attachments: DRILL-2448-varchar-varbinary-implicit-cast.patch Function resolution included a small condition to allow varchar and varbinary functions to be resolved for either incoming type. While it is valid to implicitly cast between these two, this early workaround creates a technically invalid expression tree that happens to work with the current code generation system. This however does create an issue for the interpreted expression evaluator. Removing the code simply causes an implicit cast to be added during materialization, this works for both generated code expression evaluation as well as the interpreter. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2539) NullReader should allocate an empty vector in copy* methods
[ https://issues.apache.org/jira/browse/DRILL-2539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2539: - Component/s: Execution - Data Types NullReader should allocate an empty vector in copy* methods --- Key: DRILL-2539 URL: https://issues.apache.org/jira/browse/DRILL-2539 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types, Execution - Flow Reporter: Hanifi Gunes Assignee: Hanifi Gunes Fix For: 0.9.0 Projecting an non-existing field from a repeated type fails with an NPE mainly because projected vector is not allocated and underlying buffer is dead. This issue proposes to allocate an empty vector in NullReader's copy* methods. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2611) Fixed length value vectors report invalid value count
[ https://issues.apache.org/jira/browse/DRILL-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2611: - Component/s: Execution - Data Types Fixed length value vectors report invalid value count - Key: DRILL-2611 URL: https://issues.apache.org/jira/browse/DRILL-2611 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Reporter: Hanifi Gunes Assignee: Hanifi Gunes Fix For: 0.9.0 We maintain an exclusive value count variable in fixed vectors however we don't update it upon calling set/Safe. Accessor reports the value count from the variable ignoring values that are already in the buffer or written via set/Safe. This causes execution failures manifested as IOOB when underlying data is sparse. We should either remove the variable and report value count directly investigating the buffer if not computationally expensive or update the variable each time we make a write to the vector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2164) Composite vectors should rely on VectorContainer for maintaining child vectors
[ https://issues.apache.org/jira/browse/DRILL-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2164: - Fix Version/s: (was: 0.9.0) 1.0.0 Composite vectors should rely on VectorContainer for maintaining child vectors -- Key: DRILL-2164 URL: https://issues.apache.org/jira/browse/DRILL-2164 Project: Apache Drill Issue Type: Sub-task Components: Execution - Data Types Reporter: Hanifi Gunes Assignee: Hanifi Gunes Fix For: 1.0.0 As of DRILL-1885, the logic that handles maintaining vectors is mostly scattered and repeated across AbstractContainerVector and VectorContainer. We should come up with an abstraction unifying vector container logic for better code re-use. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2141) Data type error in group by and order by for JSON
[ https://issues.apache.org/jira/browse/DRILL-2141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2141: - Fix Version/s: (was: 0.9.0) 1.0.0 Data type error in group by and order by for JSON - Key: DRILL-2141 URL: https://issues.apache.org/jira/browse/DRILL-2141 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Affects Versions: 0.7.0 Reporter: Andries Engelbrecht Assignee: Hanifi Gunes Fix For: 1.0.0 Attachments: FlumeData.1422748800086, drillbit.log, new_drillbit.log When doing group by and oder by on complex nested JSON getting Data type errors. Query: select t.retweeted_status.`user`.name as name, count(t.retweeted_status.id) as rt_count from `./nfl` t where t.retweeted_status.`user`.name is not null group by t.retweeted_status.`user`.name order by count(t.retweeted_status.id) desc limit 10; Screen output: Query failed: Query failed: Failure while running fragment., Failure while reading vector. Expected vector class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.NullableVarCharVector. [ c6ea670f-5fa0-491c-acfb-5ccd128ec324 on drilldemo:31010 ] [ c6ea670f-5fa0-491c-acfb-5ccd128ec324 on drilldemo:31010 ] java.lang.RuntimeException: java.sql.SQLException: Failure while executing query. at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514) at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148) at sqlline.SqlLine.print(SqlLine.java:1809) at sqlline.SqlLine$Commands.execute(SqlLine.java:3766) at sqlline.SqlLine$Commands.sql(SqlLine.java:3663) at sqlline.SqlLine.dispatch(SqlLine.java:889) at sqlline.SqlLine.begin(SqlLine.java:763) at sqlline.SqlLine.start(SqlLine.java:498) at sqlline.SqlLine.main(SqlLine.java:460) Drill log attached -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2504) Aggregate query with grouping results in Error
[
https://issues.apache.org/jira/browse/DRILL-2504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2504:
-
Fix Version/s: (was: 0.9.0)
1.0.0
Aggregate query with grouping results in Error
--
Key: DRILL-2504
URL: https://issues.apache.org/jira/browse/DRILL-2504
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.8.0
Environment: 4 node cluster
{code}
0: jdbc:drill: select * from sys.version;
+++-+-++
| commit_id | commit_message | commit_time | build_email | build_time |
+++-+-++
| f658a3c513ddf7f2d1b0ad7aa1f3f65049a594fe | DRILL-2209 Insert
ProjectOperator with MuxExchange | 09.03.2015 @ 01:49:18 EDT | Unknown |
09.03.2015 @ 04:52:49 EDT |
+++-+-++
1 row selected (0.062 seconds)
{code}
Reporter: Khurram Faraaz
Assignee: Mehant Baid
Fix For: 1.0.0
The below aggregate query with group by over distinct/non-distinct data
results in an Exception. Please note that I had set enable_hashagg=false and
I was querying from a CSV file. Query was run over a four node cluster.
alter system set `planner.enable_hashagg`=true;
alter session set `planner.enable_hashagg`=true;
{code}
0: jdbc:drill: alter system set `planner.enable_hashagg`=false;
+++
| ok | summary |
+++
| true | planner.enable_hashagg updated. |
+++
1 row selected (0.075 seconds)
0: jdbc:drill: select columns[4], sum(columns[0]), count(distinct
columns[1]), max(columns[2]), count(distinct columns[3]), max(columns[5]),
min(columns[6]), avg(columns[7])
. . . . . . . from `conftest.csv`
. . . . . . . group by columns[4];
Query failed: Query stopped., Failure while trying to materialize incoming
schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[castINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--.. [
6cd09ba7-3e4b-4b3b-b111-39f74f53e1b0 on centos-01.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
{code}
Stack trace from drillbit.log
2015-03-19 17:47:43,123 [2af4f441-8c04-99f9-1a12-a55a7c72ece7:frag:0:0] ERROR
o.a.d.e.w.f.AbstractStatusReporter - Error
bab1babd-48fe-4719-8a77-dc5826027ba7: Failure while running fragment.
org.apache.drill.exec.exception.SchemaChangeException: Failure while trying
to materialize incoming schema. Errors:
Error in expression at index -1. Error: Missing function implementation:
[castINT(BIT-OPTIONAL)]. Full expression: --UNKNOWN EXPRESSION--..
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:390)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:78)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
[jira] [Updated] (DRILL-2598) Order by with limit on complex type throw IllegalStateException
[
https://issues.apache.org/jira/browse/DRILL-2598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2598:
-
Fix Version/s: (was: 0.9.0)
1.0.0
Order by with limit on complex type throw IllegalStateException
---
Key: DRILL-2598
URL: https://issues.apache.org/jira/browse/DRILL-2598
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Affects Versions: 0.8.0
Reporter: Chun Chang
Assignee: Hanifi Gunes
Priority: Blocker
Fix For: 1.0.0
drill 0.8 release candidate:
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select * from sys.version;
+++-+-++
| commit_id | commit_message | commit_time | build_email | build_time |
+++-+-++
| 462e50ce9c4b829c2a4bafdeb9763bfba677c726 | DRILL-2575:
FragmentExecutor.cancel() blasts through state transitions regardless of
current state | 25.03.2015 @ 21:11:23 PDT |
{code}
The following query involving limit and group by caused the
IllegalStateException:
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select t.id, t.soa from
`complex.json` t order by t.id limit 10;
+++
| id |soa |
+++
| 1 |
[{in:1},{in:1,fl:1.12345},{in:1,fl:10.12345},{in:1,fl:10.6789,bool:true,str:here
is a string at row 1}] |
| 2 |
[{in:2},{in:2,fl:2.12345},{in:2,fl:20.12345},{in:2,fl:20.6789,bool:true,str:here
is a string at row 2,nul:not null}] |
| 3 |
[{in:3},{in:3,fl:3.12345},{in:3,fl:30.12345,nul:not
null},{in:3,fl:30.6789,bool:true,str:here is a string at row 3}] |
| 4 |
[{in:4},{in:4,fl:4.12345},{in:4,fl:40.12345,nul:not
null},{in:4,fl:40.6789,bool:false,str:here is a string at row
4,nul:not null}] |
| 5 |
[{in:5},{in:5,fl:5.12345},{in:5,fl:50.12345,nul:not
null},{in:5,fl:50.6789,bool:false,str:here is a string at row 5}] |
| 6 |
[{in:6},{in:6,fl:6.12345},{in:6,fl:60.12345,nul:not
null},{in:6,fl:60.6789,bool:false,str:here is a string at row 6}] |
| 7 |
[{in:7},{in:7,fl:7.12345},{in:7,fl:70.12345,nul:not
null},{in:7,fl:70.6789,bool:false,str:here is a string at row
7,nul:not null}] |
| 8 |
[{in:8},{in:8,fl:8.12345},{in:8,fl:80.12345,nul:not
null},{in:8,fl:80.6789,bool:true,str:here is a string at row
8,nul:not null}] |
| 9 |
[{in:9},{in:9,fl:9.12345},{in:9,fl:90.12345,nul:not
null},{in:9,fl:90.6789,bool:true,str:here is a string at row 9}] |
| 10 |
[{in:10},{in:10,fl:10.12345},{in:10,fl:100.12345,nul:not
null},{in:10,fl:100.6789,bool:false,str:here is a string at row
10,nul:not null}] |
Query failed: RemoteRpcException: Failure while running fragment., Attempted
to close accountor with 25 buffer(s) still allocatedfor QueryId:
2aeb3baf-acc1-5615-4537-f215a47d4893, MajorFragmentId: 0, MinorFragmentId: 0.
Total 25 allocation(s) of byte size(s): 512, 512, 512, 512, 512, 512,
512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, 512,
512, 512, 512, 512, at stack location:
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:231)
org.apache.drill.exec.vector.BitVector.allocateNewSafe(BitVector.java:95)
org.apache.drill.exec.vector.BitVector.allocateNew(BitVector.java:78)
org.apache.drill.exec.vector.NullableBitVector.allocateNew(NullableBitVector.java:168)
org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapTransferPair.init(RepeatedMapVector.java:282)
org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapTransferPair.init(RepeatedMapVector.java:260)
org.apache.drill.exec.vector.complex.RepeatedMapVector.getTransferPair(RepeatedMapVector.java:126)
org.apache.drill.exec.physical.impl.sort.RecordBatchData.init(RecordBatchData.java:57)
org.apache.drill.exec.physical.impl.TopN.TopNBatch.innerNext(TopNBatch.java:222)
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
[jira] [Updated] (DRILL-1951) Can't cast numeric value with decimal point read from CSV file into integer data type
[
https://issues.apache.org/jira/browse/DRILL-1951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-1951:
-
Fix Version/s: (was: 0.9.0)
1.0.0
Can't cast numeric value with decimal point read from CSV file into integer
data type
-
Key: DRILL-1951
URL: https://issues.apache.org/jira/browse/DRILL-1951
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Affects Versions: 0.8.0
Reporter: Victoria Markman
Assignee: Mehant Baid
Fix For: 1.0.0
sales.csv file:
{code}
997,Ford,ME350,3000.00, comment#1
1999,Chevy,Venture,4900.00, comment#2
1999,Chevy,Venture,5000.00, comment#3
1996,Jeep,Cherokee,1.01, comment#4
0: jdbc:drill:schema=dfs select cast(columns[3] as decimal(18,2)) from
`sales.csv`;
++
| EXPR$0 |
++
| 3000.00|
| 4900.00|
| 5000.00|
| 1.01 |
++
4 rows selected (0.093 seconds)
{code}
-- Can cast to decimal
{code}
0: jdbc:drill:schema=dfs select cast(columns[3] as decimal(18,2)) from
`sales.csv`;
++
| EXPR$0 |
++
| 3000.00|
| 4900.00|
| 5000.00|
| 1.01 |
++
4 rows selected (0.095 seconds)
{code}
-- Can cast to float
{code}
0: jdbc:drill:schema=dfs select cast(columns[3] as float) from `sales.csv`;
++
| EXPR$0 |
++
| 3000.0 |
| 4900.0 |
| 5000.0 |
| 1.01 |
++
4 rows selected (0.112 seconds)
{code}-- Can't cast to INT/BIGINT
{code}
0: jdbc:drill:schema=dfs select cast(columns[3] as bigint) from `sales.csv`;
Query failed: Query failed: Failure while running fragment., 3000.00 [
4818451a-c731-48a9-9992-1e81ab1d520d on atsqa4-134.qa.lab:31010 ]
[ 4818451a-c731-48a9-9992-1e81ab1d520d on atsqa4-134.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
-- Same works with json/parquet files
{code}
0: jdbc:drill:schema=dfs select a1 from `t1.json`;
++
| a1 |
++
| 10.01 |
++
1 row selected (0.077 seconds)
0: jdbc:drill:schema=dfs select cast(a1 as int) from `t1.json`;
++
| EXPR$0 |
++
| 10 |
++
0: jdbc:drill:schema=dfs select * from test_cast;
++
| a1 |
++
| 10.0100|
++
1 row selected (0.06 seconds)
0: jdbc:drill:schema=dfs select cast(a1 as int) from test_cast;
++
| EXPR$0 |
++
| 10 |
++
1 row selected (0.094 seconds)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2150) Create an abstraction for repeated value vectors.
[ https://issues.apache.org/jira/browse/DRILL-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2150: - Fix Version/s: (was: 0.9.0) 1.0.0 Create an abstraction for repeated value vectors. - Key: DRILL-2150 URL: https://issues.apache.org/jira/browse/DRILL-2150 Project: Apache Drill Issue Type: Sub-task Components: Execution - Data Types Reporter: Hanifi Gunes Assignee: Hanifi Gunes Fix For: 1.0.0 This sub-task is concerned about creating an abstraction for repeated value vectors. The existing abstraction seems invalid. The purpose is to provide a minimal interface that enables code re-usability. The proposal is to preserve existing functionalities such as exposing group count, group size as well as providing low level access to underlying offsets and data vectors. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2669) Error happening without limit clause and works with limit clause
[
https://issues.apache.org/jira/browse/DRILL-2669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2669:
-
Fix Version/s: (was: 0.9.0)
1.0.0
Error happening without limit clause and works with limit clause
Key: DRILL-2669
URL: https://issues.apache.org/jira/browse/DRILL-2669
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 0.8.0
Environment: mapr sandbox 4.0.2
Reporter: Sudhakar Thota
Assignee: Mehant Baid
Fix For: 1.0.0
Perhaps this could be a bug. I get the same results.
But the plan is very different, the UnionExchange is set up immediately after
the scan operation in successful case( Case 1 ), where as UnionExchange is
happening after scanproject (Case -2).
Case -1.Successful case.
{code}
0: jdbc:drill: explain plan for select to_timestamp(t.t,
'-MM-dd''T''HH:mm:ss.SSS''Z''') FROM (select * from
dfs.sthota_prq.`/tstamp_test/*.parquet` limit 13015351) t;
--+
text json
--+
00-00 Screen
00-01 Project(EXPR$0=[TO_TIMESTAMP(ITEM($0, 't'),
'-MM-dd''T''HH:mm:ss.SSS''Z''')])
00-02 SelectionVectorRemover
00-03 Limit(fetch=[13015351])
00-04 UnionExchange
01-01 Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
[path=maprfs:/mapr/demo.mapr.com/user/sthota/parquet/tstamp_test/1_2_0.parquet],
ReadEntryWithPath
[path=maprfs:/mapr/demo.mapr.com/user/sthota/parquet/tstamp_test/1_1_0.parquet],
ReadEntryWithPath
[path=maprfs:/mapr/demo.mapr.com/user/sthota/parquet/tstamp_test/1_0_0.parquet]],
selectionRoot=/mapr/demo.mapr.com/user/sthota/parquet/tstamp_test,
numFiles=3, columns=[`*`]]])
{
head :
Unknown macro: { version }
,
{code}
Case -2. Unsuccessful case:
{code}
0: jdbc:drill: explain plan for select to_timestamp(t.t,
'-MM-dd''T''HH:mm:ss.SSS''Z''') FROM (select * from
dfs.sthota_prq.`/tstamp_test/*.parquet` ) t;
--+
text json
--+
00-00 Screen
00-01 UnionExchange
01-01 Project(EXPR$0=[TO_TIMESTAMP(ITEM($0, 't'),
'-MM-dd''T''HH:mm:ss.SSS''Z''')])
01-02 Scan(groupscan=[ParquetGroupScan [entries=[ReadEntryWithPath
[path=maprfs:/mapr/demo.mapr.com/user/sthota/parquet/tstamp_test/1_2_0.parquet],
ReadEntryWithPath
[path=maprfs:/mapr/demo.mapr.com/user/sthota/parquet/tstamp_test/1_1_0.parquet],
ReadEntryWithPath
[path=maprfs:/mapr/demo.mapr.com/user/sthota/parquet/tstamp_test/1_0_0.parquet]],
selectionRoot=/mapr/demo.mapr.com/user/sthota/parquet/tstamp_test,
numFiles=3, columns=[`*`]]])
{
head :
Unknown macro: { version }
,
{code}
{code}
0: jdbc:drill: select to_timestamp(t.t, '-MM-dd''T''HH:mm:ss.SSS''Z''')
FROM (select * from dfs.sthota_prq.`/tstamp_test/*.parquet` limit 10) t;
EXPR$0
2015-01-27 13:43:53.0
2015-01-27 13:43:49.0
2015-01-27 13:43:47.0
2015-01-27 13:43:47.0
2015-01-27 13:43:47.0
2015-01-27 13:43:45.0
2015-01-27 13:43:43.0
2015-01-27 13:43:43.0
2015-01-27 13:43:43.0
2015-01-27 13:43:39.0
10 rows selected (1.127 seconds)
{code}
{code}
0: jdbc:drill: select to_timestamp(t.t, '-MM-dd''T''HH:mm:ss.SSS''Z''')
FROM (select * from dfs.sthota_prq.`/tstamp_test/*.parquet`) t;
{code}
{code}
0: jdbc:drill: select to_timestamp(t.t, '-MM-dd''T''HH:mm:ss.SSS''Z''')
FROM (select * from dfs.sthota_prq.`/tstamp_test/*.parquet`) t;
Query failed: RemoteRpcException: Failure while trying to start remote
fragment, Expression has syntax error! line 1:30:mismatched input 'T'
expecting CParen [ ab817e5a-9b74-47dd-b3c6-3bbf025c7de9 on maprdemo:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2685) Unique-ify local Hive metastore directory or unit test fails
[ https://issues.apache.org/jira/browse/DRILL-2685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2685: - Fix Version/s: (was: 0.9.0) 1.0.0 Unique-ify local Hive metastore directory or unit test fails Key: DRILL-2685 URL: https://issues.apache.org/jira/browse/DRILL-2685 Project: Apache Drill Issue Type: Bug Reporter: Hanifi Gunes Assignee: Hanifi Gunes Priority: Blocker Fix For: 1.0.0 Hive test suites subclasses HiveTestBase that in turn generates data. When tests are run in a concurrent setting, one removes the common directory while the other is working on it, failing unit-tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2125) Add input template file in the source files generated by freemarker
[ https://issues.apache.org/jira/browse/DRILL-2125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2125: - Fix Version/s: (was: 0.9.0) 1.0.0 Add input template file in the source files generated by freemarker --- Key: DRILL-2125 URL: https://issues.apache.org/jira/browse/DRILL-2125 Project: Apache Drill Issue Type: Improvement Components: Tools, Build Test Reporter: Mehant Baid Assignee: Mehant Baid Fix For: 1.0.0 Attachments: DRILL-2125.patch Currently only some generated source files include information as to which template was used to create the sources. For better readability and modifying the template it'd be good to include which template was used to generate the sources. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2277) COUNT(*) should return 0 instead of an empty result set when there are no records
[
https://issues.apache.org/jira/browse/DRILL-2277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2277:
-
Fix Version/s: (was: 0.9.0)
1.0.0
COUNT(*) should return 0 instead of an empty result set when there are no
records
-
Key: DRILL-2277
URL: https://issues.apache.org/jira/browse/DRILL-2277
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Reporter: Rahul Challapalli
Assignee: Mehant Baid
Priority: Critical
Fix For: 1.0.0
git.commit.id.abbrev=6676f2d
Data Set :
{code}
{
id:1
}
{code}
Query :
{code}
select count(*) from `temp.json` where uid 1;
++
| EXPR$0 |
++
++
{code}
Postgres returns 0 in this case.
Marking it as critical since we return an incorrect result
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2425) Wrong results when identifier change cases within the same data file
[
https://issues.apache.org/jira/browse/DRILL-2425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2425:
-
Fix Version/s: (was: 0.9.0)
1.0.0
Wrong results when identifier change cases within the same data file
Key: DRILL-2425
URL: https://issues.apache.org/jira/browse/DRILL-2425
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Affects Versions: 0.8.0
Reporter: Chun Chang
Assignee: Hanifi Gunes
Priority: Critical
Fix For: 1.0.0
#Fri Mar 06 16:51:10 EST 2015
git.commit.id.abbrev=fb293ba
I have the following JSON file that one of the identifier change cases:
{code}
[root@qa-node120 md-83]# hadoop fs -cat
/drill/testdata/complex_type/json/schema/a.json
{SOURCE: ebm,msAddressIpv6Array: null}
{SOURCE: ebm,msAddressIpv6Array: {msAddressIpv6_1:99.111.222.0,
msAddressIpv6_2:88.222.333.0}}
{SOURCE: ebm,msAddressIpv6Array: {msAddressIpv6_1:99.111.222.1,
msAddressIpv6_2:88.222.333.1}}
{SOURCE: ebm,msAddressIpv6Array: {msaddressipv6_1:99.111.222.2,
msAddressIpv6_2:88.222.333.2}}
{code}
Query this file through drill gives wrong results:
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select
t.msAddressIpv6Array.msAddressIpv6_1 as msAddressIpv6_1 from `schema/a.json`
t;
+-+
| msAddressIpv6_1 |
+-+
| null|
| null|
| null|
| 99.111.222.2|
+-+
{code}
plan:
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ explain plan for select
t.msAddressIpv6Array.msAddressIpv6_1 as msAddressIpv6_1 from `schema/a.json`
t;
+++
|text|json|
+++
| 00-00Screen
00-01 Project(msAddressIpv6_1=[ITEM($0, 'msAddressIpv6_1')])
00-02Scan(groupscan=[EasyGroupScan
[selectionRoot=/drill/testdata/complex_type/json/schema/a.json, numFiles=1,
columns=[`msAddressIpv6Array`.`msAddressIpv6_1`],
files=[maprfs:/drill/testdata/complex_type/json/schema/a.json]]])
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2171) Test framework throws IOOB for tests changing schema
[ https://issues.apache.org/jira/browse/DRILL-2171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2171: - Fix Version/s: (was: 0.9.0) 1.0.0 Test framework throws IOOB for tests changing schema Key: DRILL-2171 URL: https://issues.apache.org/jira/browse/DRILL-2171 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Reporter: Hanifi Gunes Assignee: Hanifi Gunes Fix For: 1.0.0 I added a unit test as part of DRILL-1605 that resolves a problem with schema change. Unfortunately test framework suffers from a similar problem throwing IOOB while trying to verify the results. TestSchemaChange#testMultiFilesWithDifferentSchema is currently ignored until a patch is available for this issue. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2234) IOOB when streaming aggregate is on the left side of hash join
[ https://issues.apache.org/jira/browse/DRILL-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2234: - Fix Version/s: (was: 0.9.0) 1.0.0 IOOB when streaming aggregate is on the left side of hash join -- Key: DRILL-2234 URL: https://issues.apache.org/jira/browse/DRILL-2234 Project: Apache Drill Issue Type: Bug Components: Execution - Relational Operators Reporter: Mehant Baid Assignee: Mehant Baid Fix For: 1.0.0 This issue is similar to DRILL-2107. Issue can be reproduced by enabling SwapJoinRule in DrillRuleSets and running the following query. alter session set `planner.slice_target` = 1; alter session set `planner.enable_hashagg` = false; alter session set `planner.enable_streamagg` = true; select l_suppkey, sum(l_extendedprice)/sum(l_quantity) as avg_price from cp.`tpch/lineitem.parquet` where l_orderkey in (select o_orderkey from cp.`tpch/orders.parquet` where o_custkey = 2) group by l_suppkey having sum(l_extendedprice)/sum(l_quantity) 1850.0; -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2340) count(*) fails with subquery not containing limit
[
https://issues.apache.org/jira/browse/DRILL-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2340:
-
Fix Version/s: (was: 0.9.0)
1.0.0
count(*) fails with subquery not containing limit
-
Key: DRILL-2340
URL: https://issues.apache.org/jira/browse/DRILL-2340
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Reporter: Chun Chang
Assignee: Hanifi Gunes
Priority: Critical
Fix For: 1.0.0
#Wed Feb 25 17:07:31 EST 2015
git.commit.id.abbrev=f7ef5ec
count(*) with subquery containing limit works fine:
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select count(*) from (select
t.soa[0] soa0, t.soa[1] soa1, t.soa[2] soa2 from `complex.json` t limit
20) tt;
++
| EXPR$0 |
++
| 20 |
++
{code}
But if I remove the limit, query fails with IllegalStateException:
{code}
0: jdbc:drill:schema=dfs.drillTestDirComplexJ select count(*) from (select
t.soa[0] soa0, t.soa[1] soa1, t.soa[2] soa2 from `complex.json` t) tt;
++
| EXPR$0 |
++
Query failed: RemoteRpcException: Failure while running fragment., You tried
to do a batch data read operation when you were in a state of STOP. You can
only do this type of operation when you are in a state of OK or
OK_NEW_SCHEMA. [ d3226020-a2b0-4497-948f-34ea2309ddb7 on
qa-node120.qa.lab:31010 ]
[ d3226020-a2b0-4497-948f-34ea2309ddb7 on qa-node120.qa.lab:31010 ]
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Here is the exception in drill bit.log:
{code}
2015-02-27 14:17:32,247 [2b0f1303-61ec-2350-4b62-b6b29d11c534:foreman] INFO
o.a.drill.exec.work.foreman.Foreman - State change requested. PENDING --
RUNNING
2015-02-27 14:17:32,267 [2b0f1303-61ec-2350-4b62-b6b29d11c534:frag:0:0] ERROR
o.a.drill.exec.ops.FragmentContext - Fragment Context received failure.
java.lang.IllegalStateException: Needed to be in state INIT or IN_FLOAT8 but
in mode IN_BIGINT
at
org.apache.drill.exec.vector.complex.impl.SingleListWriter.float8(SingleListWriter.java:427)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:418)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.fn.JsonReader.writeData(JsonReader.java:256)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.fn.JsonReader.writeDataSwitch(JsonReader.java:208)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.fn.JsonReader.writeToVector(JsonReader.java:182)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.fn.JsonReader.write(JsonReader.java:156)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.easy.json.JSONRecordReader.next(JSONRecordReader.java:125)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:165)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:99)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:89)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:51)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:134)
[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
[jira] [Updated] (DRILL-1782) Error in drill-root pom - Typo in excludeSubprojects
[ https://issues.apache.org/jira/browse/DRILL-1782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1782: - Fix Version/s: (was: 0.9.0) 1.0.0 Error in drill-root pom - Typo in excludeSubprojects Key: DRILL-1782 URL: https://issues.apache.org/jira/browse/DRILL-1782 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Affects Versions: 0.7.0 Reporter: Yash Sharma Assignee: Mehant Baid Fix For: 1.0.0 Attachments: DRILL-1782.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2227) Using flatten in a project, filter (where in ...), and order by throws an ClassCastException
[
https://issues.apache.org/jira/browse/DRILL-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2227:
-
Fix Version/s: (was: 0.9.0)
1.0.0
Using flatten in a project, filter (where in ...), and order by throws an
ClassCastException
Key: DRILL-2227
URL: https://issues.apache.org/jira/browse/DRILL-2227
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Reporter: Rahul Challapalli
Assignee: Hanifi Gunes
Fix For: 1.0.0
Attachments: data.json, error.log
git.commit.id.abbrev=3d863b5
The below query fails :
{code}
select uid, flatten(lst_lst[0]) from `data.json` where 2 in (select
flatten(lst_lst[1]) from `data.json`) order by flatten(lst_lst[1]);
Query failed: RemoteRpcException: Failure while running fragment.,
org.apache.drill.exec.vector.BigIntVector cannot be cast to
org.apache.drill.exec.vector.RepeatedVector
{code}
However if we apply flatten on the same column in the select part, it
succeeds
{code}
0: jdbc:drill:schema=dfs_eea select uid, flatten(lst_lst[1]) from
`data.json` where 2 in (select flatten(lst_lst[1]) from `data.json`) order by
flatten(lst_lst[1]);
+++
|uid | EXPR$1 |
+++
| 1 | 2 |
| 2 | 2 |
| 1 | 3 |
| 2 | 3 |
| 1 | 4 |
| 2 | 4 |
| 1 | 5 |
| 2 | 5 |
| 1 | 6 |
| 2 | 6 |
+++
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2681) add sql functions docs (string, convert_to/from)
[ https://issues.apache.org/jira/browse/DRILL-2681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn updated DRILL-2681: - Description: Includes SQL scalar functions, maprdb plugin doc fixes, time zone limitations section, Query HBase doc fixes, fix broken links in contribution guidelines and contribution ideas, prelim. overview of config options add sql functions docs (string, convert_to/from) Key: DRILL-2681 URL: https://issues.apache.org/jira/browse/DRILL-2681 Project: Apache Drill Issue Type: Task Components: Documentation Reporter: Kristine Hahn Assignee: Kristine Hahn Includes SQL scalar functions, maprdb plugin doc fixes, time zone limitations section, Query HBase doc fixes, fix broken links in contribution guidelines and contribution ideas, prelim. overview of config options -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2684) Unique-ify view names in TestUnionAll TestExampleQueries test suites
[ https://issues.apache.org/jira/browse/DRILL-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanifi Gunes updated DRILL-2684: Assignee: Parth Chandra (was: Hanifi Gunes) Unique-ify view names in TestUnionAll TestExampleQueries test suites -- Key: DRILL-2684 URL: https://issues.apache.org/jira/browse/DRILL-2684 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Reporter: Hanifi Gunes Assignee: Parth Chandra Priority: Blocker Fix For: 0.9.0 Two test classes rely on the same view, region_name. One drops the view while the other is working on it and suddenly execution fails. Bummer! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2684) Unique-ify view names in TestUnionAll TestExampleQueries test suites
[ https://issues.apache.org/jira/browse/DRILL-2684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanifi Gunes updated DRILL-2684: Attachment: DRILL-2684.1.patch.txt Use unique view names to avoid failing concurrent test runs. Unique-ify view names in TestUnionAll TestExampleQueries test suites -- Key: DRILL-2684 URL: https://issues.apache.org/jira/browse/DRILL-2684 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Reporter: Hanifi Gunes Assignee: Parth Chandra Priority: Blocker Fix For: 0.9.0 Attachments: DRILL-2684.1.patch.txt Two test classes rely on the same view, region_name. One drops the view while the other is working on it and suddenly execution fails. Bummer! -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2039) use forks-per-CPU units in forkCount
[
https://issues.apache.org/jira/browse/DRILL-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482137#comment-14482137
]
Jacques Nadeau commented on DRILL-2039:
---
Should this be 0.5 per logical core? I assume that on quad with ht, that would
be 8 logical cores. However, we find we hit timeouts after 4 threads on that
type of machine.
use forks-per-CPU units in forkCount
Key: DRILL-2039
URL: https://issues.apache.org/jira/browse/DRILL-2039
Project: Apache Drill
Issue Type: Improvement
Components: Tools, Build Test
Reporter: Daniel Barclay (Drill)
Assignee: Parth Chandra
Priority: Minor
Fix For: 0.9.0
Attachments: DRILL-2039.1.patch.txt
Can Surefire's fork count be set using units of forks per CPU instead of the
absolute number of forks (i.e., {{forkCount1C/forkCount}} instead of
{{forkCount4/forkcount}})?
That would let developers with more powerful machines still run tests in
parallel while letting others avoid or reduce test timeout errors without the
risk of forgetting to re-edit pom.xml after every switch to a different git
branch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-658) Add support for multiple distinct counts without a grouping key.
[
https://issues.apache.org/jira/browse/DRILL-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14482235#comment-14482235
]
Victoria Markman commented on DRILL-658:
Can we please try fixing this error for GA possibly ? This will have to be
documented as a limitation in aggregate function section.
Add support for multiple distinct counts without a grouping key.
Key: DRILL-658
URL: https://issues.apache.org/jira/browse/DRILL-658
Project: Apache Drill
Issue Type: New Feature
Components: Query Planning Optimization
Reporter: Chun Chang
Assignee: Jinfeng Ni
Fix For: Future
postgres:
foodmart=# select count(distinct(c_int)), count(distinct(c_bigint)),
count(distinct(c_float4)), count(distinct(c_float8)) from data;
count | count | count | count
---+---+---+---
21 |23 |22 |20
(1 row)
drill:
0: jdbc:drill:schema=dfs select count(distinct(c_int)),
count(distinct(c_bigint)), count(distinct(c_float4)),
count(distinct(c_float8)) from data;
Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
running query.[error_id: 658d1c88-3968-457f-82e1-26c76ff7b54b
endpoint {
address: qa-node120.qa.lab
user_port: 31010
control_port: 31011
data_port: 31012
}
error_type: 0
message: Failure while parsing sql. CannotPlanException:[ Node
[rel#5852:Subset#33.PHYSICAL.SINGLETON([]).[]] could not be implemented;
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2611) Fixed length value vectors report invalid value count
[ https://issues.apache.org/jira/browse/DRILL-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2611: - Priority: Critical (was: Major) Fixed length value vectors report invalid value count - Key: DRILL-2611 URL: https://issues.apache.org/jira/browse/DRILL-2611 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Reporter: Hanifi Gunes Assignee: Hanifi Gunes Priority: Critical Fix For: 0.9.0 We maintain an exclusive value count variable in fixed vectors however we don't update it upon calling set/Safe. Accessor reports the value count from the variable ignoring values that are already in the buffer or written via set/Safe. This causes execution failures manifested as IOOB when underlying data is sparse. We should either remove the variable and report value count directly investigating the buffer if not computationally expensive or update the variable each time we make a write to the vector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2611) Fixed length value vectors report invalid value count
[ https://issues.apache.org/jira/browse/DRILL-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2611: - Fix Version/s: 0.9.0 Fixed length value vectors report invalid value count - Key: DRILL-2611 URL: https://issues.apache.org/jira/browse/DRILL-2611 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Reporter: Hanifi Gunes Assignee: Hanifi Gunes Fix For: 0.9.0 We maintain an exclusive value count variable in fixed vectors however we don't update it upon calling set/Safe. Accessor reports the value count from the variable ignoring values that are already in the buffer or written via set/Safe. This causes execution failures manifested as IOOB when underlying data is sparse. We should either remove the variable and report value count directly investigating the buffer if not computationally expensive or update the variable each time we make a write to the vector. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2701) configuration options
Kristine Hahn created DRILL-2701: Summary: configuration options Key: DRILL-2701 URL: https://issues.apache.org/jira/browse/DRILL-2701 Project: Apache Drill Issue Type: Task Components: Documentation Reporter: Kristine Hahn Assignee: Kristine Hahn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
