[jira] [Updated] (DRILL-1832) Select * from json file failed with java.lang.IllegalArgumentException: null
[ https://issues.apache.org/jira/browse/DRILL-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1832: - Attachment: DRILL-1832.2.patch.diff Select * from json file failed with java.lang.IllegalArgumentException: null Key: DRILL-1832 URL: https://issues.apache.org/jira/browse/DRILL-1832 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types, Execution - Flow Affects Versions: 0.7.0 Reporter: Krystal Assignee: Parth Chandra Fix For: 1.0.0 Attachments: DRILL-1832.2.patch.diff, drill-1832.log, twitter_43.json git.commit.id.abbrev=201280e I have a json file that contains nested data. When I do a select * , the query failed as follows: 0: jdbc:drill:schema=dfs.root select * from `./json/twitter_43.json`; Query failed: Query stopped.[ cc231dec-8d28-4532-a938-c5f67592f262 on qa-node114.qa.lab:31010 ] Error: exception while executing query: Failure while executing query. (state=,code=0) Also, the following select fails: 0: jdbc:drill:schema=dfs.root select SUM(1) as `sum_Number_of_Records_ok` from `./json/twitter_43.json` having (COUNT(1) 0); Query failed: Query failed: Failure while running fragment., You tried to do a batch data read operation when you were in a state of STOP. You can only do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [ 364d911a-3bc8-4042-b5a2-6ed074c999ce on qa-node114.qa.lab:31010 ] [ 364d911a-3bc8-4042-b5a2-6ed074c999ce on qa-node114.qa.lab:31010 ] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1832) Select * from json file failed with java.lang.IllegalArgumentException: null
[ https://issues.apache.org/jira/browse/DRILL-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1832: - Attachment: (was: DRILL-1832.patch.diff) Select * from json file failed with java.lang.IllegalArgumentException: null Key: DRILL-1832 URL: https://issues.apache.org/jira/browse/DRILL-1832 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types, Execution - Flow Affects Versions: 0.7.0 Reporter: Krystal Assignee: Parth Chandra Fix For: 1.0.0 Attachments: DRILL-1832.2.patch.diff, drill-1832.log, twitter_43.json git.commit.id.abbrev=201280e I have a json file that contains nested data. When I do a select * , the query failed as follows: 0: jdbc:drill:schema=dfs.root select * from `./json/twitter_43.json`; Query failed: Query stopped.[ cc231dec-8d28-4532-a938-c5f67592f262 on qa-node114.qa.lab:31010 ] Error: exception while executing query: Failure while executing query. (state=,code=0) Also, the following select fails: 0: jdbc:drill:schema=dfs.root select SUM(1) as `sum_Number_of_Records_ok` from `./json/twitter_43.json` having (COUNT(1) 0); Query failed: Query failed: Failure while running fragment., You tried to do a batch data read operation when you were in a state of STOP. You can only do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [ 364d911a-3bc8-4042-b5a2-6ed074c999ce on qa-node114.qa.lab:31010 ] [ 364d911a-3bc8-4042-b5a2-6ed074c999ce on qa-node114.qa.lab:31010 ] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1643) Group count incorrect in repeated map vector
[ https://issues.apache.org/jira/browse/DRILL-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-1643: --- Component/s: Execution - Data Types Group count incorrect in repeated map vector Key: DRILL-1643 URL: https://issues.apache.org/jira/browse/DRILL-1643 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Reporter: Jason Altekruse Fix For: 0.7.0 The group count should indicate the number of groups or lists in the entire batch, this should be equal to the record count of the batch, or the value count on a scalar vector. Currently the repeatedMapVector incorrectly returns the number of map fields instead. This is causing issues with flatten. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2832) JDBC : Client side exception when applying flatten after a join on a repeated index
Rahul Challapalli created DRILL-2832:
Summary: JDBC : Client side exception when applying flatten after
a join on a repeated index
Key: DRILL-2832
URL: https://issues.apache.org/jira/browse/DRILL-2832
Project: Apache Drill
Issue Type: Bug
Components: Client - JDBC, Execution - Relational Operators
Reporter: Rahul Challapalli
Assignee: Daniel Barclay (Drill)
git.commit.id.abbrev=5cd36c5
Query :
{code}
select flatten(t1.events) from `data.json` t1 inner join `data.json` t2 on
t1.events[0].campaign_id = t2.events[2].campaign_id;
++
| EXPR$0 |
++
java.lang.RuntimeException: java.sql.SQLException: Failure while executing
query.
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2514)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
The below 2 queries works fine
{code}
1. Without Flatten : select * from `data.json` t1 inner join `data.json` t2 on
t1.events[0].campaign_id = t2.events[2].campaign_id;
2. Join on simple column : select flatten(t1.events) a from `data.json` t1
inner join `data.json` t2 on t1.uid = t2.uid;
{code}
There is no error in the logs. I attached the data file. Let me know if you
need anything.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-1472) Function resolution cannot handle repeated list as an input type to a UDF
[ https://issues.apache.org/jira/browse/DRILL-1472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-1472: --- Component/s: Functions - Drill Function resolution cannot handle repeated list as an input type to a UDF - Key: DRILL-1472 URL: https://issues.apache.org/jira/browse/DRILL-1472 Project: Apache Drill Issue Type: Bug Components: Functions - Drill Reporter: Jason Altekruse Fix For: 0.8.0 I tried adding a function to allow counting the elements of a repeated list. Using an input type of RepeatedListHolder does not appear to be handled in the current function resolver. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2834) Applying multiple flattens after a join results in an NPE
[
https://issues.apache.org/jira/browse/DRILL-2834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rahul Challapalli updated DRILL-2834:
-
Attachment: data.json
error.log
Applying multiple flattens after a join results in an NPE
-
Key: DRILL-2834
URL: https://issues.apache.org/jira/browse/DRILL-2834
Project: Apache Drill
Issue Type: Bug
Components: Execution - Relational Operators
Reporter: Rahul Challapalli
Assignee: Chris Westin
Attachments: data.json, error.log
git.commit.id.abbrev=5cd36c5
The below query fails :
{code}
select flatten(t1.lst_lst), flatten(t1.events) from `data.json` t1 inner join
`data.json` t2 on t1.uid = t2.uid;
Query failed: SYSTEM ERROR: null
Fragment 0:0
[a27e431d-a598-4ec6-86b8-eb91d5d54161 on qa-node190.qa.lab:31010]
{code}
However if we only apply a single flatten things work as expected.
I attached the error logs and the data file. Let me know if you need anything
more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-2117) tpcds impala q53 query randomly fails with memory leak
[
https://issues.apache.org/jira/browse/DRILL-2117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krystal closed DRILL-2117.
--
close as a duplicate
tpcds impala q53 query randomly fails with memory leak
--
Key: DRILL-2117
URL: https://issues.apache.org/jira/browse/DRILL-2117
Project: Apache Drill
Issue Type: Bug
Components: Execution - RPC
Affects Versions: 0.8.0
Reporter: Krystal
Assignee: Victoria Markman
Attachments: drill2117.log
Commit 54b5f80
The following tpcds impala sf1 q53 randomly fails:
select
*
from
(select
i.i_manufact_id as imid,
sum(ss.ss_sales_price) sum_sales
-- avg(sum(ss.ss_sales_price)) over (partition by i.i_manufact_id)
avg_quarterly_sales
from
item as i,
store_sales as ss,
date_dim as d,
store as s
where
ss.ss_item_sk = i.i_item_sk
and ss.ss_sold_date_sk = d.d_date_sk
and ss.ss_store_sk = s.s_store_sk
and d.d_month_seq in (1212, 1212 + 1, 1212 + 2, 1212 + 3, 1212 + 4, 1212
+ 5, 1212 + 6, 1212 + 7, 1212 + 8, 1212 + 9, 1212 + 10, 1212 + 11)
and ((i.i_category in ('Books', 'Children', 'Electronics')
and i.i_class in ('personal', 'portable', 'reference', 'self-help')
and i.i_brand in ('scholaramalgamalg #14', 'scholaramalgamalg #7',
'exportiunivamalg #9', 'scholaramalgamalg #9'))
or (i.i_category in ('Women', 'Music', 'Men')
and i.i_class in ('accessories', 'classical', 'fragrances', 'pants')
and i.i_brand in ('amalgimporto #1', 'edu packscholar #1',
'exportiimporto #1', 'importoamalg #1')))
and ss.ss_sold_date_sk between 2451911 and 2452275 -- partition key filter
group by
i.i_manufact_id,
d.d_qoy
) tmp1
-- where
-- case when avg_quarterly_sales 0 then abs (sum_sales -
avg_quarterly_sales) / avg_quarterly_sales else null end 0.1
order by
-- avg_quarterly_sales,
sum_sales,
tmp1.imid
limit 100
Query failed: RemoteRpcException: Failure while running fragment., Attempted
to close accountor with 1 buffer(s) still allocatedfor QueryId:
2b35963d-aae2-d069-c5cc-c8ce3d414cf6, MajorFragmentId: 4, MinorFragmentId: 9.
Total 1 allocation(s) of byte size(s): 23058, at stack location:
org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.takeOwnership(TopLevelAllocator.java:197)
org.apache.drill.exec.rpc.data.DataServer.handle(DataServer.java:119)
org.apache.drill.exec.rpc.data.DataServer.handle(DataServer.java:48)
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:194)
org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:173)
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:161)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:319)
io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:787)
io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:130)
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
[jira] [Closed] (DRILL-1941) select from subquery having an order by fails
[ https://issues.apache.org/jira/browse/DRILL-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krystal closed DRILL-1941. -- Close as a duplicate select from subquery having an order by fails - Key: DRILL-1941 URL: https://issues.apache.org/jira/browse/DRILL-1941 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Affects Versions: 0.8.0 Reporter: Krystal Assignee: Sean Hsuan-Yi Chu Fix For: 0.9.0 Attachments: drill_err.log git.commit.id.abbrev=b491cdb The following query fails: 0: jdbc:drill:schema=dfs select q.name as name, q.registration as registration from (select cast(student.name as varchar(30)) as name, cast(voter.registration as varchar(20)) as registration from `dfs`.`default`.`voter` voter full outer join `dfs`.`default`.`./student` student on (student.name = voter.name) where student.age 30 order by student.name) q; Query failed: Query failed: Unexpected exception during fragment initialization: -1 Error: exception while executing query: Failure while executing query. (state=,code=0) Running the same query with explain plan for also fail with the same error. If I remove the order by student.name from the sub-query, the query run successfully. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1787) Memory leak in kvgen function
[ https://issues.apache.org/jira/browse/DRILL-1787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-1787: --- Component/s: Functions - Drill Memory leak in kvgen function - Key: DRILL-1787 URL: https://issues.apache.org/jira/browse/DRILL-1787 Project: Apache Drill Issue Type: Bug Components: Functions - Drill Reporter: Jason Altekruse Assignee: Jason Altekruse Fix For: 0.7.0 Running a kvgen on a map with large values will produce a memory leak. This is due to improper use of the reallocIfNeeded() method on DrillBuf. This method returns a new buffer, it does not resize the buffer within the one that is calling the method. This buffer is automatically freed by the method itself, so a reference just needs to be saved in the same variable holding the original buffer. Error message produced is standard memory leak error in Drill, pasted below. Query failed: Failure while running fragment., Attempted to close accountor with 6 buffer(s) still allocatedfor QueryId: 64a39e3c-42a9-455d-aaee-0c7374492835, MajorFragmentId: 0, MinorFragmentId: 0. Total 1 allocation(s) of byte size(s): 16384, at stack location: org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:212) org.apache.drill.exec.vector.UInt4Vector.allocateNewSafe(UInt4Vector.java:137) org.apache.drill.exec.vector.complex.RepeatedMapVector.allocateNewSafe(RepeatedMapVector.java:232) org.apache.drill.exec.vector.complex.impl.RepeatedMapWriter.allocate(RepeatedMapWriter.java:135) org.apache.drill.exec.vector.complex.impl.SingleListWriter.allocate(SingleListWriter.java:116) org.apache.drill.exec.vector.complex.impl.ComplexWriterImpl.allocate(ComplexWriterImpl.java:157) org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doAlloc(ProjectRecordBatch.java:217) org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.doWork(ProjectRecordBatch.java:144) org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:89) org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129) org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:106) org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:124) org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:86) org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:76) org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext(AbstractSingleRecordBatch.java:52) org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:129) org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:106) org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:124) org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67) org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:122) org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57) org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:113) org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:249) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:744) Total 1 allocation(s) of byte size(s): 4096, at stack location: org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:212) org.apache.drill.exec.vector.UInt1Vector.allocateNewSafe(UInt1Vector.java:137) org.apache.drill.exec.vector.NullableVarCharVector.allocateNewSafe(NullableVarCharVector.java:170) org.apache.drill.exec.vector.complex.impl.NullableVarCharWriterImpl.allocate(NullableVarCharWriterImpl.java:113) org.apache.drill.exec.vector.complex.impl.RepeatedMapWriter.allocate(RepeatedMapWriter.java:137) org.apache.drill.exec.vector.complex.impl.SingleListWriter.allocate(SingleListWriter.java:116)
[jira] [Closed] (DRILL-1826) Cannot run queries against latest 0.70 tar ball
[ https://issues.apache.org/jira/browse/DRILL-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krystal closed DRILL-1826. -- Close issue. Opened another bug. Cannot run queries against latest 0.70 tar ball --- Key: DRILL-1826 URL: https://issues.apache.org/jira/browse/DRILL-1826 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Affects Versions: 0.7.0 Reporter: Krystal Assignee: Patrick Wong Fix For: 0.7.0 Attachments: drill-1826.log [root@mfs41 ~]# cat /opt/drill/git.properties #Generated by Git-Commit-Id-Plugin #Mon Dec 08 10:59:10 EST 2014 git.commit.id.abbrev=503a5b2 git.commit.user.email=jacq...@apache.org git.commit.message.full=DRILL-1769\: Fix issue where option manager not holding settings.\n git.commit.id=503a5b2a4b23c6dbd5c82843bfc81b04f76ed7c7 git.commit.message.short=DRILL-1769\: Fix issue where option manager not holding settings. git.commit.user.name=Jacques Nadeau git.build.user.name=Unknown git.commit.id.describe=drill-1.0.0-m1-1002-g503a5b2-dirty git.build.user.email=Unknown git.branch=503a5b2a4b23c6dbd5c82843bfc81b04f76ed7c7 git.commit.time=08.12.2014 @ 10\:55\:02 EST git.build.time=08.12.2014 @ 10\:59\:10 EST git.remote.origin.url=https\://git-wip-us.apache.org/repos/asf/drill.git/ Installed the latest 0.7 drill tar ball. Queries fail with Queue closed due to channel closure error: [root@mfs41 bin]# /opt/drill/bin/sqlline --maxWidth=1 -n admin -p admin -u jdbc:drill:schema=dfs;zk=10.10.10.41:5181 Drill log directory /var/log/drill does not exist, defaulting to /opt/drill/log sqlline version 1.1.6 0: jdbc:drill:schema=dfs select * from voter limit 5; Query failed: Queue closed due to channel closure. Error: exception while executing query: Failure while executing query. (state=,code=0) The rpm version works fine. So something is different between the tar ball and rpm package. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2383) add exception and pause injections for testing drillbit stability
[
https://issues.apache.org/jira/browse/DRILL-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sudheesh Katkam updated DRILL-2383:
---
Attachment: DRILL-2383.8.patch.txt
add exception and pause injections for testing drillbit stability
-
Key: DRILL-2383
URL: https://issues.apache.org/jira/browse/DRILL-2383
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow
Reporter: Chris Westin
Assignee: Jacques Nadeau
Fix For: 0.9.0
Attachments: DRILL-2383.1.patch.txt, DRILL-2383.3.patch.txt,
DRILL-2383.6.patch.txt, DRILL-2383.7.patch.txt, DRILL-2383.8.patch.txt
Use the exception injection mechanism to add exception injections to test a
variety of distributed failure scenarios.
Here are some scenarios we've worked out before:
1. Cancellation:
TC1: cancel before any result set is returned
TC2: cancel in the middle of fetching result set
TC3: cancel after all result set are produced but not all are fetched
TC4: cancel after everything is completed and fetched
As test setup, we need:
- query dataset large enough to be sent to different drillbits, e.g., TPCH
100
- queries that force multiple drillbits to work on them; e.g., count ...
group by
2. Completed (in each case check all drillbits are still up and running):
TC1: success
TC2: failed query - before query is executed - while sql parsing
TC3: failed query - before query is executed - while sending fragments to
other drillbits for execution
TC4: failed query - during query execution
It is currently not possible to create a scenario in which a query may hang.
To check all drillbits up and running and in a clean state, run:
-select count(*) from sys.drillbits;-
{code}
select count(*) from sys.memory;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2383) add exception and pause injections for testing drillbit stability
[
https://issues.apache.org/jira/browse/DRILL-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sudheesh Katkam updated DRILL-2383:
---
Assignee: Parth Chandra (was: Jacques Nadeau)
add exception and pause injections for testing drillbit stability
-
Key: DRILL-2383
URL: https://issues.apache.org/jira/browse/DRILL-2383
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow
Reporter: Chris Westin
Assignee: Parth Chandra
Fix For: 0.9.0
Attachments: DRILL-2383.1.patch.txt, DRILL-2383.3.patch.txt,
DRILL-2383.6.patch.txt, DRILL-2383.7.patch.txt, DRILL-2383.8.patch.txt
Use the exception injection mechanism to add exception injections to test a
variety of distributed failure scenarios.
Here are some scenarios we've worked out before:
1. Cancellation:
TC1: cancel before any result set is returned
TC2: cancel in the middle of fetching result set
TC3: cancel after all result set are produced but not all are fetched
TC4: cancel after everything is completed and fetched
As test setup, we need:
- query dataset large enough to be sent to different drillbits, e.g., TPCH
100
- queries that force multiple drillbits to work on them; e.g., count ...
group by
2. Completed (in each case check all drillbits are still up and running):
TC1: success
TC2: failed query - before query is executed - while sql parsing
TC3: failed query - before query is executed - while sending fragments to
other drillbits for execution
TC4: failed query - during query execution
It is currently not possible to create a scenario in which a query may hang.
To check all drillbits up and running and in a clean state, run:
-select count(*) from sys.drillbits;-
{code}
select count(*) from sys.memory;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-1471) Hbase queries fail with / by zero
[ https://issues.apache.org/jira/browse/DRILL-1471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krystal closed DRILL-1471. -- Close bug as a duplicate Hbase queries fail with / by zero - Key: DRILL-1471 URL: https://issues.apache.org/jira/browse/DRILL-1471 Project: Apache Drill Issue Type: Bug Components: Storage - HBase Reporter: Krystal Assignee: Aditya Kishore Fix For: 0.6.0 git.commit.id.abbrev=81fb18a The following hbase query fails: 0: jdbc:drill:schema=dfs select cast(row_key as integer) voter_id, convert_from(onecf['name'], 'UTF8') name, cast(twocf['age'] as integer) age, cast(twocf['registration'] as varchar(20)) registration, cast(threecf['contributions'] as decimal(6,2)) contributions, cast(threecf['voterzone'] as integer) voterzone,cast(fourcf['create_date'] as timestamp) create_date from M7.m7voter where onecf['name'] not similar to '%(young|u|a|i|e|m)%'; Query failed: Failure while setting up Foreman. / by zero [176bbea0-b240-4ed2-88a6-b0e707c0e442] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1688) Drill fails to read wide parquet records with complex schemas
[ https://issues.apache.org/jira/browse/DRILL-1688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-1688: --- Component/s: Storage - Parquet Drill fails to read wide parquet records with complex schemas - Key: DRILL-1688 URL: https://issues.apache.org/jira/browse/DRILL-1688 Project: Apache Drill Issue Type: Bug Components: Storage - Parquet Reporter: Jason Altekruse Fix For: 0.7.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-470) Add stack traces to drill error messages
[ https://issues.apache.org/jira/browse/DRILL-470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-470: -- Component/s: Execution - RPC Add stack traces to drill error messages Key: DRILL-470 URL: https://issues.apache.org/jira/browse/DRILL-470 Project: Apache Drill Issue Type: Bug Components: Execution - RPC Reporter: Jason Altekruse Assignee: Jason Altekruse Fix For: 0.7.0 Errors occurring during drill execution currently only return the error message, not the stack trace leading up to the error. Often time we are debugging errors that cannot be meaningfully diagnosed without a stack trace. The current architecture is designed for a release, where we would be wanting to hide the stack traces from users, but the current state of the project would seem to make it more useful to include the stack trace until we reach GA. Current solution is to use Lilith log viewer, while this does work, lilith can be a pain and is an extra step needed almost any time a bug is explored. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-101) Value Vector External Artifact with demo
[ https://issues.apache.org/jira/browse/DRILL-101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-101: -- Component/s: Execution - Data Types Value Vector External Artifact with demo Key: DRILL-101 URL: https://issues.apache.org/jira/browse/DRILL-101 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Reporter: Jason Altekruse Labels: demo, interface, value, vector Fix For: Future Allow other projects to easily make use of value vectors without depending on all of drill. Develop an example use case for value vectors outside of drill. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-81) Test written for the Constant operator is not robust
[ https://issues.apache.org/jira/browse/DRILL-81?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-81: - Component/s: Tools, Build Test Test written for the Constant operator is not robust Key: DRILL-81 URL: https://issues.apache.org/jira/browse/DRILL-81 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Reporter: Jason Altekruse Assignee: Jason Altekruse Labels: constantROP, test Fix For: 0.7.0 Attachments: 0001-Improved-the-test-for-the-new-constant-reference-ope.patch Original Estimate: 1h Remaining Estimate: 1h Comparing results of the toString method of an object to hard coded strings is not a robust test verification mechanism, the test should be rewritten. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1451) Classpath storage plugin cannot be used in test queries referencing delimited text files, fails with file not found
[ https://issues.apache.org/jira/browse/DRILL-1451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-1451: --- Component/s: Tools, Build Test Classpath storage plugin cannot be used in test queries referencing delimited text files, fails with file not found --- Key: DRILL-1451 URL: https://issues.apache.org/jira/browse/DRILL-1451 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Reporter: Jason Altekruse Assignee: Jason Altekruse Fix For: 0.7.0 Attachments: DRILL-1451.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-314) Default Build fails on linux box with 4 gigs of ram
[ https://issues.apache.org/jira/browse/DRILL-314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-314: -- Component/s: Tools, Build Test Default Build fails on linux box with 4 gigs of ram --- Key: DRILL-314 URL: https://issues.apache.org/jira/browse/DRILL-314 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Environment: ubuntu 12.04 java version 1.7.0_45 Java(TM) SE Runtime Environment (build 1.7.0_45-b18) Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mode) Reporter: Jason Altekruse Fix For: 0.4.0 watching the system monitor I can see the memory climb throughout the build process. Need to look at how to get memory to be freed sooner after each test. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2544) Quitting sqlline results in a debug message Closing: org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection
[
https://issues.apache.org/jira/browse/DRILL-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Barclay (Drill) updated DRILL-2544:
--
Component/s: (was: Client - JDBC)
Quitting sqlline results in a debug message Closing:
org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection
---
Key: DRILL-2544
URL: https://issues.apache.org/jira/browse/DRILL-2544
Project: Apache Drill
Issue Type: Bug
Reporter: Ramana Inukonda Nagaraj
Assignee: Daniel Barclay (Drill)
Fix For: 1.1.0
{code}
[root@atsqa6c57 ~]# /opt/drill/bin/sqlline -u jdbc:drill:zk=localhost:5181
/opt/drill/bin/drill-config.sh: line 94: [: =: unary operator expected
sqlline version 1.1.6
0: jdbc:drill:zk=localhost:5181 !q
Closing: org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2757) Verify sort spill and other operators correctly handle low memory conditions and cancellations
[ https://issues.apache.org/jira/browse/DRILL-2757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Westin updated DRILL-2757: Assignee: Deneche A. Hakim (was: Chris Westin) Verify sort spill and other operators correctly handle low memory conditions and cancellations -- Key: DRILL-2757 URL: https://issues.apache.org/jira/browse/DRILL-2757 Project: Apache Drill Issue Type: Bug Components: Execution - Flow, Execution - Relational Operators Affects Versions: 0.8.0 Reporter: Chris Westin Assignee: Deneche A. Hakim Fix For: 1.0.0 Check that the path through sort that notices low memory conditions and causes the sort to spill (out of memory condition management). Also check to make sure we handle queries and fragment failures properly under these conditions. hashjoin, hashagg, and topn use large amounts of memory,and may be unable to complete if their memory needs can't be met; for all others, the idea is that they can complete if they get their reservation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1648) Incorrect schema reported during fast schema phase of flatten causes downstream code compilation to fail
[ https://issues.apache.org/jira/browse/DRILL-1648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-1648: --- Component/s: Execution - Relational Operators Incorrect schema reported during fast schema phase of flatten causes downstream code compilation to fail Key: DRILL-1648 URL: https://issues.apache.org/jira/browse/DRILL-1648 Project: Apache Drill Issue Type: Bug Components: Execution - Relational Operators Reporter: Jason Altekruse Fix For: 0.7.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-341) Cannot pass instance specific information to JSON Reader
[ https://issues.apache.org/jira/browse/DRILL-341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-341: -- Component/s: Storage - JSON Cannot pass instance specific information to JSON Reader Key: DRILL-341 URL: https://issues.apache.org/jira/browse/DRILL-341 Project: Apache Drill Issue Type: Bug Components: Storage - JSON Reporter: Jason Altekruse Assignee: Jason Altekruse Fix For: 0.4.0 Code in the JSON reader currently is all inside of an enum. All Enums in Java are static, so there is no way to pass in instance specific filtering information to each reader. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-340) Ensure methods intended to create new copies of plan nodes are actually creating copies of all nested objects
[ https://issues.apache.org/jira/browse/DRILL-340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-340: -- Component/s: Query Planning Optimization Ensure methods intended to create new copies of plan nodes are actually creating copies of all nested objects - Key: DRILL-340 URL: https://issues.apache.org/jira/browse/DRILL-340 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Reporter: Jason Altekruse Fix For: Future Methods such as getNewWithChildren declared in the PhysicalOperator interface are designed to create copies of plan nodes. These copy operations should not reference any part of the old nodes, thus child objects should be deep copied, including lists of objects. For more information on the desired functionality see here: http://en.wikipedia.org/wiki/Clone_(Java_method) Existing implementations should be checked for correctness. We may want to create this functionality for logical operators as well, but I do not believe that we should need to do manipulations on logical plans that would require it. Our optimization of plans will happen in the logical to physical plan transformation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-342) Parquet Tests are poorly organized
[ https://issues.apache.org/jira/browse/DRILL-342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-342: -- Component/s: Storage - Parquet Parquet Tests are poorly organized -- Key: DRILL-342 URL: https://issues.apache.org/jira/browse/DRILL-342 Project: Apache Drill Issue Type: Bug Components: Storage - Parquet Reporter: Jason Altekruse Assignee: Jason Altekruse Fix For: 0.4.0 Attachments: 0001-Drill-238-342-Drill-nullable-column-bug-and-test-reo.patch, 0001-Drill-342-238-rebased-changes-on-top-of-current-mast.patch Parquet tests diverged after the test class was moved. A bad git merge allowed the old test to remain in its original directory while the modified version of the tests were in a sub-package. Need to consolidate into a single class and reorganize to remove redundancies. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-381) Implement SQL Options Statements
[ https://issues.apache.org/jira/browse/DRILL-381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-381: -- Component/s: SQL Parser Query Planning Optimization Implement SQL Options Statements Key: DRILL-381 URL: https://issues.apache.org/jira/browse/DRILL-381 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization, SQL Parser Reporter: Jason Altekruse Assignee: Jason Altekruse Fix For: 0.4.0 Attachments: 0001-start-of-storage-system-for-drill-options.patch, Drill-381-Options-4-14-14.patch, Drill-381-Options-Mar-24-14.patch, Drill-381-options-3_30_14.patch We need to implement a system for passing and storing options at the global and session level. Some example options include, explain plan format, allowing quoted identifiers, null handling (concat null yields null). This involves creating a notion of a user session, as previously nothing was stored per user session. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-401) MockRecordReader is not releasing all buffers
[ https://issues.apache.org/jira/browse/DRILL-401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-401: -- Component/s: Execution - Relational Operators MockRecordReader is not releasing all buffers -- Key: DRILL-401 URL: https://issues.apache.org/jira/browse/DRILL-401 Project: Apache Drill Issue Type: Bug Components: Execution - Relational Operators Reporter: Jason Altekruse Assignee: Jacques Nadeau Fix For: 0.4.0 TestDistributedFragmentRun.oneBitOneExchangeTwoEntryRunLogical was running a basic scan/screen with a logical plan. It is logging an error about an outstanding buffer allocated when attempting to close the accountor, but this is not causing the test to fail. Stack trace gelow. java.lang.IllegalStateException Attempted to close accountor with 1 buffer(s) still allocatedfor QueryId: 8095683d-6d40-4ef2-9fea-eafe12d3af0d, MajorFragmentId: 0, MinorFragmentId: 0. Total 1 allocation(s) of byte size(s): 5000, at stack location: org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:96) org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.buffer(TopLevelAllocator.java:101) org.apache.drill.exec.vector.VarCharVector.allocateNew(VarCharVector.java:222) org.apache.drill.exec.vector.AllocationHelper.allocate(AllocationHelper.java:31) org.apache.drill.exec.store.mock.MockRecordReader.next(MockRecordReader.java:96) org.apache.drill.exec.physical.impl.ScanBatch.next(ScanBatch.java:94) org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:107) org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.next(ScreenCreator.java:80) org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:83) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) java.lang.Thread.run(Thread.java:722) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (DRILL-2544) Quitting sqlline results in a debug message Closing: org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection
[
https://issues.apache.org/jira/browse/DRILL-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504012#comment-14504012
]
Daniel Barclay (Drill) commented on DRILL-2544:
---
That Closing: ... line is coming from SQLLine, not Drill's JDBC driver.
Quitting sqlline results in a debug message Closing:
org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection
---
Key: DRILL-2544
URL: https://issues.apache.org/jira/browse/DRILL-2544
Project: Apache Drill
Issue Type: Bug
Reporter: Ramana Inukonda Nagaraj
Assignee: Daniel Barclay (Drill)
Fix For: 1.1.0
{code}
[root@atsqa6c57 ~]# /opt/drill/bin/sqlline -u jdbc:drill:zk=localhost:5181
/opt/drill/bin/drill-config.sh: line 94: [: =: unary operator expected
sqlline version 1.1.6
0: jdbc:drill:zk=localhost:5181 !q
Closing: org.apache.drill.jdbc.DrillJdbc41Factory$DrillJdbc41Connection
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2753) Implicit cast fails when comparing a double column and a varchar literal
[
https://issues.apache.org/jira/browse/DRILL-2753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504122#comment-14504122
]
Jinfeng Ni commented on DRILL-2753:
---
+1.
Implicit cast fails when comparing a double column and a varchar literal
Key: DRILL-2753
URL: https://issues.apache.org/jira/browse/DRILL-2753
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Reporter: Abhishek Girish
Assignee: Mehant Baid
Priority: Critical
Fix For: 1.0.0
Attachments: DRILL-2753.patch
Query fails when an implicit cast is used between a column of double data
type and a double literal.
*Drill:*
{code:sql}
select ss_customer_sk, ss_ticket_number from store_sales where ss_promo_sk
= '50' order by ss_promo_sk limit 1;
++--+
| ss_customer_sk | ss_ticket_number |
++--+
| 53792 | 44 |
++--+
1 row selected (1.045 seconds)
select ss_customer_sk, ss_ticket_number from store_sales where
ss_wholesale_cost = '38.19' order by ss_promo_sk limit 1;
Query failed: RemoteRpcException: Failure while running fragment., 38.19 [
d8f86a4f-226a-4e30-bb23-5b20ae5294e0 on abhi7.qa.lab:31010 ]
[ d8f86a4f-226a-4e30-bb23-5b20ae5294e0 on abhi7.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
*Postgres:*
{code:sql}
# select ss_customer_sk, ss_ticket_number from store_sales where
ss_wholesale_cost = '38.19' order by ss_promo_sk limit 1;
ss_customer_sk | ss_ticket_number
+--
44923 | 148425
(1 row)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2831) Line number column number are not part of Exception
Khurram Faraaz created DRILL-2831:
-
Summary: Line number column number are not part of Exception
Key: DRILL-2831
URL: https://issues.apache.org/jira/browse/DRILL-2831
Project: Apache Drill
Issue Type: Bug
Components: Execution - Flow
Affects Versions: 0.9.0
Environment: 64e3ec52b93e9331aa5179e040eca19afece8317 | DRILL-2611:
value vectors should report valid value count | 16.04.2015 @ 13:53:34 EDT
Reporter: Khurram Faraaz
Assignee: Chris Westin
Priority: Minor
In the Exception below, line number and column number are not reported.
{code}
0: jdbc:drill: eplain plan for select max( key1 ) maximum, count( key1 )
count_key1, sum( key1 ) sum_key1, min( key1 ) minimum, avg( key1 ) average,
key2 from `twoKeyJsn.json` group by key2 order by key2;
Query failed: SYSTEM ERROR: Failure parsing SQL. Non-query expression
encountered in illegal context
[351fe762-0c51-48a5-823e-1fd69b2ec00d on centos-02.qa.lab:31010]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-1832) Select * from json file failed with java.lang.IllegalArgumentException: null
[ https://issues.apache.org/jira/browse/DRILL-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503575#comment-14503575 ] Parth Chandra commented on DRILL-1832: -- This is fixed in master. Attaching patch with unit test for this one. Select * from json file failed with java.lang.IllegalArgumentException: null Key: DRILL-1832 URL: https://issues.apache.org/jira/browse/DRILL-1832 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types, Execution - Flow Affects Versions: 0.7.0 Reporter: Krystal Assignee: Parth Chandra Fix For: 1.0.0 Attachments: DRILL-1832.patch.diff, drill-1832.log, twitter_43.json git.commit.id.abbrev=201280e I have a json file that contains nested data. When I do a select * , the query failed as follows: 0: jdbc:drill:schema=dfs.root select * from `./json/twitter_43.json`; Query failed: Query stopped.[ cc231dec-8d28-4532-a938-c5f67592f262 on qa-node114.qa.lab:31010 ] Error: exception while executing query: Failure while executing query. (state=,code=0) Also, the following select fails: 0: jdbc:drill:schema=dfs.root select SUM(1) as `sum_Number_of_Records_ok` from `./json/twitter_43.json` having (COUNT(1) 0); Query failed: Query failed: Failure while running fragment., You tried to do a batch data read operation when you were in a state of STOP. You can only do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [ 364d911a-3bc8-4042-b5a2-6ed074c999ce on qa-node114.qa.lab:31010 ] [ 364d911a-3bc8-4042-b5a2-6ed074c999ce on qa-node114.qa.lab:31010 ] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1832) Select * from json file failed with java.lang.IllegalArgumentException: null
[ https://issues.apache.org/jira/browse/DRILL-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1832: - Attachment: DRILL-1832.patch.diff Select * from json file failed with java.lang.IllegalArgumentException: null Key: DRILL-1832 URL: https://issues.apache.org/jira/browse/DRILL-1832 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types, Execution - Flow Affects Versions: 0.7.0 Reporter: Krystal Assignee: Parth Chandra Fix For: 1.0.0 Attachments: DRILL-1832.patch.diff, drill-1832.log, twitter_43.json git.commit.id.abbrev=201280e I have a json file that contains nested data. When I do a select * , the query failed as follows: 0: jdbc:drill:schema=dfs.root select * from `./json/twitter_43.json`; Query failed: Query stopped.[ cc231dec-8d28-4532-a938-c5f67592f262 on qa-node114.qa.lab:31010 ] Error: exception while executing query: Failure while executing query. (state=,code=0) Also, the following select fails: 0: jdbc:drill:schema=dfs.root select SUM(1) as `sum_Number_of_Records_ok` from `./json/twitter_43.json` having (COUNT(1) 0); Query failed: Query failed: Failure while running fragment., You tried to do a batch data read operation when you were in a state of STOP. You can only do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [ 364d911a-3bc8-4042-b5a2-6ed074c999ce on qa-node114.qa.lab:31010 ] [ 364d911a-3bc8-4042-b5a2-6ed074c999ce on qa-node114.qa.lab:31010 ] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1647) Flatten operator cannot be used twice in a select clause
[ https://issues.apache.org/jira/browse/DRILL-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-1647: --- Component/s: Execution - Relational Operators Flatten operator cannot be used twice in a select clause Key: DRILL-1647 URL: https://issues.apache.org/jira/browse/DRILL-1647 Project: Apache Drill Issue Type: Bug Components: Execution - Relational Operators Reporter: Jason Altekruse Fix For: 0.7.0 Attachments: Drill-1647-multiple-flattens-complex-expression-rewriting.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-908) NullableValueVector bug causes all values pulled out to be reported as non-null
[ https://issues.apache.org/jira/browse/DRILL-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-908: -- Component/s: Execution - Data Types NullableValueVector bug causes all values pulled out to be reported as non-null --- Key: DRILL-908 URL: https://issues.apache.org/jira/browse/DRILL-908 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types, Execution - Relational Operators Reporter: Jason Altekruse Fix For: 0.4.0 Attachments: Drill-908-nullable-valuevec-bug.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-921) Building with an empty maven repo fails due to a missing snapshot jar
[ https://issues.apache.org/jira/browse/DRILL-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-921: -- Component/s: Tools, Build Test Building with an empty maven repo fails due to a missing snapshot jar - Key: DRILL-921 URL: https://issues.apache.org/jira/browse/DRILL-921 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Reporter: Jason Altekruse Priority: Critical Fix For: 0.4.0 Attachments: Drill-921-build-fails-missing-snapshot-jar.patch One of the referenced jars has been removed as a snapshot release. Haven't noticed as everyone has it in their local repos. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2834) Applying multiple flattens after a join results in an NPE
Rahul Challapalli created DRILL-2834:
Summary: Applying multiple flattens after a join results in an NPE
Key: DRILL-2834
URL: https://issues.apache.org/jira/browse/DRILL-2834
Project: Apache Drill
Issue Type: Bug
Components: Execution - Relational Operators
Reporter: Rahul Challapalli
Assignee: Chris Westin
Attachments: data.json, error.log
git.commit.id.abbrev=5cd36c5
The below query fails :
{code}
select flatten(t1.lst_lst), flatten(t1.events) from `data.json` t1 inner join
`data.json` t2 on t1.uid = t2.uid;
Query failed: SYSTEM ERROR: null
Fragment 0:0
[a27e431d-a598-4ec6-86b8-eb91d5d54161 on qa-node190.qa.lab:31010]
{code}
However if we only apply a single flatten things work as expected.
I attached the error logs and the data file. Let me know if you need anything
more
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2383) add exception and pause injections for testing drillbit stability
[
https://issues.apache.org/jira/browse/DRILL-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sudheesh Katkam updated DRILL-2383:
---
Attachment: DRILL-2383.9.patch.txt
add exception and pause injections for testing drillbit stability
-
Key: DRILL-2383
URL: https://issues.apache.org/jira/browse/DRILL-2383
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow
Reporter: Chris Westin
Assignee: Parth Chandra
Fix For: 0.9.0
Attachments: DRILL-2383.1.patch.txt, DRILL-2383.3.patch.txt,
DRILL-2383.6.patch.txt, DRILL-2383.7.patch.txt, DRILL-2383.8.patch.txt,
DRILL-2383.9.patch.txt
Use the exception injection mechanism to add exception injections to test a
variety of distributed failure scenarios.
Here are some scenarios we've worked out before:
1. Cancellation:
TC1: cancel before any result set is returned
TC2: cancel in the middle of fetching result set
TC3: cancel after all result set are produced but not all are fetched
TC4: cancel after everything is completed and fetched
As test setup, we need:
- query dataset large enough to be sent to different drillbits, e.g., TPCH
100
- queries that force multiple drillbits to work on them; e.g., count ...
group by
2. Completed (in each case check all drillbits are still up and running):
TC1: success
TC2: failed query - before query is executed - while sql parsing
TC3: failed query - before query is executed - while sending fragments to
other drillbits for execution
TC4: failed query - during query execution
It is currently not possible to create a scenario in which a query may hang.
To check all drillbits up and running and in a clean state, run:
-select count(*) from sys.drillbits;-
{code}
select count(*) from sys.memory;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-2511) Assert with full outer join when one of the join predicates is of a required type (nullabe parquet)
[
https://issues.apache.org/jira/browse/DRILL-2511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mehant Baid resolved DRILL-2511.
Resolution: Fixed
This should be fixed as part of DRILL-2707
Assert with full outer join when one of the join predicates is of a required
type (nullabe parquet)
---
Key: DRILL-2511
URL: https://issues.apache.org/jira/browse/DRILL-2511
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Affects Versions: 0.8.0
Reporter: Victoria Markman
Assignee: Mehant Baid
Fix For: 1.0.0
Attachments: j3.parquet.required, j4.parquet.required
Columns in tables j3 and j4 are created as 'required' data type:
{code}
[Fri Mar 20 11:30:42 root@~/parquet-tools-1.5.1-SNAPSHOT ] # ./parquet-schema
~/0_0_0.parquet
message root {
required binary c_varchar (UTF8);
required int32 c_integer;
required int64 c_bigint;
required float c_float;
required double c_double;
required int32 c_date (DATE);
required int32 c_time (TIME);
required int64 c_timestamp (TIMESTAMP);
required boolean c_boolean;
required double d9;
required double d18;
required double d28;
required double d38;
}
{code}
Full outer join on j3/j4 asserts.
This is happening with the join predicate of every SQL type except boolean.
{code}
select * from j3 full outer join j4 on (j3.c_varchar = j4.c_varchar);
java.lang.AssertionError at
org.apache.drill.exec.vector.VarCharVector$Accessor.get(VarCharVector.java:382)
at
org.apache.drill.exec.vector.VarCharVector$Accessor.getObject(VarCharVector.java:408)
at
org.apache.drill.exec.vector.accessor.VarCharAccessor.getObject(VarCharAccessor.java:98)
at
org.apache.drill.exec.vector.accessor.BoundCheckingAccessor.getObject(BoundCheckingAccessor.java:137)
at
org.apache.drill.jdbc.AvaticaDrillSqlAccessor.getObject(AvaticaDrillSqlAccessor.java:146)
at
net.hydromatic.avatica.AvaticaResultSet.getObject(AvaticaResultSet.java:351)
at sqlline.SqlLine$Rows$Row.init(SqlLine.java:2388)
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2504)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Same problem happens if you one table column types are optional and the other
one is required.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-2615) 'DESC' should be short for 'DESCRIBE'
[ https://issues.apache.org/jira/browse/DRILL-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman closed DRILL-2615. --- 'DESC' should be short for 'DESCRIBE' - Key: DRILL-2615 URL: https://issues.apache.org/jira/browse/DRILL-2615 Project: Apache Drill Issue Type: Bug Components: SQL Parser Reporter: Patrick Toole Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2615) 'DESC' should be short for 'DESCRIBE'
[ https://issues.apache.org/jira/browse/DRILL-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victoria Markman resolved DRILL-2615. - Resolution: Duplicate 'DESC' should be short for 'DESCRIBE' - Key: DRILL-2615 URL: https://issues.apache.org/jira/browse/DRILL-2615 Project: Apache Drill Issue Type: Bug Components: SQL Parser Reporter: Patrick Toole Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-901) Reading VarBinary data in parquet throws an error
[ https://issues.apache.org/jira/browse/DRILL-901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-901: -- Component/s: Storage - Parquet Reading VarBinary data in parquet throws an error - Key: DRILL-901 URL: https://issues.apache.org/jira/browse/DRILL-901 Project: Apache Drill Issue Type: Bug Components: Storage - Parquet Reporter: Jason Altekruse Fix For: 0.4.0 Attachments: 0001-Drill-901-Parquet-read-bug-with-VarBinary.patch When reading one of the VarBinary columns in the TPCH dataset, found an edge case where the parquet reader did not finish properly after reading all of the data. This caused an error as it tried to read a page beyond the last one in the column chunk. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-1737) Decimal type scale and precision are being flipped sometime during a cast
[ https://issues.apache.org/jira/browse/DRILL-1737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Altekruse updated DRILL-1737: --- Component/s: Execution - Data Types Decimal type scale and precision are being flipped sometime during a cast - Key: DRILL-1737 URL: https://issues.apache.org/jira/browse/DRILL-1737 Project: Apache Drill Issue Type: Bug Components: Execution - Data Types Reporter: Jason Altekruse Assignee: Jason Altekruse Fix For: 0.7.0 Running this query I was trying to test the behavior of the test framework to allow checking a baseline file, without providing type information for each column in a csv. The idea is that the types that come out of the test query can drive the interpretation of the baseline where strict type checking isn't necessary. Unfortunately I ran into a bug when I was trying to make sure decimal scale and precision could be retrieved from a result set with a decimal column. It appears that the two values are being switched somewhere in the process of running a cast. I plan on fixing this soon, but wanted to create a JIRA to track the issue so I can post the patch for the test framework for review. To replicate: select cast(dec_col as decimal(38,2)) dec_col from cp.`testframework/decimal_test.json` The file need only contain a single decimal in a field called dec_col in the input file. I used a value of 3.7 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (DRILL-2833) Exception on select * from INFORMATION_SCHEMA
Victoria Markman created DRILL-2833:
---
Summary: Exception on select * from INFORMATION_SCHEMA
Key: DRILL-2833
URL: https://issues.apache.org/jira/browse/DRILL-2833
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Affects Versions: 0.9.0
Reporter: Victoria Markman
Assignee: Daniel Barclay (Drill)
#Sat Apr 18 21:26:53 EDT 2015
git.commit.id.abbrev=9ec257e
{code}
0: jdbc:drill:schema=dfs select * from INFORMATION_SCHEMA.COLUMNS;
Query failed: SYSTEM ERROR: null
Fragment 0:0
[bd2a0477-90ea-423b-ad77-ad9784f4116b on atsqa4-133.qa.lab:31010]
(java.lang.NullPointerException) null
org.eigenbase.sql.type.IntervalSqlType.init():41
org.eigenbase.sql.type.SqlTypeFactoryImpl.createSqlIntervalType():104
org.apache.drill.exec.dotdrill.View.getRowType():236
org.apache.drill.exec.planner.logical.DrillViewTable.getRowType():46
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():123
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():97
org.apache.drill.exec.store.ischema.SelectedTable.getRecordReader():47
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():35
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():30
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan():127
org.apache.drill.exec.physical.base.AbstractSubScan.accept():39
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
org.apache.drill.exec.physical.config.IteratorValidator.accept():34
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitProject():77
org.apache.drill.exec.physical.config.Project.accept():51
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
org.apache.drill.exec.physical.config.IteratorValidator.accept():34
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():59
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitStore():132
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitScreen():195
org.apache.drill.exec.physical.config.Screen.accept():97
org.apache.drill.exec.physical.impl.ImplCreator.getExec():87
org.apache.drill.exec.work.fragment.FragmentExecutor.run():148
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1145
java.util.concurrent.ThreadPoolExecutor$Worker.run():615
java.lang.Thread.run():745
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-2268) Applying flatten after a join fails with IOBE
[
https://issues.apache.org/jira/browse/DRILL-2268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rahul Challapalli closed DRILL-2268.
Verified the fix and added the below testcases :
Functional/Passing/flatten_operators/2rows/hash-join/flatten_lstlst_DRILL-2268.q
Functional/Passing/flatten_operators/2rows/hash-join/join3_DRILL-2268.q
Functional/Passing/flatten_operators/2rows/hash-join/join7_DRILL-2268.q
Also raised the below 2 jira's
DRILL-2834
DRILL-2832
Applying flatten after a join fails with IOBE
--
Key: DRILL-2268
URL: https://issues.apache.org/jira/browse/DRILL-2268
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Reporter: Rahul Challapalli
Assignee: Parth Chandra
Priority: Critical
Fix For: 0.9.0
git.commit.id.abbrev=6676f2d
Data Set :
{code}
{
id : 1,
events : [
{ evnt_id:e1, campaign_id:c1, event_name:e1_name,
event_time:100, type : cmpgn9},
{ evnt_id:e2, campaign_id:c1, event_name:e2_name,
event_time:200, type : cmpgn4}
],
lst_lst : [[1,2],[3,4]],
lst : [1,2,3,4]
}
{
id : 2,
events : [
{ evnt_id:e1, campaign_id:c1, event_name:e1_name,
event_time:100, type : cmpgn9},
{ evnt_id:e2, campaign_id:c1, event_name:e2_name,
event_time:200, type : cmpgn4}
],
lst_lst : [[1,2],[3,4]],
lst : [1,2,3,4]
}
{code}
The below query which tries to flatten a repeated list fails :
{code}
select flatten(t1.lst_lst) from `temp.json` t1 inner join `temp.json` t2 on
t1.uid=t2.uid;
Query failed: RemoteRpcException: Failure while running fragment., index: -4,
length: 4 (expected: range(0, 16384)) [ e32cd0c6-a84a-4bbb-9812-bc0f7de17a68
on qa-node190.qa.lab:31010 ]
[ e32cd0c6-a84a-4bbb-9812-bc0f7de17a68 on qa-node190.qa.lab:31010 ]
{code}
The below query which tries to flatten a repeated map also fails
{code}
select flatten(t1.events) from `temp.json` t1 inner join `temp.json` t2 on
t1.uid=t2.uid;
Query failed: RemoteRpcException: Failure while running fragment., index: -4,
length: 4 (expected: range(0, 16384)) [ c9adffb0-44c6-4ecb-9093-1f3fd13837f9
on qa-node190.qa.lab:31010 ]
[ c9adffb0-44c6-4ecb-9093-1f3fd13837f9 on qa-node190.qa.lab:31010 ]
{code}
The below 2 queries return empty results. I can open a different JIRA if the
below 2 failures are because of a completely different reason
{code}
select flatten(t1.lst_lst[0]) from `temp.json` t1 inner join `temp.json` t2
on t1.uid=t2.uid;
++
| EXPR$0 |
++
++
{code}
{code}
select flatten(t1.lst) from `temp.json` t1 inner join `temp.json` t2 on
t1.uid=t2.uid;
++
| EXPR$0 |
++
++
{code}
Let me know if you have any questions
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2833) Exception on select * from INFORMATION_SCHEMA
[
https://issues.apache.org/jira/browse/DRILL-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504291#comment-14504291
]
Daniel Barclay (Drill) commented on DRILL-2833:
---
What does your data look like? (What are the data types of the columns in
tables from HBase?)
Also, what is the output of
select * from INFORMATION_SCHEMA.SCHEMATA;
and
select * from INFORMATION_SCHEMA.`TABLES`;
?
Exception on select * from INFORMATION_SCHEMA
-
Key: DRILL-2833
URL: https://issues.apache.org/jira/browse/DRILL-2833
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Affects Versions: 0.9.0
Reporter: Victoria Markman
Assignee: Daniel Barclay (Drill)
#Sat Apr 18 21:26:53 EDT 2015
git.commit.id.abbrev=9ec257e
{code}
0: jdbc:drill:schema=dfs select * from INFORMATION_SCHEMA.COLUMNS;
Query failed: SYSTEM ERROR: null
Fragment 0:0
[bd2a0477-90ea-423b-ad77-ad9784f4116b on atsqa4-133.qa.lab:31010]
(java.lang.NullPointerException) null
org.eigenbase.sql.type.IntervalSqlType.init():41
org.eigenbase.sql.type.SqlTypeFactoryImpl.createSqlIntervalType():104
org.apache.drill.exec.dotdrill.View.getRowType():236
org.apache.drill.exec.planner.logical.DrillViewTable.getRowType():46
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():123
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():97
org.apache.drill.exec.store.ischema.SelectedTable.getRecordReader():47
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():35
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():30
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan():127
org.apache.drill.exec.physical.base.AbstractSubScan.accept():39
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
org.apache.drill.exec.physical.config.IteratorValidator.accept():34
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitProject():77
org.apache.drill.exec.physical.config.Project.accept():51
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
org.apache.drill.exec.physical.config.IteratorValidator.accept():34
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():59
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitStore():132
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitScreen():195
org.apache.drill.exec.physical.config.Screen.accept():97
org.apache.drill.exec.physical.impl.ImplCreator.getExec():87
org.apache.drill.exec.work.fragment.FragmentExecutor.run():148
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1145
java.util.concurrent.ThreadPoolExecutor$Worker.run():615
java.lang.Thread.run():745
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Only happens when Hbase plugin is enabled. Thanks Rahul for helping diagnose
the problem !
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-2382) enhance exception injection to support node-specific injections
[ https://issues.apache.org/jira/browse/DRILL-2382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam resolved DRILL-2382. Resolution: Fixed Fix Version/s: (was: 0.9.0) 1.0.0 enhance exception injection to support node-specific injections --- Key: DRILL-2382 URL: https://issues.apache.org/jira/browse/DRILL-2382 Project: Apache Drill Issue Type: New Feature Components: Execution - Flow Reporter: Chris Westin Assignee: Sudheesh Katkam Fix For: 1.0.0 The exception injection mechanism we plan to use to test drillbit stability currently injects exceptions globally. Once we have the system tables to check on status, we'll want to be able to choose a node to cause the exception to happen on. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2384) QueryState is not making it back to the client
[ https://issues.apache.org/jira/browse/DRILL-2384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sudheesh Katkam resolved DRILL-2384. Resolution: Fixed Target Version/s: 1.0.0 (was: 0.9.0) QueryState is not making it back to the client -- Key: DRILL-2384 URL: https://issues.apache.org/jira/browse/DRILL-2384 Project: Apache Drill Issue Type: Bug Components: Execution - Flow Affects Versions: 0.7.0 Reporter: Chris Westin Assignee: Sudheesh Katkam Fix For: 0.9.0 In TestDrillbitResilience, I tried using the QueryState in messages from the server to determine whether a query had failed/was cancelled/completed normally, and it was always null. It looks like this is somehow not being propagated to the client from where it is set/sent in Foreman. We need the appropriate QueryState to be returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2383) add exception and pause injections for testing drillbit stability
[
https://issues.apache.org/jira/browse/DRILL-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sudheesh Katkam updated DRILL-2383:
---
Fix Version/s: (was: 0.9.0)
1.0.0
add exception and pause injections for testing drillbit stability
-
Key: DRILL-2383
URL: https://issues.apache.org/jira/browse/DRILL-2383
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow
Reporter: Chris Westin
Assignee: Parth Chandra
Fix For: 1.0.0
Attachments: DRILL-2383.1.patch.txt, DRILL-2383.3.patch.txt,
DRILL-2383.6.patch.txt, DRILL-2383.7.patch.txt, DRILL-2383.8.patch.txt,
DRILL-2383.9.patch.txt
Use the exception injection mechanism to add exception injections to test a
variety of distributed failure scenarios.
Here are some scenarios we've worked out before:
1. Cancellation:
TC1: cancel before any result set is returned
TC2: cancel in the middle of fetching result set
TC3: cancel after all result set are produced but not all are fetched
TC4: cancel after everything is completed and fetched
As test setup, we need:
- query dataset large enough to be sent to different drillbits, e.g., TPCH
100
- queries that force multiple drillbits to work on them; e.g., count ...
group by
2. Completed (in each case check all drillbits are still up and running):
TC1: success
TC2: failed query - before query is executed - while sql parsing
TC3: failed query - before query is executed - while sending fragments to
other drillbits for execution
TC4: failed query - during query execution
It is currently not possible to create a scenario in which a query may hang.
To check all drillbits up and running and in a clean state, run:
-select count(*) from sys.drillbits;-
{code}
select count(*) from sys.memory;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (DRILL-2835) IndexOutOfBoundsException in partition sender when doing streaming aggregate with LIMIT
Aman Sinha created DRILL-2835:
-
Summary: IndexOutOfBoundsException in partition sender when doing
streaming aggregate with LIMIT
Key: DRILL-2835
URL: https://issues.apache.org/jira/browse/DRILL-2835
Project: Apache Drill
Issue Type: Bug
Components: Execution - RPC
Affects Versions: 0.8.0
Reporter: Aman Sinha
Assignee: Jacques Nadeau
Following CTAS run on a TPC-DS 100GB scale factor on a 10-node cluster:
{code}
alter session set `planner.enable_hashagg` = false;
alter session set `planner.enable_multiphase_agg` = true;
create table dfs.tmp.stream9 as
select cr_call_center_sk , cr_catalog_page_sk , cr_item_sk , cr_reason_sk ,
cr_refunded_addr_sk , count(*) from catalog_returns_dri100
group by cr_call_center_sk , cr_catalog_page_sk , cr_item_sk , cr_reason_sk ,
cr_refunded_addr_sk
limit 100
;
{code}
{code}
Caused by: java.lang.IndexOutOfBoundsException: index: 1023, length: 1
(expected: range(0, 0))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:200)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:222)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.setByte(DrillBuf.java:621)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at
org.apache.drill.exec.vector.UInt1Vector$Mutator.set(UInt1Vector.java:342)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.vector.NullableBigIntVector$Mutator.set(NullableBigIntVector.java:372)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.vector.NullableBigIntVector.copyFrom(NullableBigIntVector.java:284)
~[drill-java-exec-0.9.0-SNAPSHOT-rebuffed.jar:0.9.0-SNAPSHOT]
at
org.apache.drill.exec.test.generated.PartitionerGen4$OutgoingRecordBatch.doEval(PartitionerTemplate.java:370)
~[na:na]
at
org.apache.drill.exec.test.generated.PartitionerGen4$OutgoingRecordBatch.copy(PartitionerTemplate.java:249)
~[na:na]
at
org.apache.drill.exec.test.generated.PartitionerGen4.doCopy(PartitionerTemplate.java:208)
~[na:na]
at
org.apache.drill.exec.test.generated.PartitionerGen4.partitionBatch(PartitionerTemplate.java:176)
~[na:na]
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2833) Exception on select * from INFORMATION_SCHEMA
[
https://issues.apache.org/jira/browse/DRILL-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2833:
Description:
#Sat Apr 18 21:26:53 EDT 2015
git.commit.id.abbrev=9ec257e
{code}
0: jdbc:drill:schema=dfs select * from INFORMATION_SCHEMA.COLUMNS;
Query failed: SYSTEM ERROR: null
Fragment 0:0
[bd2a0477-90ea-423b-ad77-ad9784f4116b on atsqa4-133.qa.lab:31010]
(java.lang.NullPointerException) null
org.eigenbase.sql.type.IntervalSqlType.init():41
org.eigenbase.sql.type.SqlTypeFactoryImpl.createSqlIntervalType():104
org.apache.drill.exec.dotdrill.View.getRowType():236
org.apache.drill.exec.planner.logical.DrillViewTable.getRowType():46
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():123
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():97
org.apache.drill.exec.store.ischema.SelectedTable.getRecordReader():47
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():35
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():30
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan():127
org.apache.drill.exec.physical.base.AbstractSubScan.accept():39
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
org.apache.drill.exec.physical.config.IteratorValidator.accept():34
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitProject():77
org.apache.drill.exec.physical.config.Project.accept():51
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
org.apache.drill.exec.physical.config.IteratorValidator.accept():34
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():59
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitStore():132
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitScreen():195
org.apache.drill.exec.physical.config.Screen.accept():97
org.apache.drill.exec.physical.impl.ImplCreator.getExec():87
org.apache.drill.exec.work.fragment.FragmentExecutor.run():148
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1145
java.util.concurrent.ThreadPoolExecutor$Worker.run():615
java.lang.Thread.run():745
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Only happens when Hbase plugin is enabled. Thanks Rahul for helping diagnose
the problem !
was:
#Sat Apr 18 21:26:53 EDT 2015
git.commit.id.abbrev=9ec257e
{code}
0: jdbc:drill:schema=dfs select * from INFORMATION_SCHEMA.COLUMNS;
Query failed: SYSTEM ERROR: null
Fragment 0:0
[bd2a0477-90ea-423b-ad77-ad9784f4116b on atsqa4-133.qa.lab:31010]
(java.lang.NullPointerException) null
org.eigenbase.sql.type.IntervalSqlType.init():41
org.eigenbase.sql.type.SqlTypeFactoryImpl.createSqlIntervalType():104
org.apache.drill.exec.dotdrill.View.getRowType():236
org.apache.drill.exec.planner.logical.DrillViewTable.getRowType():46
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():123
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():97
org.apache.drill.exec.store.ischema.SelectedTable.getRecordReader():47
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():35
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():30
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan():127
org.apache.drill.exec.physical.base.AbstractSubScan.accept():39
[jira] [Updated] (DRILL-2833) Exception on select * from INFORMATION_SCHEMA
[
https://issues.apache.org/jira/browse/DRILL-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Markman updated DRILL-2833:
Description:
#Sat Apr 18 21:26:53 EDT 2015
git.commit.id.abbrev=9ec257e
{code}
0: jdbc:drill:schema=dfs select * from INFORMATION_SCHEMA.COLUMNS;
Query failed: SYSTEM ERROR: null
Fragment 0:0
[bd2a0477-90ea-423b-ad77-ad9784f4116b on atsqa4-133.qa.lab:31010]
(java.lang.NullPointerException) null
org.eigenbase.sql.type.IntervalSqlType.init():41
org.eigenbase.sql.type.SqlTypeFactoryImpl.createSqlIntervalType():104
org.apache.drill.exec.dotdrill.View.getRowType():236
org.apache.drill.exec.planner.logical.DrillViewTable.getRowType():46
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():123
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():97
org.apache.drill.exec.store.ischema.SelectedTable.getRecordReader():47
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():35
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():30
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan():127
org.apache.drill.exec.physical.base.AbstractSubScan.accept():39
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
org.apache.drill.exec.physical.config.IteratorValidator.accept():34
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitProject():77
org.apache.drill.exec.physical.config.Project.accept():51
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitIteratorValidator():215
org.apache.drill.exec.physical.config.IteratorValidator.accept():34
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():59
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitStore():132
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitScreen():195
org.apache.drill.exec.physical.config.Screen.accept():97
org.apache.drill.exec.physical.impl.ImplCreator.getExec():87
org.apache.drill.exec.work.fragment.FragmentExecutor.run():148
org.apache.drill.common.SelfCleaningRunnable.run():38
java.util.concurrent.ThreadPoolExecutor.runWorker():1145
java.util.concurrent.ThreadPoolExecutor$Worker.run():615
java.lang.Thread.run():745
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Only happens when Hbase plugin is enabled.
was:
#Sat Apr 18 21:26:53 EDT 2015
git.commit.id.abbrev=9ec257e
{code}
0: jdbc:drill:schema=dfs select * from INFORMATION_SCHEMA.COLUMNS;
Query failed: SYSTEM ERROR: null
Fragment 0:0
[bd2a0477-90ea-423b-ad77-ad9784f4116b on atsqa4-133.qa.lab:31010]
(java.lang.NullPointerException) null
org.eigenbase.sql.type.IntervalSqlType.init():41
org.eigenbase.sql.type.SqlTypeFactoryImpl.createSqlIntervalType():104
org.apache.drill.exec.dotdrill.View.getRowType():236
org.apache.drill.exec.planner.logical.DrillViewTable.getRowType():46
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():123
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():109
org.apache.drill.exec.store.ischema.RecordGenerator.scanSchema():97
org.apache.drill.exec.store.ischema.SelectedTable.getRecordReader():47
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():35
org.apache.drill.exec.store.ischema.InfoSchemaBatchCreator.getBatch():30
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():62
org.apache.drill.exec.physical.impl.ImplCreator.visitOp():39
org.apache.drill.exec.physical.base.AbstractPhysicalVisitor.visitSubScan():127
org.apache.drill.exec.physical.base.AbstractSubScan.accept():39
org.apache.drill.exec.physical.impl.ImplCreator.getChildren():74
[jira] [Commented] (DRILL-2383) add exception and pause injections for testing drillbit stability
[
https://issues.apache.org/jira/browse/DRILL-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504379#comment-14504379
]
Sudheesh Katkam commented on DRILL-2383:
Usage: http://goo.gl/AsoHDY
add exception and pause injections for testing drillbit stability
-
Key: DRILL-2383
URL: https://issues.apache.org/jira/browse/DRILL-2383
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow
Reporter: Chris Westin
Assignee: Parth Chandra
Fix For: 1.0.0
Attachments: DRILL-2383.1.patch.txt, DRILL-2383.3.patch.txt,
DRILL-2383.6.patch.txt, DRILL-2383.7.patch.txt, DRILL-2383.8.patch.txt,
DRILL-2383.9.patch.txt
Use the exception injection mechanism to add exception injections to test a
variety of distributed failure scenarios.
Here are some scenarios we've worked out before:
1. Cancellation:
TC1: cancel before any result set is returned
TC2: cancel in the middle of fetching result set
TC3: cancel after all result set are produced but not all are fetched
TC4: cancel after everything is completed and fetched
As test setup, we need:
- query dataset large enough to be sent to different drillbits, e.g., TPCH
100
- queries that force multiple drillbits to work on them; e.g., count ...
group by
2. Completed (in each case check all drillbits are still up and running):
TC1: success
TC2: failed query - before query is executed - while sql parsing
TC3: failed query - before query is executed - while sending fragments to
other drillbits for execution
TC4: failed query - during query execution
It is currently not possible to create a scenario in which a query may hang.
To check all drillbits up and running and in a clean state, run:
-select count(*) from sys.drillbits;-
{code}
select count(*) from sys.memory;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2383) add exception and pause injections for testing drillbit stability
[
https://issues.apache.org/jira/browse/DRILL-2383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14504213#comment-14504213
]
Parth Chandra commented on DRILL-2383:
--
+1.
Merged in be8d953
add exception and pause injections for testing drillbit stability
-
Key: DRILL-2383
URL: https://issues.apache.org/jira/browse/DRILL-2383
Project: Apache Drill
Issue Type: New Feature
Components: Execution - Flow
Reporter: Chris Westin
Assignee: Parth Chandra
Fix For: 0.9.0
Attachments: DRILL-2383.1.patch.txt, DRILL-2383.3.patch.txt,
DRILL-2383.6.patch.txt, DRILL-2383.7.patch.txt, DRILL-2383.8.patch.txt,
DRILL-2383.9.patch.txt
Use the exception injection mechanism to add exception injections to test a
variety of distributed failure scenarios.
Here are some scenarios we've worked out before:
1. Cancellation:
TC1: cancel before any result set is returned
TC2: cancel in the middle of fetching result set
TC3: cancel after all result set are produced but not all are fetched
TC4: cancel after everything is completed and fetched
As test setup, we need:
- query dataset large enough to be sent to different drillbits, e.g., TPCH
100
- queries that force multiple drillbits to work on them; e.g., count ...
group by
2. Completed (in each case check all drillbits are still up and running):
TC1: success
TC2: failed query - before query is executed - while sql parsing
TC3: failed query - before query is executed - while sending fragments to
other drillbits for execution
TC4: failed query - during query execution
It is currently not possible to create a scenario in which a query may hang.
To check all drillbits up and running and in a clean state, run:
-select count(*) from sys.drillbits;-
{code}
select count(*) from sys.memory;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-2707) Projecting a required varchar column after a Full Outer Join results in an IOOBException
[
https://issues.apache.org/jira/browse/DRILL-2707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rahul Challapalli closed DRILL-2707.
Verified that the issue has been fixed. Below are the relevant tests in the
Functional suite :
Functional/Passing/cross-sources/parquet-json-fulljoin_DRILL-2707.q
Functional/Passing/cross-sources/parquet-hive-fulljoin_DRILL-2707.q
Thanks for the fix mehant
Projecting a required varchar column after a Full Outer Join results in an
IOOBException
-
Key: DRILL-2707
URL: https://issues.apache.org/jira/browse/DRILL-2707
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types, Execution - Relational Operators
Affects Versions: 0.8.0
Reporter: Rahul Challapalli
Assignee: Mehant Baid
Fix For: 0.9.0
Attachments: DRILL-2707.patch, fewtypes.parquet, fewtypes_null.json
git.commit.id.abbrev=a53e123
I tried to project a required varchar column after a FOJ. Below is what I see
{code}
0: jdbc:drill:schema=dfs_eea select
. . . . . . . . . . . . . . p.varchar_col
. . . . . . . . . . . . . . from dfs.`cross-sources`.`fewtypes.parquet` p
. . . . . . . . . . . . . . full outer join
dfs.`cross-sources`.`fewtypes_null.json` o
. . . . . . . . . . . . . . on p.int_col=o.int_col;
+-+
| varchar_col |
+-+
java.lang.IndexOutOfBoundsException: index: 180, length: 10 (expected:
range(0, 180))
at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1143)
at
io.netty.buffer.PooledUnsafeDirectByteBuf.getBytes(PooledUnsafeDirectByteBuf.java:136)
at io.netty.buffer.WrappedByteBuf.getBytes(WrappedByteBuf.java:289)
at
io.netty.buffer.UnsafeDirectLittleEndian.getBytes(UnsafeDirectLittleEndian.java:25)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at io.netty.buffer.DrillBuf.getBytes(DrillBuf.java:596)
at
org.apache.drill.exec.vector.VarCharVector$Accessor.get(VarCharVector.java:387)
at
org.apache.drill.exec.vector.VarCharVector$Accessor.getObject(VarCharVector.java:411)
at
org.apache.drill.exec.vector.accessor.VarCharAccessor.getObject(VarCharAccessor.java:108)
at
org.apache.drill.exec.vector.accessor.BoundCheckingAccessor.getObject(BoundCheckingAccessor.java:137)
at
org.apache.drill.jdbc.AvaticaDrillSqlAccessor.getObject(AvaticaDrillSqlAccessor.java:165)
at
net.hydromatic.avatica.AvaticaResultSet.getObject(AvaticaResultSet.java:351)
at sqlline.SqlLine$Rows$Row.init(SqlLine.java:2388)
at sqlline.SqlLine$IncrementalRows.hasNext(SqlLine.java:2504)
at sqlline.SqlLine$TableOutputFormat.print(SqlLine.java:2148)
at sqlline.SqlLine.print(SqlLine.java:1809)
at sqlline.SqlLine$Commands.execute(SqlLine.java:3766)
at sqlline.SqlLine$Commands.sql(SqlLine.java:3663)
at sqlline.SqlLine.dispatch(SqlLine.java:889)
at sqlline.SqlLine.begin(SqlLine.java:763)
at sqlline.SqlLine.start(SqlLine.java:498)
at sqlline.SqlLine.main(SqlLine.java:460)
{code}
Not sure if this is a client-specific issue as there is no exception from the
drillbit log files
However if I project a varchar column (nullable) from a json file after a
FOJ, there seems to be no issues
{code}
0: jdbc:drill:schema=dfs_eea select
. . . . . . . . . . . . . . o.varchar_col
. . . . . . . . . . . . . . from dfs.`cross-sources`.`fewtypes.parquet` p
. . . . . . . . . . . . . . full outer join
dfs.`cross-sources`.`fewtypes_null.json` o
. . . . . . . . . . . . . . on p.int_col=o.int_col;
+-+
| varchar_col |
+-+
| jllkjsdhfg |
| null|
| gfdstweopiu |
| gjklhsdfgkjhkASDF |
| oieoiutriotureWERTgwgEWRg |
| gjkdfkjglfd |
| ioerutklsdfASDgerGWEr |
| lkjgfiurtoUYFHfahui |
| IOUfiuodsfIUfjkh |
| iweuoHUIhUwer |
| null|
| dfgoiuert |
| uitreo |
| uigoMnvjjkdf |
| NvvdfHVG|
| null|
| null|
| uiuikjk |
| null|
| hjiwgh |
| null|
| jhgduitweriuoert |
| KfijUIwre |
| Nhkhuivb|
| null|
| null|
+-+
26 rows selected (0.212 seconds)
{code}
I attached the parquet and json files used. Let me know if you need anything
more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2649) Math and Trig page seems to refer to types that are not Drill SQL types
[
https://issues.apache.org/jira/browse/DRILL-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502937#comment-14502937
]
Kristine Hahn commented on DRILL-2649:
--
Thanks, fixed and ready for review:
http://tshiran.github.io/drill/docs/math-and-trig/
{quote}
The page lists SMALLINT but not TINYINT. Is that inconsistency intended?
{quote}
Yes, this is intended. I understand from [~seanhychu] smallint might be
supported soon but afaik, not tinyint.
Math and Trig page seems to refer to types that are not Drill SQL types
---
Key: DRILL-2649
URL: https://issues.apache.org/jira/browse/DRILL-2649
Project: Apache Drill
Issue Type: Bug
Components: Documentation
Reporter: Daniel Barclay (Drill)
Assignee: Kristine Hahn
The Math and Trig page at http://drill.apache.org/docs/math-and-trig/ refers
to data types UINT1, UINT2, UINT4, UINT8, FLOAT4, FLOAT8, DECIMAL9, and
DECIMAL18.
However, these do not seem to be Drill SQL types, at least based on their
behavior in CAST expressions. For example:
0: jdbc:drill:zk=local SELECT CAST( 1 as UINT4 ) FROM
INFORMATION_SCHEMA.CATALOGS;
Mar 31, 2015 9:47:51 PM org.eigenbase.sql.validate.SqlValidatorException
init
SEVERE: org.eigenbase.sql.validate.SqlValidatorException: Unknown datatype
name 'UINT4'
Mar 31, 2015 9:47:51 PM org.eigenbase.util.EigenbaseException init
SEVERE: org.eigenbase.util.EigenbaseContextException: From line 1, column 19
to line 1, column 23: Unknown datatype name 'UINT4'
Query failed: SqlValidatorException: Unknown datatype name 'UINT4'
Error: exception while executing query: Failure while executing query.
(state=,code=0)
0: jdbc:drill:zk=local
Those names seem to refer to Drill _internal_ types (or types used in the
storage plugin interface), not Drill _SQL_ types.
Since the Math and Trig page deals with the SQL interface, it should be in
terms of Drill SQL types, not Drill internal/non-SQL types.
Additionally:
- The page says INT (a syntax-only shortcut) rather than INTEGER (the
actual name of the type).
- The page lists SMALLINT but not TINYINT. Is that inconsistency intended?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2644) Data Types page should list and describe all data typse
[
https://issues.apache.org/jira/browse/DRILL-2644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502947#comment-14502947
]
Kristine Hahn commented on DRILL-2644:
--
{quote}
Given its title and position in the page hierarchy, Data Types page at
http://drill.apache.org/docs/data-types/ should start with a listing and
description of all Drill Data Types rather than starting off talking about
casting and conversions.
{quote}
Fixed, ready for review:
http://tshiran.github.io/drill/docs/supported-data-types/
{quote}
The casting and conversions content should probably be a separate page.
(Posslbly the Data Types page should just be an introduction/downlinks page,
with its first child page being the page defining all data types, and some
later child (or deeper) page being the casting/conversion page.)
{quote}
Sorry, we're limited to 3 levels of topic hierarchy in our new doc tool.
Data Types page should list and describe all data typse
---
Key: DRILL-2644
URL: https://issues.apache.org/jira/browse/DRILL-2644
Project: Apache Drill
Issue Type: Bug
Components: Documentation
Reporter: Daniel Barclay (Drill)
Assignee: Kristine Hahn
Given its title and position in the page hierarchy, Data Types page at
http://drill.apache.org/docs/data-types/ should start with a listing and
description of all Drill Data Types rather than starting off talking about
casting and conversions.
The casting and conversions content should probably be a separate page.
(Posslbly the Data Types page should just be an introduction/downlinks page,
with its first child page being the page defining all data types, and some
later child (or deeper) page being the casting/conversion page.)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-2292) CTAS broken when we have repeated maps
[
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14502970#comment-14502970
]
Deneche A. Hakim commented on DRILL-2292:
-
[~hgunes] could you please take a look at the patch ? Thanks
CTAS broken when we have repeated maps
--
Key: DRILL-2292
URL: https://issues.apache.org/jira/browse/DRILL-2292
Project: Apache Drill
Issue Type: Bug
Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Steven Phillips
Priority: Blocker
Fix For: 1.0.0
Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt
git.commit.id.abbrev=6676f2d
Data Set :
{code}
{
id : 1,
map:{rm: [
{mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a:
foo},{b:boo}]},
{mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a:
bar},{c:1},{d:4.5}]}
]}
}
{code}
The below CTAS statement fails though the underlying query succeeds
{code}
create table rep_map as select d.map from `temp.json` d;
Query failed: Query stopped., index: -4, length: 4 (expected: range(0,
16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
{code}
Error from the logs :
{code}
Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4
(expected: range(0, 16384))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at
org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
... 19 common frames omitted
2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR
o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error
7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
java.io.IOException: Failed to initialize FieldWriters.
at
org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
[jira] [Assigned] (DRILL-2645) SQL Functions page should specify return types in terms of SQL types
[ https://issues.apache.org/jira/browse/DRILL-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn reassigned DRILL-2645: Assignee: Kristine Hahn SQL Functions page should specify return types in terms of SQL types Key: DRILL-2645 URL: https://issues.apache.org/jira/browse/DRILL-2645 Project: Apache Drill Issue Type: Bug Components: Documentation Reporter: Daniel Barclay (Drill) Assignee: Kristine Hahn On the SQL Functions page currently at http://drill.apache.org/docs/sql-functions/, the return type for string functions is specified in terms of int, text, and byte array instead of being specified in terms of Drill SQL types (e.g., INTEGER, BYTE ARRAY, etc.) Additionally, names of SQL types should be given in all uppercase, as they are used in the SQL standard, as they are specified to appear in INFORMATION_SCHEMA.COLUMNS, and as they appear in the JDBC equivalent of INFORMATION_SCHEMA.COLUMNS. (Another reason is that it's not clear, for example, which occurrence of numeric is the adjective numeric and which means the proper noun (datatype name) NUMERIC.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (DRILL-2649) Math and Trig page seems to refer to types that are not Drill SQL types
[ https://issues.apache.org/jira/browse/DRILL-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn reassigned DRILL-2649: Assignee: Kristine Hahn Math and Trig page seems to refer to types that are not Drill SQL types --- Key: DRILL-2649 URL: https://issues.apache.org/jira/browse/DRILL-2649 Project: Apache Drill Issue Type: Bug Components: Documentation Reporter: Daniel Barclay (Drill) Assignee: Kristine Hahn The Math and Trig page at http://drill.apache.org/docs/math-and-trig/ refers to data types UINT1, UINT2, UINT4, UINT8, FLOAT4, FLOAT8, DECIMAL9, and DECIMAL18. However, these do not seem to be Drill SQL types, at least based on their behavior in CAST expressions. For example: 0: jdbc:drill:zk=local SELECT CAST( 1 as UINT4 ) FROM INFORMATION_SCHEMA.CATALOGS; Mar 31, 2015 9:47:51 PM org.eigenbase.sql.validate.SqlValidatorException init SEVERE: org.eigenbase.sql.validate.SqlValidatorException: Unknown datatype name 'UINT4' Mar 31, 2015 9:47:51 PM org.eigenbase.util.EigenbaseException init SEVERE: org.eigenbase.util.EigenbaseContextException: From line 1, column 19 to line 1, column 23: Unknown datatype name 'UINT4' Query failed: SqlValidatorException: Unknown datatype name 'UINT4' Error: exception while executing query: Failure while executing query. (state=,code=0) 0: jdbc:drill:zk=local Those names seem to refer to Drill _internal_ types (or types used in the storage plugin interface), not Drill _SQL_ types. Since the Math and Trig page deals with the SQL interface, it should be in terms of Drill SQL types, not Drill internal/non-SQL types. Additionally: - The page says INT (a syntax-only shortcut) rather than INTEGER (the actual name of the type). - The page lists SMALLINT but not TINYINT. Is that inconsistency intended? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-2773) Hive version number doc bug
[ https://issues.apache.org/jira/browse/DRILL-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn closed DRILL-2773. Hive version number doc bug --- Key: DRILL-2773 URL: https://issues.apache.org/jira/browse/DRILL-2773 Project: Apache Drill Issue Type: Task Components: Documentation Reporter: Kristine Hahn Assignee: Kristine Hahn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (DRILL-2773) Hive version number doc bug
[ https://issues.apache.org/jira/browse/DRILL-2773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn resolved DRILL-2773. -- Resolution: Fixed Hive version number doc bug --- Key: DRILL-2773 URL: https://issues.apache.org/jira/browse/DRILL-2773 Project: Apache Drill Issue Type: Task Components: Documentation Reporter: Kristine Hahn Assignee: Kristine Hahn -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-461) Build failing
[
https://issues.apache.org/jira/browse/DRILL-461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra closed DRILL-461.
---
Build failing
-
Key: DRILL-461
URL: https://issues.apache.org/jira/browse/DRILL-461
Project: Apache Drill
Issue Type: Bug
Affects Versions: 0.4.0
Reporter: Bhallamudi Venkata Siva Kamesh
Priority: Blocker
Fix For: 0.4.0
While executing the following goals on the latest code , getting the
following exception
{code:xml}
mvn clean install -DskipTests
{code}
+*Exception*+
{noformat}
org.apache.maven.wagon.ResourceDoesNotExistException: File:
http://apache-drill.s3.amazonaws.com/files//sf-0.01_tpc-h_parquet.tgz does
not exist
at
org.apache.maven.wagon.shared.http.AbstractHttpClientWagon.fillInputData(AbstractHttpClientWagon.java:603)
at
org.apache.maven.wagon.StreamWagon.getInputStream(StreamWagon.java:116)
at org.apache.maven.wagon.StreamWagon.getIfNewer(StreamWagon.java:88)
at org.apache.maven.wagon.StreamWagon.get(StreamWagon.java:61)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-7) Query parser for Drill
[ https://issues.apache.org/jira/browse/DRILL-7?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra closed DRILL-7. - Query parser for Drill -- Key: DRILL-7 URL: https://issues.apache.org/jira/browse/DRILL-7 Project: Apache Drill Issue Type: Bug Reporter: Ted Dunning Priority: Blocker Fix For: 0.4.0 This is an umbrella issue for adaptation of the OpenDremel parser for use in Drill -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-366) Casting to BigInt from Varchar produces wrong results
[
https://issues.apache.org/jira/browse/DRILL-366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-366:
Component/s: Functions - Drill
Casting to BigInt from Varchar produces wrong results
-
Key: DRILL-366
URL: https://issues.apache.org/jira/browse/DRILL-366
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Reporter: Mehant Baid
Assignee: Mehant Baid
Priority: Blocker
Fix For: 0.4.0
Attachments: DRILL-366.patch
Can be reproduced using the following plan:
{
head:{
type:APACHE_DRILL_PHYSICAL,
version:1,
generator:{
type:manual
}
},
graph:[
{
@id:1,
pop:json-scan,
entries: [
{
path : /tmp/scan4.json
}
],
storageengine: {
type: json,
dfsName: file:///
}
},
{
pop:project,
@id:2,
child: 1,
exprs: [ {
ref: int,
expr: cast(integer as bigint)
} ]
},
{
@id: 3,
child: 2,
pop: screen
}
]
}
Input file, /tmp/scan4.json:
{
integer : 2008
}
{
integer : 2007
}
{
integer : 2006
}
Output:
2008
2008
2008
From the above output we notice that it always generates values corresponding
to the first row. Looks like the buffer is not getting reset to the next row.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2738) Offset with casting a column to timestamp not working
[ https://issues.apache.org/jira/browse/DRILL-2738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2738: - Component/s: Execution - Relational Operators Offset with casting a column to timestamp not working - Key: DRILL-2738 URL: https://issues.apache.org/jira/browse/DRILL-2738 Project: Apache Drill Issue Type: Bug Components: Execution - Relational Operators Affects Versions: 0.7.0 Reporter: Venkata krishnan Sowrirajan In the below query, it should skip the first row which is a header and want to cast one of the column to timestamp. But it is trying to parse the first row to cast it to timestamp. Without casting it to timestamp, simple offset query works fine. select cast(columns[0] as timestamp) from `guts-csv/CSV/guts_run_lab-app002.csv` offset 1; So I did explain plan on the above query explain plan without implementation for select cast(columns[0] as timestamp) from `guts-csv/CSV/guts_run_lab-app002.csv` offset 1; DrillScreenRel DrillLimitRel(offset=[1]) DrillProjectRel(EXPR$0=[CAST(ITEM($0, 0)):TIMESTAMP(0)]) DrillScanRel(table=[[fs, drill, guts-csv/CSV/guts_run_lab-app002.csv]], groupscan=[EasyGroupScan [selectionRoot=/mapr/yarn-test/drill/guts-csv/CSV/guts_run_lab-app002.csv, numFiles=1, columns=[`columns`[0]], files=[file:/mapr/yarn-test/drill/guts-csv/CSV/guts_run_lab-app002.csv]]]) In the plan, it looks like it tries to do casting to timestamp and then the offset operation which is why its failing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-611) TPCH: query 05 verification fails
[ https://issues.apache.org/jira/browse/DRILL-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-611: Component/s: Query Planning Optimization TPCH: query 05 verification fails - Key: DRILL-611 URL: https://issues.apache.org/jira/browse/DRILL-611 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Reporter: Zhiyong Liu Priority: Blocker Fix For: 0.4.0 1. Set up drill test framework (see https://github.com/zhiyongliu/incubator-drill/tree/master/testing) and generate TCP H dataset with a scale factor of about 500MB. 2. Query file (modified from original by drill dev, and further modified as follows): select n.n_name, sum(l.l_extendedprice * (1 - l.l_discount)) as revenue from customer c, orders o, lineitem l, supplier s, nation n, region r where c.c_custkey = o.o_custkey and l.l_orderkey = o.o_orderkey and l.l_suppkey = s.s_suppkey and c.c_nationkey = s.s_nationkey and s.s_nationkey = n.n_nationkey and n.n_regionkey = r.r_regionkey and r.r_name = 'EUROPE' and o.o_orderdate = date '1997-01-01' and o.o_orderdate date '1997-01-01' + interval '1' year group by n.n_name order by revenue desc 3. Drill build: git.commit.id=1e0e4dfc96c8e7eb83659784dfbb85cd83d7d243 4. The baseline/expected results are generated via Oracle DB. 5. Run the above query in the test framework, with verification turned on. 6. Verification fails: 2014-04-30 14:48:28 INFO TestVerifier:206 - These rows are expected but are not in result set: 2014-04-30 14:48:28 INFO TestVerifier:209 -ROMANIA 23412645.1 : 1 time(s). 2014-04-30 14:48:28 INFO TestVerifier:209 -GERMANY 29054886.4 : 1 time(s). 2014-04-30 14:48:28 INFO TestVerifier:209 -RUSSIA 26653618.4 : 1 time(s). 2014-04-30 14:48:28 INFO TestVerifier:209 -UNITED KINGDOM 23903443.3 : 1 time(s). 2014-04-30 14:48:28 INFO TestVerifier:209 -FRANCE 24893950.8 : 1 time(s). 2014-04-30 14:48:28 INFO TestVerifier:216 - Total number of expected but missing: 5 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-609) TPCH: query 01 verification fails
[ https://issues.apache.org/jira/browse/DRILL-609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-609: Component/s: Query Planning Optimization TPCH: query 01 verification fails - Key: DRILL-609 URL: https://issues.apache.org/jira/browse/DRILL-609 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Reporter: Zhiyong Liu Priority: Blocker Fix For: 0.4.0 1. Set up drill test framework (see https://github.com/zhiyongliu/incubator-drill/tree/master/testing) and generate TCP H dataset with a scale factor of about 500MB. 2. Query file (modified from original by drill dev, and further modified as follows): select l_returnflag, l_linestatus, sum(l_quantity) as sum_qty, sum(l_extendedprice) as sum_base_price, sum(l_extendedprice * (1 - l_discount)) as sum_disc_price, sum(l_extendedprice * (1 - l_discount) * (1 + l_tax)) as sum_charge, avg(l_quantity) as avg_qty, avg(l_extendedprice) as avg_price, avg(l_discount) as avg_disc, count(*) as count_order from lineitem where l_shipdate = date '1998-12-01' - interval '120' day (3) group by l_returnflag, l_linestatus order by l_returnflag, l_linestatus 3. Drill build: git.commit.id=1e0e4dfc96c8e7eb83659784dfbb85cd83d7d243 4. The baseline/expected results are generated via Oracle DB. 5. Run the above query in the test framework, with verification turned on. 6. Verification fails: 2014-04-30 14:27:24 INFO TestVerifier:192 - These rows are not expected: 2014-04-30 14:27:24 INFO TestVerifier:195 -N F 499596.0 7.23782156171E8 6.87811474102097E8 7.152735291765119E8 25.562627916496112 37032.99309250921 0.0 19544 : 1 time(s). 2014-04-30 14:27:24 INFO TestVerifier:195 -N O 3.6357561E7 5.271068187394886E105.007347520096913E105.2078457678915245E10 25.50090690001515 36970.38027096180.0 1425736 : 1 time(s). 2014-04-30 14:27:24 INFO TestVerifier:195 -A F 1.8865717E7 2.7356549949989704E10 2.598835690045046E102.7027321931296932E10 25.519179575692974 37004.03879606534 0.0 739276 : 1 time(s). 2014-04-30 14:27:24 INFO TestVerifier:195 -R F 1.8872497E7 2.734543103391961E102.5979800081985153E10 2.7018232810784813E10 25.5184443786 36974.643676062755 0.0 739563 : 1 time(s). 2014-04-30 14:27:24 INFO TestVerifier:202 - Total number of unexpected rows: 4 2014-04-30 14:27:24 INFO TestVerifier:206 - These rows are expected but are not in result set: 2014-04-30
[jira] [Updated] (DRILL-615) TPCH: query 18 fails with NullPointerException
[
https://issues.apache.org/jira/browse/DRILL-615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-615:
Component/s: Query Planning Optimization
TPCH: query 18 fails with NullPointerException
--
Key: DRILL-615
URL: https://issues.apache.org/jira/browse/DRILL-615
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Zhiyong Liu
Priority: Blocker
Fix For: 0.4.0
1. Create TCP H dataset with a scale factor of about 500MB.
2. Run the following query in sqlline:
select
c.c_name,
c.c_custkey,
o.o_orderkey,
o.o_orderdate,
o.o_totalprice,
sum(l.l_quantity)
from
customer c,
orders o,
lineitem l
where
o.o_orderkey in (
select
l_orderkey
from
lineitem
group by
l_orderkey having
sum(l_quantity) 300
)
and c.c_custkey = o.o_custkey
and o.o_orderkey = l.l_orderkey
group by
c.c_name,
c.c_custkey,
o.o_orderkey,
o.o_orderdate,
o.o_totalprice
order by
o.o_totalprice desc,
o.o_orderdate;
The following exception is encountered:
Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
running query.[error_id: f14b3f75-3d47-46cf-9af4-6ff80a34ca6b
endpoint {
address: perfnode104.perf.lab
user_port: 31010
control_port: 31011
data_port: 31012
}
error_type: 0
message: Failure while running fragment. NullPointerException:[ Schema is
currently null. You must call buildSchema(SelectionVectorMode) before this
container can return a schema. ]
]
Error: exception while executing query (state=,code=0)
In contrast, if this data source is used:
http://apache-drill.s3.amazonaws.com/files/sf-0.01_tpc-h_parquet.tgz
then query went through successfully.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-610) TPCH: query 03 verification fails
[ https://issues.apache.org/jira/browse/DRILL-610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-610: Component/s: Query Planning Optimization TPCH: query 03 verification fails - Key: DRILL-610 URL: https://issues.apache.org/jira/browse/DRILL-610 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Reporter: Zhiyong Liu Priority: Blocker Fix For: 0.4.0 1. Set up drill test framework (see https://github.com/zhiyongliu/incubator-drill/tree/master/testing) and generate TCP H dataset with a scale factor of about 500MB. 2. Query file (modified from original by drill dev, and further modified as follows): select l.l_orderkey, sum(l.l_extendedprice * (1 - l.l_discount)) as revenue, o.o_orderdate, o.o_shippriority from customer c, orders o, lineitem l where c.c_mktsegment = 'HOUSEHOLD' and c.c_custkey = o.o_custkey and l.l_orderkey = o.o_orderkey and o.o_orderdate date '1995-03-25' and l.l_shipdate date '1995-03-25' group by l.l_orderkey, o.o_orderdate, o.o_shippriority order by revenue desc, o.o_orderdate 3. Drill build: git.commit.id=1e0e4dfc96c8e7eb83659784dfbb85cd83d7d243 4. The baseline/expected results are generated via Oracle DB. 5. Run the above query in the test framework, with verification turned on. 6. Verification fails: 2014-04-30 14:42:21 INFO TestVerifier:192 - These rows are not expected: 2014-04-30 14:42:21 INFO TestVerifier:195 -0 32217.8247 1995-03-17 2474884 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:195 -0 21021.273 1995-02-10 2299361 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:195 -0 49270.9205 1995-02-18 2627745 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:195 -0 160321.524 1995-02-10 1828645 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:195 -0 195687.9626997 1995-03-16 799905 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:195 -0 125110.5975 1995-01-02 1510471 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:195 -0 179264.4125003 1995-02-10 194212 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:195 -0 48017.97 1994-12-15 521921 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:195 -0 102168.9547999 1995-02-11 2576902 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:195 -0 154239.8077 1995-02-20 263687 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:202 - Total number of unexpected rows: 5853 2014-04-30 14:42:21 INFO TestVerifier:206 - These rows are expected but are not in result set: 2014-04-30 14:42:21 INFO TestVerifier:209 -241477 11991.627 21-MAR-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:209 -191553 119125.349 07-FEB-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:209 -253728 283348.33 26-MAR-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:209 -110823 112230.552 28-JAN-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:209 -380135 122101.606 09-MAR-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:209 -385542 70513.9196 04-MAR-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:209 -21956 281118.193 02-FEB-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:209 -43491 138577.744 09-MAR-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:209 -234887 126183.41 01-MAR-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:209 -198406 42139.1435 23-FEB-95 0 : 1 time(s). 2014-04-30 14:42:21 INFO TestVerifier:216 - Total number of expected but missing: 788 -- This message was sent
[jira] [Closed] (DRILL-1245) Drill should pinpoint to the Problem Record when it fails to parse a json file
[
https://issues.apache.org/jira/browse/DRILL-1245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deneche A. Hakim closed DRILL-1245.
---
Resolution: Fixed
This was fixed by DRILL-2675. The same query now returns the following error
message:
{noformat}
0: jdbc:drill:zk=local select * from dfs.tmp.`t.json`;
Query failed: DATA_READ ERROR: Error parsing JSON. - Invalid numeric value:
Leading zeroes not allowed
Filename /tmp/t.json
Record 4
Column 24
Fragment 0:0
[245cba11-e428-4cbf-9d3b-6ca165b864bc on abdel-11.qa.lab:31010]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{noformat}
Drill should pinpoint to the Problem Record when it fails to parse a json
file
Key: DRILL-1245
URL: https://issues.apache.org/jira/browse/DRILL-1245
Project: Apache Drill
Issue Type: Bug
Components: Storage - JSON
Reporter: Rahul Challapalli
Assignee: Deneche A. Hakim
Fix For: 1.0.0
Attachments: DRILL-1245.1.patch.txt
git.commit.id.abbrev=98b208e
Data :
{code}
{name:name1, id:1}
{name:name2, id:2}
{name:name3, id:3}
{name:name4, id:04}
{name:name5, id:5}
{code}
Query :
{code}
select * from cp.`file.json`;
Query failed: Screen received stop request sent. Invalid numeric value:
Leading zeroes not allowed
at [Source: java.io.BufferedReader@202fbdb4; line: 1, column: 24]
[c11a17bd-1a3a-4eed-a848-6d79225399d3]
Error: exception while executing query: Failure while trying to get next
result batch. (state=,code=0)
{code}
The msg should point to the exact record which is causing the problem as it
will be hard looking into the data and finding out the problem record.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-2685) Unique-ify local Hive metastore directory or unit test fails
[ https://issues.apache.org/jira/browse/DRILL-2685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-2685: - Component/s: Storage - Hive Unique-ify local Hive metastore directory or unit test fails Key: DRILL-2685 URL: https://issues.apache.org/jira/browse/DRILL-2685 Project: Apache Drill Issue Type: Bug Components: Tools, Build Test Reporter: Hanifi Gunes Assignee: Hanifi Gunes Priority: Blocker Fix For: 0.9.0 Attachments: DRILL-2685-1.patch Hive test suites subclasses HiveTestBase that in turn generates data. When tests are run in a concurrent setting, one removes the common directory while the other is working on it, failing unit-tests. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2359) ClassPathFileSystem.getFileStatus() should throw FileNotFoundException when path doesn't exist
[
https://issues.apache.org/jira/browse/DRILL-2359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-2359:
-
Component/s: Execution - Flow
ClassPathFileSystem.getFileStatus() should throw FileNotFoundException when
path doesn't exist
--
Key: DRILL-2359
URL: https://issues.apache.org/jira/browse/DRILL-2359
Project: Apache Drill
Issue Type: Bug
Components: Execution - Flow
Affects Versions: 0.8.0
Reporter: Deneche A. Hakim
Assignee: Steven Phillips
Priority: Blocker
Fix For: 0.8.0
Attachments: DRILL-2359.1.patch.txt
{{ClassPathFileSystem.getFileStatus(Path args0)}} throws {{IOException}} when
the path doesn't exist. It should throw {{FileNotFoundException}} instead.
One particular method that fails to work correctly because of this is
{{org.apache.hadoop.FileSystem.exists(Path f)}}:
{code}
public boolean exists(Path f) throws IOException {
try {
return getFileStatus(f) != null;
} catch (FileNotFoundException e) {
return false;
}
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-1339) Apache Drill lists dead/killed queries as 'Running Queries' in the UI
[ https://issues.apache.org/jira/browse/DRILL-1339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-1339: - Component/s: Client - HTTP Apache Drill lists dead/killed queries as 'Running Queries' in the UI - Key: DRILL-1339 URL: https://issues.apache.org/jira/browse/DRILL-1339 Project: Apache Drill Issue Type: Bug Components: Client - HTTP Affects Versions: 0.4.0 Environment: MapR v3.1 Reporter: Michael England Assignee: Jinfeng Ni Priority: Blocker Labels: apache, dead, dissapearing, drill, killed, not, profile, queries, running, ui Fix For: 0.7.0 The Apache Drill UI lists queries that are dead/have been killed as 'Running Queries' on the 'Profiles' page. When grepping for the Query GUID of a dead query that is listed as currently running in the drill directory, I found the following outputs in a drillbit.log: drillbit.out:11:56:58.852 [1d1d4a6d-c6e6-4f7a-87ba-35c9cb772109:frag:0:0] ERROR o.a.h.h.z.RecoverableZooKeeper - ZooKeeper exists failed after 3 retries drillbit.out:11:56:58.853 [1d1d4a6d-c6e6-4f7a-87ba-35c9cb772109:frag:0:0] ERROR o.a.h.h.zookeeper.ZooKeeperWatcher - hconnection Received unexpected KeeperException, re-throwing exception drillbit.out:11:57:15.359 [1d1d4a6d-c6e6-4f7a-87ba-35c9cb772109:frag:0:0] ERROR o.a.h.h.z.RecoverableZooKeeper - ZooKeeper exists failed after 3 retries drillbit.out:11:57:15.360 [1d1d4a6d-c6e6-4f7a-87ba-35c9cb772109:frag:0:0] ERROR o.a.h.h.zookeeper.ZooKeeperWatcher - hconnection Received unexpected KeeperException, re-throwing exception drillbit.out:11:57:31.866 [1d1d4a6d-c6e6-4f7a-87ba-35c9cb772109:frag:0:0] ERROR o.a.h.h.z.RecoverableZooKeeper - ZooKeeper exists failed after 3 retries drillbit.out:11:57:31.866 [1d1d4a6d-c6e6-4f7a-87ba-35c9cb772109:frag:0:0] ERROR o.a.h.h.zookeeper.ZooKeeperWatcher - hconnection Received unexpected KeeperException, re-throwing exception drillbit.out:11:57:48.373 [1d1d4a6d-c6e6-4f7a-87ba-35c9cb772109:frag:0:0] ERROR o.a.h.h.z.RecoverableZooKeeper - ZooKeeper exists failed after 3 retries drillbit.out:11:57:48.374 [1d1d4a6d-c6e6-4f7a-87ba-35c9cb772109:frag:0:0] ERROR o.a.h.h.zookeeper.ZooKeeperWatcher - hconnection Received unexpected KeeperException, re-throwing exception sqlline.log:2014-08-20 11:58:12,966 [SIGINT handler] DEBUG o.a.drill.exec.client.DrillClient - Cancelling query 1d1d4a6d-c6e6-4f7a-87ba-35c9cb772109 There needs to be some logic which moves these queries from the 'Running Queries' section to 'Failed Queries' or something to that effect. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-613) TPCH: query 10 fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/DRILL-613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-613:
Component/s: Query Planning Optimization
TPCH: query 10 fails with IndexOutOfBoundsException
---
Key: DRILL-613
URL: https://issues.apache.org/jira/browse/DRILL-613
Project: Apache Drill
Issue Type: Bug
Components: Query Planning Optimization
Reporter: Zhiyong Liu
Priority: Blocker
Fix For: 0.4.0
1. Create TCP H dataset with a scale factor of about 500MB.
2. Run the following query in sqlline:
select
c.c_custkey,
c.c_name,
sum(l.l_extendedprice * (1 - l.l_discount)) as revenue,
c.c_acctbal,
n.n_name,
c.c_address,
c.c_phone,
c.c_comment
from
customer c,
orders o,
lineitem l,
nation n
where
c.c_custkey = o.o_custkey
and l.l_orderkey = o.o_orderkey
and o.o_orderdate = date '1994-03-01'
and o.o_orderdate date '1994-03-01' + interval '3' month
and l.l_returnflag = 'R'
and c.c_nationkey = n.n_nationkey
group by
c.c_custkey,
c.c_name,
c.c_acctbal,
c.c_phone,
n.n_name,
c.c_address,
c.c_comment
order by
revenue desc;
The following exception is encountered:
Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
running query.[error_id: 31f98f15-fc8b-44f2-b4fc-d63475b51862
endpoint {
address: perfnode104.perf.lab
user_port: 31010
control_port: 31011
data_port: 31012
}
error_type: 0
message: Failure while running fragment. NullPointerException:[ Schema is
currently null. You must call buildSchema(SelectionVectorMode) before this
container can return a schema. ]
]
Error: exception while executing query (state=,code=0)
In contrast, if this data source is used:
http://apache-drill.s3.amazonaws.com/files/sf-0.01_tpc-h_parquet.tgz
then query went through successfully.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-623) Unable to run query containing quoted schema name
[
https://issues.apache.org/jira/browse/DRILL-623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Parth Chandra updated DRILL-623:
Component/s: Storage - Hive
Unable to run query containing quoted schema name
-
Key: DRILL-623
URL: https://issues.apache.org/jira/browse/DRILL-623
Project: Apache Drill
Issue Type: Bug
Components: Storage - Hive
Reporter: George Chow
Assignee: Venki Korukanti
Priority: Blocker
Fix For: 0.4.0
Attachments: DRILL-623-1.patch
This is particularly for the Hive storage-engine where the schema name is
composed of a root followed by the storage-engine's own schema name. E.g.,
configuring Hive leads to a schema name hivestg.default:
0: jdbc:drill:schema=hivestg select * from INFORMATION_SCHEMA.SCHEMATA;
+--+-+--+
| CATALOG_NAME | SCHEMA_NAME | SCHEMA_OWNER |
+--+-+--+
| DRILL| hivestg.default | owner |
| DRILL| hivestg | owner |
| DRILL| dfs.default | owner |
| DRILL| dfs | owner |
| DRILL| cp.default | owner |
| DRILL| cp | owner |
| DRILL| INFORMATION_SCHEMA | owner |
+--+-+--+
7 rows selected (0.131 seconds)
Given tables inside hivestg.default as follows:
0: jdbc:drill:schema=hivestg select * from INFORMATION_SCHEMA.`TABLES`;
+---+--+++
| TABLE_CATALOG | TABLE_SCHEMA | TABLE_NAME | TABLE_TYPE |
+---+--+++
| DRILL | hivestg.default | sneaky | TABLE |
| DRILL | hivestg.default | bit_table | TABLE |
| DRILL | hivestg.default | stinyint_table | TABLE |
| DRILL | hivestg.default | no_null_integer_table | TABLE |
| DRILL | hivestg.default | real_table | TABLE |
| DRILL | hivestg.default | integer_table | TABLE |
A generated query would look as follows:
SELECT * FROM `hivestg.default`.`integer_table`
This fails to execute with the following error:
0: jdbc:drill:schema=hivestg SELECT * FROM `hivestg.default`.`integer_table`;
Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
running query.[error_id: fe7731f0-c032-4049-9204-61bb7c7340cb
endpoint {
address: localhost
user_port: 31010
control_port: 31011
data_port: 31012
}
error_type: 0
message: Failure while parsing sql. ValidationException:[
org.eigenbase.util.EigenbaseContextException: From line 1, column 15 to line
1, column 47 ] EigenbaseContextException:[ From line 1, column 15 to line
1, column 47 ] SqlValidatorException:[ Table
\'hivestg.default.integer_table\' not found ]
]
Error: exception while executing query (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (DRILL-617) TPCH: query 14 verification fails
[ https://issues.apache.org/jira/browse/DRILL-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Parth Chandra updated DRILL-617: Component/s: Query Planning Optimization TPCH: query 14 verification fails - Key: DRILL-617 URL: https://issues.apache.org/jira/browse/DRILL-617 Project: Apache Drill Issue Type: Bug Components: Query Planning Optimization Reporter: Zhiyong Liu Priority: Blocker Fix For: 0.4.0 1. Set up drill test framework (see https://github.com/zhiyongliu/incubator-drill/tree/master/testing) and generate TCP H dataset with a scale factor of about 500MB. 2. Query file (modified from original by drill dev, and further modified as follows): select 100.00 * sum(case when p.p_type like 'PROMO%' then l.l_extendedprice * (1 - l.l_discount) else 0 end) / sum(l.l_extendedprice * (1 - l.l_discount)) as promo_revenue from lineitem l, part p where l.l_partkey = p.p_partkey and l.l_shipdate = date '1994-08-01' and l.l_shipdate date '1994-08-01' + interval '1' month; 3. Drill build: git.commit.id=1e0e4dfc96c8e7eb83659784dfbb85cd83d7d243 4. The baseline/expected results are generated via Oracle DB. 5. Run the above query in the test framework, with verification turned on. 6. Verification fails: 2014-05-01 10:50:32 INFO TestVerifier:192 - These rows are not expected: 2014-05-01 10:50:32 INFO TestVerifier:195 -16.718056324303113 : 1 time(s). 2014-05-01 10:50:32 INFO TestVerifier:202 - Total number of unexpected rows: 1 2014-05-01 10:50:32 INFO TestVerifier:206 - These rows are expected but are not in result set: 2014-05-01 10:50:32 INFO TestVerifier:209 -16.5451633 : 1 time(s). 2014-05-01 10:50:32 INFO TestVerifier:216 - Total number of expected but missing: 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2522) Implement INFORMATION_SCHEMA.* enough for relevant tools [umbrella/tracking bug]
[ https://issues.apache.org/jira/browse/DRILL-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Barclay (Drill) updated DRILL-2522: -- Description: See DRILL-2763 regarding INFORMATION_SCHEMA.COLUMNS (as opposed to other INFORMATION_SCHEMA.* tables/views ). Notes (to be edited/triaged/etc): - INFORMATION_SCHEMA.INFORMATION_SCHEMA_CATALOG_NAME does not exist. - Short-name views of existing tables (e.g., TABLES_S for TABLES) do not exist. - (Most standard-specified tables/views do not exist, but many probably aren't relevant for Drill.) was: See DRILL-2763 regarding INFORMATION_SCHEMA.COLUMNS (as opposed to other INFORMATION_SCHEMA.* tables/views ). Notes (to be edited/triaged/etc): - INFORMATION_SCHEMA.INFORMATION_SCHEMA_CATALOG_NAME does not exist. - Short-name views of existing tables (e.g., TABLES_S for TABLES) do not exist. - (Most standard-specified tables/views do not exist.) Implement INFORMATION_SCHEMA.* enough for relevant tools [umbrella/tracking bug] Key: DRILL-2522 URL: https://issues.apache.org/jira/browse/DRILL-2522 Project: Apache Drill Issue Type: Bug Components: Storage - Information Schema Reporter: Daniel Barclay (Drill) Assignee: Daniel Barclay (Drill) Priority: Critical Fix For: 1.0.0 See DRILL-2763 regarding INFORMATION_SCHEMA.COLUMNS (as opposed to other INFORMATION_SCHEMA.* tables/views ). Notes (to be edited/triaged/etc): - INFORMATION_SCHEMA.INFORMATION_SCHEMA_CATALOG_NAME does not exist. - Short-name views of existing tables (e.g., TABLES_S for TABLES) do not exist. - (Most standard-specified tables/views do not exist, but many probably aren't relevant for Drill.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2763) Implement INFORMATION_SCHEMA.COLUMNS enough for relevant tools [umbrella/tracking bug]
[
https://issues.apache.org/jira/browse/DRILL-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Barclay (Drill) updated DRILL-2763:
--
Description:
[TBD: intro.]
A. {{COLUMNS}} columns existing in Drill that are not compliant with standard
SQL:
1. *TDB*: {{TABLE_NAME}} holds the original form of the identifier but _might_
need to be uppercased here to be compliantt (or possibly internal matching
queries (e.g., for JDBC's getColumns(...)) could be case-insensitive).
2. *TDB*: {{COLUMN_NAME}} holds the original form of the identifier but _might_
need to be uppercased here to be compliant (or possibly internal matching
queries (e.g., for JDBC's getColumns(...)) could be case-insensitive).
3. {{ORDINAL_POSITION}} values are zero-based rather than being one-based.
2. {{CHARACTER_MAXIMUM_LENGTH}}, {{NUMERIC_PRECISION_RADIX}},
{{NUMERIC_SCALE}}, and {{NUMERIC_PRECISION}} use {{-1}} instead of {{NULL}} for
the not-applicable case.
3. {{NUMERIC_PRECISION}} for non-{{DECIMAL}}/{{NUMERIC}} exact numeric types
(e.g., {{INTEGER}}) and approximate numeric types (e.g., {{DOUBLE}}) is {{-1}}
(logical null) instead of the specified values.
3. {{NUMERIC_SCALE}} for non-{{DECIMAL}}/{{NUMERIC}} exact numeric types is
{{-1}} instead of {{0}}.
4. {{NUMERIC_SCALE}} for approximate numeric types is {{-1}} instead of the
number of bits of precision (24 and 53(?)).
5. {{NUMERIC_PRECISION_RADIX}} for integral exact numeric types is {{-1}}
instead of {{10}}.
6. {{NUMERIC_PRECISION_RADIX}} for approximate exact numeric types is {{-1}}
instead of {{2}}.
7. {{CHARACTER_MAXIMUM_LENGTH}} for types {{CHAR}}, {{BINARY}}, and {{VAR
BINARY}} is {{-1}} instead of the corresponding length.
8. {{DATA_TYPE}} values for {{INTERVAL}} with {{YEAR}} and/or {{MONTH}} and for
{{INTERVAL}} with {{DAY}}, {{HOUR}}, {{MINUTE}}, and/or {{SECOND}} are
{{INTERVAL_YEAR_MONTH}} and {{INTERVAL_DATA_TIME}}, respectively, instead
of the data type name {{INTERVAL}}.
9. {{DATA_TYPE}} values for non-atomic types seem to be type descriptors
({{data type}} syntax; e.g., {{VARCHAR(65536) ARRAY}}) instead of just data
type names (e.g., {{ARRAY}}).
B. Standard {{COLUMNS}} columns that don't exist in Drill and that probably
are more relevant:
1. {{CHARACTER_OCTET_LENGTH}} does not exist. (Drill's JDBC driver needs to
return this, and currently tries to compute it itself.)
2. {{DATETIME_PRECISION}} does not exist. (Drill's JDBC driver probably needs
it to compute its {{getColumns()}}'s {{COLUMN_SIZE}} correctly.)
3. {{INTERVAL_TYPE}} does not exist. (Drill's JDBC driver needs this to compute
its {{getColumns()}}'s {{COLUMN_SIZE}} correctly once {{COLUMNS.DATA_TYPE}} is
correct.}
4. {{INTERVAL_PRECISION}} does not exist. (Drill's JDBC driver needs this to
compute its {{getColumns()}}'s {{COLUMN_SIZE}} correctly.)
[TBD]
5. {{MAXIMUM_CARDINALITY}} does not exist. (This might be relevant for JDBC's
{{getColumns()}}'s {{COLUMN_SIZE}}.)
C. Standard {{COLUMNS}} columns that don't exist in Drill but are less likely
to be relevant (listed for completeness):
- {{COLUMN_DEFAULT}}
- {{CHARACTER_SET_CATALOG}}
- {{CHARACTER_SET_SCHEMA}}
- {{CHARACTER_SET_NAME}}
- {{COLLATION_CATALOG}}
- {{COLLATION_SCHEMA}}
- {{COLLATION_NAME}}
- {{DOMAIN_CATALOG}}
- {{DOMAIN_SCHEMA}}
- {{DOMAIN_NAME}}
- {{UDT_CATALOG}}
- {{UDT_SCHEMA}}
- {{UDT_NAME}}
- {{SCOPE_CATALOG}}
- {{SCOPE_SCHEMA}}
- {{SCOPE_NAME}}
- {{DTD_IDENTIFIER}}
- {{IS_SELF_REFERENCING}}
- {{IS_IDENTITY}}
- {{IDENTITY_GENERATION}}
- {{IDENTITY_START}}
- {{IDENTITY_INCREMENT}}
- {{IDENTITY_MAXIMUM}}
- {{IDENTITY_MINIMUM}}
- {{IDENTITY_CYCLE}}
- {{IS_GENERATED}}
- {{GENERATION_EXPRESSION}}
- {{IS_SYSTEM_TIME_PERIOD_START}}
- {{IS_SYSTEM_TIME_PERIOD_END}}
- {{SYSTEM_TIME_PERIOD_TIMESTAMP_GENERATION}}
- {{IS_UPDATABLE}}
- {{DECLARED_DATA_TYPE}}
- {{DECLARED_NUMERIC_PRECISION}}
- {{DECLARED_NUMERIC_SCALE}}
was:
[TBD: intro.]
A. {{COLUMNS}} columns existing in Drill that are not compliant with standard
SQL:
1. *TDB*: {{TABLE_NAME}} holds the original form of the identifier but _might_
need to be uppercased here to be compliant.
2. *TDB*: {{COLUMN_NAME}} holds the original form of the identifier but _might_
need to be uppercased here to be compliant.
3. {{ORDINAL_POSITION}} values are zero-based rather than being one-based.
2. {{CHARACTER_MAXIMUM_LENGTH}}, {{NUMERIC_PRECISION_RADIX}},
{{NUMERIC_SCALE}}, and {{NUMERIC_PRECISION}} use {{-1}} instead of {{NULL}} for
the not-applicable case.
3. {{NUMERIC_PRECISION}} for non-{{DECIMAL}}/{{NUMERIC}} exact numeric types
(e.g., {{INTEGER}}) and approximate numeric types (e.g., {{DOUBLE}}) is {{-1}}
(logical null) instead of the specified values.
3. {{NUMERIC_SCALE}} for non-{{DECIMAL}}/{{NUMERIC}} exact numeric types is
{{-1}} instead of {{0}}.
4. {{NUMERIC_SCALE}} for approximate numeric types is {{-1}} instead of the
number of bits of precision (24 and
[jira] [Updated] (DRILL-2763) Implement INFORMATION_SCHEMA.COLUMNS enough for relevant tools [umbrella/tracking bug]
[
https://issues.apache.org/jira/browse/DRILL-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Barclay (Drill) updated DRILL-2763:
--
Description:
[TBD: intro.]
A. {{COLUMNS}} columns existing in Drill that are not compliant with standard
SQL:
1. *TDB*: {{TABLE_NAME}} holds the original form of the identifier but _might_
need to be uppercased here to be compliantt (or possibly internal matching
queries (e.g., for JDBC's getColumns(...)) could be case-insensitive).
2. *TDB*: {{COLUMN_NAME}} holds the original form of the identifier but _might_
need to be uppercased here to be compliant (or possibly internal matching
queries (e.g., for JDBC's getColumns(...)) could be case-insensitive).
3. {{ORDINAL_POSITION}} values are zero-based rather than being one-based.
4. {{CHARACTER_MAXIMUM_LENGTH}}, {{NUMERIC_PRECISION_RADIX}},
{{NUMERIC_SCALE}}, and {{NUMERIC_PRECISION}} use {{-1}} instead of {{NULL}} for
the not-applicable case.
5. {{NUMERIC_PRECISION}} for non-{{DECIMAL}}/{{NUMERIC}} exact numeric types
(e.g., {{INTEGER}}) and approximate numeric types (e.g., {{DOUBLE}}) is {{-1}}
(logical null) instead of the specified values.
6. {{NUMERIC_SCALE}} for non-{{DECIMAL}}/{{NUMERIC}} exact numeric types is
{{-1}} instead of {{0}}.
7. {{NUMERIC_SCALE}} for approximate numeric types is {{-1}} instead of the
number of bits of precision (24 and 53(?)).
8. {{NUMERIC_PRECISION_RADIX}} for integral exact numeric types is {{-1}}
instead of {{10}}.
9. {{NUMERIC_PRECISION_RADIX}} for approximate exact numeric types is {{-1}}
instead of {{2}}.
10. {{CHARACTER_MAXIMUM_LENGTH}} for types {{CHAR}}, {{BINARY}}, and {{VAR
BINARY}} is {{-1}} instead of the corresponding length.
11. {{DATA_TYPE}} values for {{INTERVAL}} with {{YEAR}} and/or {{MONTH}} and
for {{INTERVAL}} with {{DAY}}, {{HOUR}}, {{MINUTE}}, and/or {{SECOND}} are
{{INTERVAL_YEAR_MONTH}} and {{INTERVAL_DATA_TIME}}, respectively, instead
of the data type name {{INTERVAL}}.
12. {{DATA_TYPE}} values for non-atomic types seem to be type descriptors
({{data type}} syntax; e.g., {{VARCHAR(65536) ARRAY}}) instead of just data
type names (e.g., {{ARRAY}}).
B. Standard {{COLUMNS}} columns that don't exist in Drill and that probably
are more relevant:
1. {{CHARACTER_OCTET_LENGTH}} does not exist. (Drill's JDBC driver needs to
return this, and currently tries to compute it itself.)
2. {{DATETIME_PRECISION}} does not exist. (Drill's JDBC driver probably needs
it to compute its {{getColumns()}}'s {{COLUMN_SIZE}} correctly.)
3. {{INTERVAL_TYPE}} does not exist. (Drill's JDBC driver needs this to compute
its {{getColumns()}}'s {{COLUMN_SIZE}} correctly once {{COLUMNS.DATA_TYPE}} is
correct.}
4. {{INTERVAL_PRECISION}} does not exist. (Drill's JDBC driver needs this to
compute its {{getColumns()}}'s {{COLUMN_SIZE}} correctly.)
[TBD]
5. {{MAXIMUM_CARDINALITY}} does not exist. (This might be relevant for JDBC's
{{getColumns()}}'s {{COLUMN_SIZE}}.)
C. Standard {{COLUMNS}} columns that don't exist in Drill but are less likely
to be relevant (listed for completeness):
- {{COLUMN_DEFAULT}}
- {{CHARACTER_SET_CATALOG}}
- {{CHARACTER_SET_SCHEMA}}
- {{CHARACTER_SET_NAME}}
- {{COLLATION_CATALOG}}
- {{COLLATION_SCHEMA}}
- {{COLLATION_NAME}}
- {{DOMAIN_CATALOG}}
- {{DOMAIN_SCHEMA}}
- {{DOMAIN_NAME}}
- {{UDT_CATALOG}}
- {{UDT_SCHEMA}}
- {{UDT_NAME}}
- {{SCOPE_CATALOG}}
- {{SCOPE_SCHEMA}}
- {{SCOPE_NAME}}
- {{DTD_IDENTIFIER}}
- {{IS_SELF_REFERENCING}}
- {{IS_IDENTITY}}
- {{IDENTITY_GENERATION}}
- {{IDENTITY_START}}
- {{IDENTITY_INCREMENT}}
- {{IDENTITY_MAXIMUM}}
- {{IDENTITY_MINIMUM}}
- {{IDENTITY_CYCLE}}
- {{IS_GENERATED}}
- {{GENERATION_EXPRESSION}}
- {{IS_SYSTEM_TIME_PERIOD_START}}
- {{IS_SYSTEM_TIME_PERIOD_END}}
- {{SYSTEM_TIME_PERIOD_TIMESTAMP_GENERATION}}
- {{IS_UPDATABLE}}
- {{DECLARED_DATA_TYPE}}
- {{DECLARED_NUMERIC_PRECISION}}
- {{DECLARED_NUMERIC_SCALE}}
was:
[TBD: intro.]
A. {{COLUMNS}} columns existing in Drill that are not compliant with standard
SQL:
1. *TDB*: {{TABLE_NAME}} holds the original form of the identifier but _might_
need to be uppercased here to be compliantt (or possibly internal matching
queries (e.g., for JDBC's getColumns(...)) could be case-insensitive).
2. *TDB*: {{COLUMN_NAME}} holds the original form of the identifier but _might_
need to be uppercased here to be compliant (or possibly internal matching
queries (e.g., for JDBC's getColumns(...)) could be case-insensitive).
3. {{ORDINAL_POSITION}} values are zero-based rather than being one-based.
2. {{CHARACTER_MAXIMUM_LENGTH}}, {{NUMERIC_PRECISION_RADIX}},
{{NUMERIC_SCALE}}, and {{NUMERIC_PRECISION}} use {{-1}} instead of {{NULL}} for
the not-applicable case.
3. {{NUMERIC_PRECISION}} for non-{{DECIMAL}}/{{NUMERIC}} exact numeric types
(e.g., {{INTEGER}}) and approximate numeric types (e.g., {{DOUBLE}}) is {{-1}}
(logical null) instead of the specified values.
3.
[jira] [Commented] (DRILL-2619) Unsupported implicit casts should throw a proper error message
[
https://issues.apache.org/jira/browse/DRILL-2619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503060#comment-14503060
]
Deneche A. Hakim commented on DRILL-2619:
-
[~rkins] could you attach the file {{fewtypes_null.tbl}} ? Thanks
Unsupported implicit casts should throw a proper error message
--
Key: DRILL-2619
URL: https://issues.apache.org/jira/browse/DRILL-2619
Project: Apache Drill
Issue Type: Bug
Components: Execution - Data Types
Reporter: Rahul Challapalli
Assignee: Deneche A. Hakim
Priority: Minor
Labels: error_message_must_fix
Fix For: 1.0.0
git.commit.id.abbrev=c11fcf7
When I have a where clause with an implicit cast, I get back a wierd message
which does not indicate the problem
{code}
select columns[9] from `fewtypes_null.tbl` where columns[0] = 6;
Query failed: RemoteRpcException: Failure while running fragment., null [
0faf63c5-cdca-4b5b-a2ab-5f3ef02d5c9b on qa-node191.qa.lab:31010 ]
[ 0faf63c5-cdca-4b5b-a2ab-5f3ef02d5c9b on qa-node191.qa.lab:31010 ]
{code}
Attached the stacktrace. Let me know if you need anything more.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (DRILL-1891) Error message does not get propagated correctly when reading from JSON file
[
https://issues.apache.org/jira/browse/DRILL-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14503055#comment-14503055
]
Deneche A. Hakim commented on DRILL-1891:
-
This was fixed by DRILL-2675. Here is the error message I get when running the
2nd query:
{noformat}
0: jdbc:drill:zk=local select a1 from dfs.tmp.`test.json` where a1 is not null
group by a1;
Query failed: DATA_READ ERROR: Error parsing JSON. - Unexpected character (':'
(code 58)): expected a valid value (number, String, array, object, 'true',
'false' or 'null')
Filename /tmp/test.json
Record 4
Column 9
Fragment 0:0
[d502c2a9-34f8-4363-a6a3-ed1a3151dd71 on abdel-11.qa.lab:31010]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{noformat}
Error message does not get propagated correctly when reading from JSON file
---
Key: DRILL-1891
URL: https://issues.apache.org/jira/browse/DRILL-1891
Project: Apache Drill
Issue Type: Bug
Components: Storage - JSON
Affects Versions: 0.7.0
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
Fix For: 1.0.0
I made a mistake in t.json file (extra colon in the last row):
{code}
{ a1: 0 , b1: a}
{ a1: 1 , b1: b}
{ a1: 2 , b1: c}
{ a1:: 3 , b1: c}
{code}
Error message below pretty much tells me everything that went wrong.
{code}
0: jdbc:drill:schema=dfs select a1 from `t.json` where a1 is not null;
Query failed: Query stopped., Unexpected character (':' (code 58)): expected
a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: org.apache.drill.exec.vector.complex.fn.JsonReader@53c10ede;
line: 3, column: 9] [ 64182782-ebba-4c6a-a963-005b8cb48339 on
atsqa4-133.qa.lab:31010 ]
{code}
However, if a result of query above is an input to any other operator, I get
this error message:
{code}
0: jdbc:drill:schema=dfs select a1 from `t.json` where a1 is not null group
by a1;
Query failed: Query failed: Failure while running fragment., You tried to do
a batch data read operation when you were in a state of STOP. You can only
do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [
955aac65-5e43-4430-baf6-ed6bb8a020d9 on atsqa4-133.qa.lab:31010 ]
[ 955aac65-5e43-4430-baf6-ed6bb8a020d9 on atsqa4-133.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Very painful for the user if query is really complex.
The same behavior if file does not exist.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-1891) Error message does not get propagated correctly when reading from JSON file
[
https://issues.apache.org/jira/browse/DRILL-1891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deneche A. Hakim closed DRILL-1891.
---
Resolution: Fixed
Error message does not get propagated correctly when reading from JSON file
---
Key: DRILL-1891
URL: https://issues.apache.org/jira/browse/DRILL-1891
Project: Apache Drill
Issue Type: Bug
Components: Storage - JSON
Affects Versions: 0.7.0
Reporter: Victoria Markman
Assignee: Deneche A. Hakim
Fix For: 1.0.0
I made a mistake in t.json file (extra colon in the last row):
{code}
{ a1: 0 , b1: a}
{ a1: 1 , b1: b}
{ a1: 2 , b1: c}
{ a1:: 3 , b1: c}
{code}
Error message below pretty much tells me everything that went wrong.
{code}
0: jdbc:drill:schema=dfs select a1 from `t.json` where a1 is not null;
Query failed: Query stopped., Unexpected character (':' (code 58)): expected
a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: org.apache.drill.exec.vector.complex.fn.JsonReader@53c10ede;
line: 3, column: 9] [ 64182782-ebba-4c6a-a963-005b8cb48339 on
atsqa4-133.qa.lab:31010 ]
{code}
However, if a result of query above is an input to any other operator, I get
this error message:
{code}
0: jdbc:drill:schema=dfs select a1 from `t.json` where a1 is not null group
by a1;
Query failed: Query failed: Failure while running fragment., You tried to do
a batch data read operation when you were in a state of STOP. You can only
do this type of operation when you are in a state of OK or OK_NEW_SCHEMA. [
955aac65-5e43-4430-baf6-ed6bb8a020d9 on atsqa4-133.qa.lab:31010 ]
[ 955aac65-5e43-4430-baf6-ed6bb8a020d9 on atsqa4-133.qa.lab:31010 ]
Error: exception while executing query: Failure while executing query.
(state=,code=0)
{code}
Very painful for the user if query is really complex.
The same behavior if file does not exist.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (DRILL-2634) Profile UI: Cancelled queries are not moved to Completed Queries section
[ https://issues.apache.org/jira/browse/DRILL-2634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krystal closed DRILL-2634. -- Close bug for now. Will open a new bug ff encounter same query states in the future. Profile UI: Cancelled queries are not moved to Completed Queries section Key: DRILL-2634 URL: https://issues.apache.org/jira/browse/DRILL-2634 Project: Apache Drill Issue Type: Bug Components: Client - HTTP Affects Versions: 0.9.0 Reporter: Krystal Assignee: Sudheesh Katkam I started a query and then manually cancelled the query from sqlline via (cntr-C). From the Profile UI. the state of the query is correctly displayed as CANCELLATION_REQUESTED. However, the query remained under the Running Query section. It should be moved to the Completed Queries section with status of CANCELLATION_REQUESTED. Otherwise, if the drillbit is restarted, data from the Running Query section will be removed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (DRILL-2292) CTAS broken when we have repeated maps
[
https://issues.apache.org/jira/browse/DRILL-2292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deneche A. Hakim updated DRILL-2292:
Assignee: Hanifi Gunes (was: Steven Phillips)
CTAS broken when we have repeated maps
--
Key: DRILL-2292
URL: https://issues.apache.org/jira/browse/DRILL-2292
Project: Apache Drill
Issue Type: Bug
Components: Storage - Parquet
Reporter: Rahul Challapalli
Assignee: Hanifi Gunes
Priority: Blocker
Fix For: 1.0.0
Attachments: DRILL-2292.1.patch.txt, DRILL-2292.2.patch.txt
git.commit.id.abbrev=6676f2d
Data Set :
{code}
{
id : 1,
map:{rm: [
{mapid:m1,mapvalue:{col1:1,col2:[0,1,2,3,4,5]},rptd: [{ a:
foo},{b:boo}]},
{mapid:m2,mapvalue:{col1:0,col2:[]},rptd: [{ a:
bar},{c:1},{d:4.5}]}
]}
}
{code}
The below CTAS statement fails though the underlying query succeeds
{code}
create table rep_map as select d.map from `temp.json` d;
Query failed: Query stopped., index: -4, length: 4 (expected: range(0,
16384)) [ d76e3f74-7e2c-406f-a7fd-5efc68227e75 on qa-node190.qa.lab:31010 ]
{code}
Error from the logs :
{code}
Caused by: java.lang.IndexOutOfBoundsException: index: -4, length: 4
(expected: range(0, 16384))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.getInt(DrillBuf.java:447)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at
org.apache.drill.exec.vector.UInt4Vector$Accessor.get(UInt4Vector.java:309)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.RepeatedMapVector$RepeatedMapAccessor.get(RepeatedMapVector.java:451)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.setPosition(RepeatedMapReaderImpl.java:90)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.vector.complex.impl.RepeatedMapReaderImpl.reader(RepeatedMapReaderImpl.java:60)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.ParquetRecordWriter$RepeatedMapParquetConverter.init(ParquetRecordWriter.java:269)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewRepeatedMapConverter(ParquetRecordWriter.java:259)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:116)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.ParquetRecordWriter$MapParquetConverter.init(ParquetRecordWriter.java:240)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.parquet.ParquetRecordWriter.getNewMapConverter(ParquetRecordWriter.java:230)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.EventBasedRecordWriter.getConverter(EventBasedRecordWriter.java:114)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:73)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
... 19 common frames omitted
2015-02-24 19:05:56,898 [2b13346a-d929-d659-c074-354c9f5d9d4f:frag:0:0] ERROR
o.a.d.e.p.i.ScreenCreator$ScreenRoot - Error
7c8fa189-8f0a-4763-a266-fd68a11bbb66: Query stopped.
java.io.IOException: Failed to initialize FieldWriters.
at
org.apache.drill.exec.store.EventBasedRecordWriter.initFieldWriters(EventBasedRecordWriter.java:78)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.store.EventBasedRecordWriter.init(EventBasedRecordWriter.java:46)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.setupNewSchema(WriterRecordBatch.java:176)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.physical.impl.WriterRecordBatch.innerNext(WriterRecordBatch.java:113)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at
[jira] [Closed] (DRILL-495) Drill does not return columns in appropriate order for select all and aggregate selects
[ https://issues.apache.org/jira/browse/DRILL-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krystal closed DRILL-495. - This is already verified. Drill does not return columns in appropriate order for select all and aggregate selects --- Key: DRILL-495 URL: https://issues.apache.org/jira/browse/DRILL-495 Project: Apache Drill Issue Type: Bug Affects Versions: 1.0.0 Environment: CentOS release 6.5 Reporter: Krystal Fix For: 0.4.0 I have a parquet file with the following columns: rownum name age registration contributions voterzone create_time I ran the following query via jdbc/sqlline: select * from voter where age =20; I would expect the columns returned are the same as the order in the parquet data file. However the returned result is as follows: 0: jdbc:drill:schema=dfs select * from voter where age =20; +++---+++--+-+ | rownum |age | contributions | voterzone |name| registration | create_time | +++---+++--+-+ | 3 | 18 | 128.2 | 8750 | [B@3a942c3a | [B@55e3b1e1 | [B@57b356d4 | | 22 | 19 | 16.25 | 27833 | [B@3f09a547 | [B@1241f8a6 | [B@292b63a7 | | 57 | 19 | 265.9 | 11041 | [B@5f9b7e0e | [B@5970fa2b | [B@8384aed | | 59 | 18 | 835.31| 11276 | [B@27bf11d2 | [B@712b0660 | [B@3fafc2ab | | 60 | 20 | 53.19 | 7506 | [B@2c158937 | [B@39e3907d | [B@b231c3c | | 70 | 18 | 94.03 | 12853 | [B@2e12acda | [B@4c1233d7 | [B@3f098f45 | | 85 | 19 | 497.94| 8981 | [B@c9d1b58 | [B@1e6e34e0 | [B@55516dbb | I observed the same column ordering problem when using aggregates in the select statement. For example: 0: jdbc:drill:schema=dfs select name as col1, sum(contributions) as col2 from voter group by name order by name; +++ |col2|col1| +++ | 412.64 | [B@e2d9423 | | 777.23 | [B@52588d1a | | 4024.16| [B@3397d032 | | 1417.03002 | [B@33e59d14 | The columns returned should be col1 col2. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (DRILL-497) Query with != fails
[
https://issues.apache.org/jira/browse/DRILL-497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krystal closed DRILL-497.
-
Close as won't fix.
Query with != fails
--
Key: DRILL-497
URL: https://issues.apache.org/jira/browse/DRILL-497
Project: Apache Drill
Issue Type: Bug
Components: Functions - Drill
Affects Versions: 1.0.0
Environment: CentOS release 6.2
Reporter: Krystal
Assignee: Jacques Nadeau
Fix For: 0.4.0
Ran the following query via jdbc/sqlline:
select distinct(registration) from voter where registration != 'independent';
Got the following error:
Query failed: org.apache.drill.exec.rpc.RpcException: Remote failure while
running query.[error_id: 34efdf20-56fe-4bf3-a66f-d0dbf4744814
endpoint {
address: qa-node56.qa.lab
user_port: 31010
control_port: 31011
data_port: 31012
}
error_type: 0
message: Failure while parsing sql. SqlParseException:[ Lexical error at
line 1, column 61. Encountered: \!\ (33), after : \\ ] TokenMgrError:[
Lexical error at line 1, column 61. Encountered: \!\ (33), after : \\ ]
]
Error: exception while executing query (state=,code=0)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (DRILL-2645) SQL Functions page should specify return types in terms of SQL types
[ https://issues.apache.org/jira/browse/DRILL-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kristine Hahn resolved DRILL-2645. -- Resolution: Fixed Data type naming and capitalization changes have been made, mainly on these pages http://tshiran.github.io/drill/docs/sql-functions, but also made changes to these pages: * http://tshiran.github.io/drill/docs/json-data-model/ * http://tshiran.github.io/drill/docs/lexical-structure/ * http://tshiran.github.io/drill/docs/handling-different-data-types/ Some lowercase data type names have probably not been found yet and editing will be ongoing. SQL Functions page should specify return types in terms of SQL types Key: DRILL-2645 URL: https://issues.apache.org/jira/browse/DRILL-2645 Project: Apache Drill Issue Type: Bug Components: Documentation Reporter: Daniel Barclay (Drill) Assignee: Kristine Hahn On the SQL Functions page currently at http://drill.apache.org/docs/sql-functions/, the return type for string functions is specified in terms of int, text, and byte array instead of being specified in terms of Drill SQL types (e.g., INTEGER, BYTE ARRAY, etc.) Additionally, names of SQL types should be given in all uppercase, as they are used in the SQL standard, as they are specified to appear in INFORMATION_SCHEMA.COLUMNS, and as they appear in the JDBC equivalent of INFORMATION_SCHEMA.COLUMNS. (Another reason is that it's not clear, for example, which occurrence of numeric is the adjective numeric and which means the proper noun (datatype name) NUMERIC.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
