Re: Failure while reading messages from kafka

2018-09-04 Thread Matt
https://issues.apache.org/jira/browse/DRILL-6723 On Mon, Aug 27, 2018 at 12:27 PM Matt wrote: > I have a Kafka topic with some non-JSON test messages in it, resulting in > errors "Error: DATA_READ ERROR: Failure while reading messages from kafka. > Recordreader was at record: 1" > > I don't

Session handling with multiple drillbits

2018-09-04 Thread Joe Auty
Hello, We need to have some queries executed with store.format set to parquet and some with this option set to CSV. To date we have experimented with setting the store format for sessions controlled by using two separate user logins as a sort of context switch, but I'm wondering if the group

Re: Failure while reading messages from kafka

2018-09-04 Thread Khurram Faraaz
Can you please share the stack trace from drillbit.log and the version of Kafka that you are on ? Thanks, Khurram On Tue, Sep 4, 2018 at 11:39 AM, Matt wrote: > https://urldefense.proofpoint.com/v2/url?u=https-3A__issues. > apache.org_jira_browse_DRILL-2D6723=DwIBaQ=cskdkSMqhcnjZxdQVpwTXg= >

Re: Session handling with multiple drillbits

2018-09-04 Thread John Omernik
Are these ETL ish type queries? store.format should only apply when Drill is writing data, when it is reading, it uses the filenames and other hints to read. Thus, if you do HA, say with DNS (like like in the other thread) and prior to running your CREATE TABLE AS (I Am assuming this is what you

Re: Session handling with multiple drillbits

2018-09-04 Thread Joe Auty
Thanks for your response John! We are using Drill both in an ETL context, as well as for general warehouse queries. One Drill user uses store format set to Parquet while the other uses store format set to CSV to read and write from HDFS. We are currently using Kubernetes Services rather than