date:20170723

RE: drill error connecting to Hbase

2017-07-23 Thread Kunal Khatua

This means that the connectivity with ZK appears to be working. 

What are the HBase, ZK and Hadoop versions that you are working with? I presume 
that the student table is otherwise accessible.

-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Sunday, July 23, 2017 2:58 AM
To: user@drill.apache.org
Cc: Shai Shapira 
Subject: RE: drill error connecting to Hbase

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai

illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator

[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)

Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira 
Subject: drill error connecting to Hbase

Shai Shapira
*  shai.shap...@amdocs.com
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer

Re: CTAS and save as parquet last column values are shown as null

2017-07-23 Thread Divya Gehlot

0: jdbc:drill:zk=local> select * FROM
 
dfs.`installedsoftwares/ApacheDrill/apache-drill-1.10.0.tar/apache-drill-1.10.0/sample-data/jll/data/mapping/PublicHoliday/PublicHoliday.csv`
limit 10 ;
+-+
| columns |
+-+
| ["Day","Date","Area\r"] |
| ["Friday","15/1/2016","Karnataka\r"]|
| ["Tuesday","26/1/2016","Karnataka\r"]   |
| ["Monday","7/3/2016","Karnataka\r"] |
| ["Friday","25/3/2016","Karnataka\r"]|
| ["Friday","1/4/2016","Karnataka\r"] |
| ["Friday","8/4/2016","Karnataka\r"] |
| ["Thursday","14/4/2016","Karnataka\r"]  |
| ["Tuesday","19/4/2016","Karnataka\r"]   |
| ["Sunday","1/5/2016","Karnataka\r"] |
+-+
10 rows selected (0.122 seconds)
0: jdbc:drill:zk=local> select * from
`dfs`.`tmp`.`installedsoftwares/ApacheDrill/apache-drill-1.10.0.tar/apache-drill-1.10.0/sample-data/jll/publicholiday.parquet`
limit 10 ;
+---++---+
|Day|Date| Area  |
+---++---+
| Friday| 15/1/2016  | null  |
| Tuesday   | 26/1/2016  | null  |
| Monday| 7/3/2016   | null  |
| Friday| 25/3/2016  | null  |
| Friday| 1/4/2016   | null  |
| Friday| 8/4/2016   | null  |
| Thursday  | 14/4/2016  | null  |
| Tuesday   | 19/4/2016  | null  |
| Sunday| 1/5/2016   | null  |
| Monday| 9/5/2016   | null  |
+---++---+
10 rows selected (0.1 seconds)
0: jdbc:drill:zk=local>


*Drill set up* : Aapche drill is set up on Windows machine in embedded mode
.

On 24 July 2017 at 13:30, Divya Gehlot  wrote:

>
> Pasting the result set in text format
>
> *Reading parquet file format :*
>
>> Day   Date Area
>> Friday 15/1/2016 null
>> Tuesday 26/1/2016 null
>> Monday 7/3/2016 null
>> Friday 25/3/2016 null
>> Friday 1/4/2016 null
>> Friday 8/4/2016 null
>
>
>
> *Reading csv file format *
>
>> columns
>> ["Day","Date","Area\r"]
>> ["Friday","1/4/2016","Karnataka\r"]
>> ["Friday","15/1/2016","Karnataka\r"]
>> ["Friday","25/3/2016","Karnataka\r"]
>> ["Friday","8/4/2016","Karnataka\r"]
>> ["Monday","7/3/2016","Karnataka\r"]
>
>
>
>
>
> *CTAS query csv to parquet :*
>
> Create table `dfs`.`tmp`.`publicholiday.parquet` AS
>> SELECT
>> CASE WHEN `Day` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Day` AS
>> VARCHAR(100)) END AS `Day`,
>> CASE WHEN `Date` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Date` AS
>> VARCHAR(100)) END AS `Date`,
>> CASE WHEN `Area` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Area` AS
>> VARCHAR(100)) END AS `Area`
>> FROM TABLE (dfs.`PublicHoliday.csv`(type => 'text',fieldDelimiter => ',',
>> extractHeader => true))
>
>
>
> Thanks,
> Divya
>
> On 24 July 2017 at 13:20, Abhishek Girish  wrote:
>
>> Unfortunately, the attachments / pictures haven't come through. Mailing
>> lists sometimes do not support these. Can you paste as text or share links
>> to it instead?
>>
>> On Sun, Jul 23, 2017 at 9:14 PM, Divya Gehlot 
>> wrote:
>>
>> > yes it shows the proper values when I query the csv file.
>> > CTAS query csv to parquet :
>> > Create table `dfs`.`tmp`.`publicholiday.parquet` AS
>> > SELECT
>> > CASE WHEN `Day` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Day` AS
>> > VARCHAR(100)) END AS `Day`,
>> > CASE WHEN `Date` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Date`
>> AS
>> > VARCHAR(100)) END AS `Date`,
>> > CASE WHEN `Area` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Area`
>> AS
>> > VARCHAR(100)) END AS `Area`
>> > FROM TABLE (dfs.`PublicHoliday.csv`(type => 'text',fieldDelimiter =>
>> ',',
>> > extractHeader => true))
>> >
>> > CSV File
>> >
>> > Parquet File
>> >
>> >
>> >
>> > Appreciate the help !
>> >
>> > Thanks,
>> > Divya 
>> >
>> > On 24 July 2017 at 11:52, Abhishek Girish  wrote:
>> >
>> >> Can you share a sample row from the CSV and the CTAS query? Also test
>> if a
>> >> select columns[n] query on the CSV file works as expected [1] ?
>> >>
>> >> It could be an issue with delimiters.
>> >>
>> >> [1]
>> >> https://drill.apache.org/docs/querying-plain-text-files/#col
>> >> umns[n]-syntax
>> >> On Sun, Jul 23, 2017 at 8:44 PM Divya Gehlot 
>> >> wrote:
>> >>
>> >> > Hi ,
>> >> > I am facing as weird issue when I CTAS and save the csv file as
>> parquet
>> >> it
>> >> > displays the last column values as null .
>> >> > This is not the case with one file .
>> >> > If I take any csv file with even with any data type and do a
>> >> > select column1,column2,column3 from table.parquet
>> >> > it shows the column3 values as null.
>> >> >
>> >> > Appreciate the help.
>> >> >
>> >> > Thanks,
>> >> > Divya
>> >> >
>> >>
>> >
>> >
>>
>
>

Re: CTAS and save as parquet last column values are shown as null

2017-07-23 Thread Divya Gehlot

Pasting the result set in text format

*Reading parquet file format :*

> Day   Date Area
> Friday 15/1/2016 null
> Tuesday 26/1/2016 null
> Monday 7/3/2016 null
> Friday 25/3/2016 null
> Friday 1/4/2016 null
> Friday 8/4/2016 null



*Reading csv file format *

> columns
> ["Day","Date","Area\r"]
> ["Friday","1/4/2016","Karnataka\r"]
> ["Friday","15/1/2016","Karnataka\r"]
> ["Friday","25/3/2016","Karnataka\r"]
> ["Friday","8/4/2016","Karnataka\r"]
> ["Monday","7/3/2016","Karnataka\r"]





*CTAS query csv to parquet :*

Create table `dfs`.`tmp`.`publicholiday.parquet` AS
> SELECT
> CASE WHEN `Day` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Day` AS
> VARCHAR(100)) END AS `Day`,
> CASE WHEN `Date` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Date` AS
> VARCHAR(100)) END AS `Date`,
> CASE WHEN `Area` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Area` AS
> VARCHAR(100)) END AS `Area`
> FROM TABLE (dfs.`PublicHoliday.csv`(type => 'text',fieldDelimiter => ',',
> extractHeader => true))



Thanks,
Divya

On 24 July 2017 at 13:20, Abhishek Girish  wrote:

> Unfortunately, the attachments / pictures haven't come through. Mailing
> lists sometimes do not support these. Can you paste as text or share links
> to it instead?
>
> On Sun, Jul 23, 2017 at 9:14 PM, Divya Gehlot 
> wrote:
>
> > yes it shows the proper values when I query the csv file.
> > CTAS query csv to parquet :
> > Create table `dfs`.`tmp`.`publicholiday.parquet` AS
> > SELECT
> > CASE WHEN `Day` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Day` AS
> > VARCHAR(100)) END AS `Day`,
> > CASE WHEN `Date` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Date` AS
> > VARCHAR(100)) END AS `Date`,
> > CASE WHEN `Area` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Area` AS
> > VARCHAR(100)) END AS `Area`
> > FROM TABLE (dfs.`PublicHoliday.csv`(type => 'text',fieldDelimiter => ',',
> > extractHeader => true))
> >
> > CSV File
> >
> > Parquet File
> >
> >
> >
> > Appreciate the help !
> >
> > Thanks,
> > Divya 
> >
> > On 24 July 2017 at 11:52, Abhishek Girish  wrote:
> >
> >> Can you share a sample row from the CSV and the CTAS query? Also test
> if a
> >> select columns[n] query on the CSV file works as expected [1] ?
> >>
> >> It could be an issue with delimiters.
> >>
> >> [1]
> >> https://drill.apache.org/docs/querying-plain-text-files/#col
> >> umns[n]-syntax
> >> On Sun, Jul 23, 2017 at 8:44 PM Divya Gehlot 
> >> wrote:
> >>
> >> > Hi ,
> >> > I am facing as weird issue when I CTAS and save the csv file as
> parquet
> >> it
> >> > displays the last column values as null .
> >> > This is not the case with one file .
> >> > If I take any csv file with even with any data type and do a
> >> > select column1,column2,column3 from table.parquet
> >> > it shows the column3 values as null.
> >> >
> >> > Appreciate the help.
> >> >
> >> > Thanks,
> >> > Divya
> >> >
> >>
> >
> >
>

Re: CTAS and save as parquet last column values are shown as null

2017-07-23 Thread Abhishek Girish

Unfortunately, the attachments / pictures haven't come through. Mailing
lists sometimes do not support these. Can you paste as text or share links
to it instead?

On Sun, Jul 23, 2017 at 9:14 PM, Divya Gehlot 
wrote:

> yes it shows the proper values when I query the csv file.
> CTAS query csv to parquet :
> Create table `dfs`.`tmp`.`publicholiday.parquet` AS
> SELECT
> CASE WHEN `Day` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Day` AS
> VARCHAR(100)) END AS `Day`,
> CASE WHEN `Date` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Date` AS
> VARCHAR(100)) END AS `Date`,
> CASE WHEN `Area` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Area` AS
> VARCHAR(100)) END AS `Area`
> FROM TABLE (dfs.`PublicHoliday.csv`(type => 'text',fieldDelimiter => ',',
> extractHeader => true))
>
> CSV File
>
> Parquet File
>
>
>
> Appreciate the help !
>
> Thanks,
> Divya 
>
> On 24 July 2017 at 11:52, Abhishek Girish  wrote:
>
>> Can you share a sample row from the CSV and the CTAS query? Also test if a
>> select columns[n] query on the CSV file works as expected [1] ?
>>
>> It could be an issue with delimiters.
>>
>> [1]
>> https://drill.apache.org/docs/querying-plain-text-files/#col
>> umns[n]-syntax
>> On Sun, Jul 23, 2017 at 8:44 PM Divya Gehlot 
>> wrote:
>>
>> > Hi ,
>> > I am facing as weird issue when I CTAS and save the csv file as parquet
>> it
>> > displays the last column values as null .
>> > This is not the case with one file .
>> > If I take any csv file with even with any data type and do a
>> > select column1,column2,column3 from table.parquet
>> > it shows the column3 values as null.
>> >
>> > Appreciate the help.
>> >
>> > Thanks,
>> > Divya
>> >
>>
>
>

Re: CTAS and save as parquet last column values are shown as null

2017-07-23 Thread Divya Gehlot

yes it shows the proper values when I query the csv file.
CTAS query csv to parquet :
Create table `dfs`.`tmp`.`publicholiday.parquet` AS
SELECT
CASE WHEN `Day` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Day` AS
VARCHAR(100)) END AS `Day`,
CASE WHEN `Date` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Date` AS
VARCHAR(100)) END AS `Date`,
CASE WHEN `Area` = '' THEN CAST(NULL AS VARCHAR(100)) ELSE CAST(`Area` AS
VARCHAR(100)) END AS `Area`
FROM TABLE (dfs.`PublicHoliday.csv`(type => 'text',fieldDelimiter => ',',
extractHeader => true))

CSV File

Parquet File

Appreciate the help !

Thanks,
Divya 

On 24 July 2017 at 11:52, Abhishek Girish  wrote:

> Can you share a sample row from the CSV and the CTAS query? Also test if a
> select columns[n] query on the CSV file works as expected [1] ?
>
> It could be an issue with delimiters.
>
> [1]
> https://drill.apache.org/docs/querying-plain-text-files/#columns[n]-syntax
> On Sun, Jul 23, 2017 at 8:44 PM Divya Gehlot 
> wrote:
>
> > Hi ,
> > I am facing as weird issue when I CTAS and save the csv file as parquet
> it
> > displays the last column values as null .
> > This is not the case with one file .
> > If I take any csv file with even with any data type and do a
> > select column1,column2,column3 from table.parquet
> > it shows the column3 values as null.
> >
> > Appreciate the help.
> >
> > Thanks,
> > Divya
> >
>

kudu hash partition query error

2017-07-23 Thread zhongpoc...@kuaizi.co

Hi,
When I query kudu throught drill ,I got an exception : 
NonRecoverableException: Invalid scan stop key: Error decoding composite key 
component 'metric': Missing separator after composite key string component.
My create kudu table code as follow:

===
ColumnSchema host = new ColumnSchema.ColumnSchemaBuilder("host", 
Type.STRING).key(true).build();
ColumnSchema metric = new ColumnSchema.ColumnSchemaBuilder("metric", 
Type.STRING).key(true).build();
ColumnSchema timestamp = new ColumnSchema.ColumnSchemaBuilder("timestamp", 
Type.UNIXTIME_MICROS).key(true).build();
ColumnSchema value = new ColumnSchema.ColumnSchemaBuilder("value", 
Type.DOUBLE).build();
Schema schema = new Schema(ImmutableList.of(host, metric, timestamp, value));


CreateTableOptions tableBuilder = new CreateTableOptions();
tableBuilder.setRangePartitionColumns(ImmutableList.of("timestamp"));
tableBuilder.setNumReplicas(1);


PartialRow split = schema.newPartialRow();
split.addLong("timestamp", 10);
tableBuilder.addSplitRow(split);


tableBuilder.addHashPartitions(ImmutableList.of("host", "metric"), 2);


String tableName = "testPruning";
client.createTable(tableName, schema, tableBuilder);
=
When I query "select * from kudu.testPruning"  in drill, I got this exception: 
"NonRecoverableException: Invalid scan stop key: Error decoding composite key 
component 'metric': Missing separator after composite key string component."


Thank you for your reply.

Re: CTAS and save as parquet last column values are shown as null

2017-07-23 Thread Abhishek Girish

Can you share a sample row from the CSV and the CTAS query? Also test if a
select columns[n] query on the CSV file works as expected [1] ?

It could be an issue with delimiters.

[1]
https://drill.apache.org/docs/querying-plain-text-files/#columns[n]-syntax
On Sun, Jul 23, 2017 at 8:44 PM Divya Gehlot 
wrote:

> Hi ,
> I am facing as weird issue when I CTAS and save the csv file as parquet it
> displays the last column values as null .
> This is not the case with one file .
> If I take any csv file with even with any data type and do a
> select column1,column2,column3 from table.parquet
> it shows the column3 values as null.
>
> Appreciate the help.
>
> Thanks,
> Divya
>

CTAS and save as parquet last column values are shown as null

2017-07-23 Thread Divya Gehlot

Hi ,
I am facing as weird issue when I CTAS and save the csv file as parquet it
displays the last column values as null .
This is not the case with one file .
If I take any csv file with even with any data type and do a
select column1,column2,column3 from table.parquet
it shows the column3 values as null.

Appreciate the help.

Thanks,
Divya

RE: drill error connecting to Hbase

2017-07-23 Thread Shai Shapira

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai




illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)


Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira 
Subject: drill error connecting to Hbase



Shai Shapira
*  shai.shap...@amdocs.com
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer

RE: drill error connecting to Hbase

Re: CTAS and save as parquet last column values are shown as null

Re: CTAS and save as parquet last column values are shown as null

Re: CTAS and save as parquet last column values are shown as null

Re: CTAS and save as parquet last column values are shown as null

kudu hash partition query error

Re: CTAS and save as parquet last column values are shown as null

CTAS and save as parquet last column values are shown as null

RE: drill error connecting to Hbase

9 matches

Site Navigation

Mail list logo

Footer information