RE: drill error connecting to Hbase

2017-08-03 Thread Shai Shapira 
Attached the relevant part from the sqlline.log, 
Hope it helps


Thanks,
Shai


-Original Message-
From: Shai Shapira 
Sent: Thursday, August 03, 2017 11:04 AM
To: kkha...@mapr.com
Cc: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

Hi,

My versions are:
Hbase - 1.2.0   
Hive - 1.1.0 

I'll send the complete stack trace.

Is Drill is so version sensitive?
Can I build a solution for production based on Drill? Or should I stick to what 
is coming with the Cloudera/Hortonworks distribution?

Thanks,
Shai


hbase shell
17/08/03 10:54:05 INFO Configuration.deprecation: hadoop.native.lib is 
deprecated. Instead, use io.native.lib.available HBase Shell; enter 
'help' for list of supported commands.
Type "exit" to leave the HBase Shell Version 1.2.0-cdh5.8.2, rUnknown, 
Sun Sep 11 11:52:54 PDT 2016


hive shell

Logging initialized using configuration in 
jar:file:/opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p1980.2070/jars/hive-common-1.1.0-cdh5.8.2.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com]
Sent: Wednesday, July 26, 2017 8:58 PM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

The bundled projects (HBase, Hive) in CDH have their own versions. I'm 
wondering if that is what is the difference.

Drill has been tested with HBase 1.1.1 and Hive 1.2.1 . For higher versions, as 
long as APIs have not changed, things should be backward compatible. 

Also, the error message you see in the SQLLine session... there is a complete 
stack trace in the Drill logs. Can you share that stack trace as well?


-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com]
Sent: Wednesday, July 26, 2017 5:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

It is CDH 5.8.2

I believe it is reliable versions, isn't it?

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com]
Sent: Monday, July 24, 2017 8:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

This means that the connectivity with ZK appears to be working. 

What are the HBase, ZK and Hadoop versions that you are working with? I presume 
that the student table is otherwise accessible.

-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com]
Sent: Sunday, July 23, 2017 2:58 AM
To: user@drill.apache.org
Cc: Shai Shapira 
Subject: RE: drill error connecting to Hbase

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai




illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)


Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira 
Subject: drill error connecting to Hbase



Shai Shapira
*  shai.shap...@amdocs.com
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 


This message and the information contained herein is proprietary and 
confidential and subject to the

RE: drill error connecting to Hbase

2017-08-03 Thread Shai Shapira 
Hi,

My versions are:
Hbase - 1.2.0   
Hive - 1.1.0 

I'll send the complete stack trace.

Is Drill is so version sensitive?
Can I build a solution for production based on Drill? Or should I stick to what 
is coming with the Cloudera/Hortonworks distribution?

Thanks,
Shai


hbase shell
17/08/03 10:54:05 INFO Configuration.deprecation: hadoop.native.lib is 
deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help' for list of supported commands.
Type "exit" to leave the HBase Shell
Version 1.2.0-cdh5.8.2, rUnknown, Sun Sep 11 11:52:54 PDT 2016


hive shell

Logging initialized using configuration in 
jar:file:/opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p1980.2070/jars/hive-common-1.1.0-cdh5.8.2.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com] 
Sent: Wednesday, July 26, 2017 8:58 PM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

The bundled projects (HBase, Hive) in CDH have their own versions. I'm 
wondering if that is what is the difference.

Drill has been tested with HBase 1.1.1 and Hive 1.2.1 . For higher versions, as 
long as APIs have not changed, things should be backward compatible. 

Also, the error message you see in the SQLLine session... there is a complete 
stack trace in the Drill logs. Can you share that stack trace as well?


-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Wednesday, July 26, 2017 5:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

It is CDH 5.8.2

I believe it is reliable versions, isn't it?

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com] 
Sent: Monday, July 24, 2017 8:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

This means that the connectivity with ZK appears to be working. 

What are the HBase, ZK and Hadoop versions that you are working with? I presume 
that the student table is otherwise accessible.

-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Sunday, July 23, 2017 2:58 AM
To: user@drill.apache.org
Cc: Shai Shapira 
Subject: RE: drill error connecting to Hbase

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai




illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)


Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira 
Subject: drill error connecting to Hbase



Shai Shapira
*  shai.shap...@amdocs.com
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 


This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer

Re: delimiter in column values

2017-08-03 Thread Divya Gehlot 
Hi ,
This is my output when run in sqlline on Windows Embedded mode

0: jdbc:drill:zk=local> select * from
`dfs`.`installedsoftwares/ApacheDrill/apache-drill-1.10.0.tar/apache-drill-1.10.0/sample-data/sample_data.csv`;
++---++++
|  col_Column1   |  Column2  |Column3 |Column4 |
 Column5 |
++---++++
| "colonedata1"  | "coltwodata1" | "-35.924476"   | "138.5987123"  | ""
|
| "colonedata2"  | "coltwodata2" | "-27.4372536"  | "153.0304583"  |
"137"  |
| "colonedata3"  | "coltwodata3" | "-35.2793885"  | "149.1233503"  |
"134"  |
| "colonedata4"  | "coltwodata4" | "-33.8724176"  | "151.2067579"  | ""
|
| "colonedata5"  | "coltwodata5" | "" | "" | ""
|
| "This  |  col6 data"   | "coltwodata6"  | "-33.869732"   |
"151.203"  |
| "This  |  col7 data yes."  | "coltwodata7"  | "1.2845045"|
"103.8482739"  |
| "Chifley"  | "coltwodata5" | "" | "" | ""
|
++---++++
8 rows selected (0.147 seconds)
0: jdbc:drill:zk=local> select `col_Column1` from
`dfs`.`installedsoftwares/ApacheDrill/apache-drill-1.10.0.tar/apache-drill-1.10.0/sample-data/sample_data.csv`;
++
|  col_Column1   |
++
| "colonedata1"  |
| "colonedata2"  |
| "colonedata3"  |
| "colonedata4"  |
| "colonedata5"  |
| "This  |
| "This  |
| "Chifley"  |
++
8 rows selected (0.1 seconds)


The query returning the different results due to host operating system?


Thanks,
Divya

On 3 August 2017 at 12:45, Kunal Khatua  wrote:

> Based on your sample data, which contains this:
> Column1,Column2,Column3,Column4,Column5
> "colonedata1","coltwodata1","-35.924476","138.5987123",""
> "colonedata2","coltwodata2","-27.4372536","153.0304583","137"
> colonedata3","coltwodata3","-35.2793885","149.1233503","134"
> "colonedata4","coltwodata4","-33.8724176","151.2067579",""
> "colonedata5","coltwodata5","","",""
> "This, col6 data","coltwodata6","-33.869732","151.203","351"
> "This, col7 data yes.","coltwodata7","1.2845045","103.8482739","80"
> "Chifley","coltwodata5","","",""
>
> I got this and it looks like this...
>
>
> 0: jdbc:drill:schema=dfs.root> select * from `sample_data.csv`;
> ++
> |columns |
> ++
> | ["Column1","Column2","Column3","Column4","Column5"]
>  |
> | ["colonedata1","coltwodata1","-35.924476","138.5987123",""]|
> | ["colonedata2","coltwodata2","-27.4372536","153.0304583","137"]|
> | ["colonedata3\"","coltwodata3","-35.2793885","149.1233503","134"]  |
> | ["colonedata4","coltwodata4","-33.8724176","151.2067579",""]   |
> | ["colonedata5","coltwodata5","","",""] |
> | ["This, col6 data","coltwodata6","-33.869732","151.203","351"] |
> | ["This, col7 data yes.","coltwodata7","1.2845045","103.8482739","80"]  |
> | ["Chifley","coltwodata5","","",""] |
> ++
> 9 rows selected (0.502 seconds)
> 0: jdbc:drill:schema=dfs.root> select columns[0] from `sample_data.csv`;
> +---+
> |EXPR$0 |
> +---+
> | Column1  |
> | colonedata1   |
> | colonedata2   |
> | colonedata3"  |
> | colonedata4   |
> | colonedata5   |
> | This, col6 data   |
> | This, col7 data yes.  |
> | Chifley   |
> +---+
> 9 rows selected (0.581 seconds)
>
> I was wondering if there is something else you're seeing because you're
> running this on Windows. So I tried after converting the Unix format and
> got the exact same result. Is this what you're getting?
> I'm running this on a Linux machine.
>
> -Original Message-
> From: Divya Gehlot [mailto:divya.htco...@gmail.com]
> Sent: Wednesday, August 02, 2017 6:45 PM
> To: user@drill.apache.org
> Subject: Re: delimiter in column values
>
> Hi ,
>
> I am using Drill 1.11  and with all the setting which you have mentioned
> in plugin configurations .
> As Kunal advised to surrounded  the column values with quotes which acts
> as a string delimiter as one of my column value includes delimiter  same as
> field delimiter,  still getting the same results i.e the first column
> values are getting split into two columns as posted my earlier posts.
> I am kind of wondering how to resolve the column split issue  , as the
>