Re: Drill error

[Kunal Khatua]

That could be a possibility. See if you can skip that value in the column... 
try using a filter condition with a such as 
NOT LIKE '%' 
On 6/27/2018 11:25:10 PM, Nitin Pawar  wrote:
Could this cause an issue if one of the field in concat function has large
text ?

On Thu, Jun 28, 2018 at 11:10 AM, Nitin Pawar
wrote:

> Hi Khurram,
>
> This is a parquet table.
> all the columns in the table are string columns (even date column is
> stored as string)
>
> I am currently using drill 1.13.0. Same query used to work fine on drill
> 1.8.0
>
> Thanks,
> Nitin
>
> On Thu, Jun 28, 2018 at 12:17 AM, Khurram Faraaz wrote:
>
>> Hi Nitin,
>>
>> Can you please share the description of the table (i.e. column types) ?
>> Is this a parquet table or JSON ?
>> Also please share the version of Drill and the drillbit.log
>>
>> Thanks,
>> Khurram
>>
>> On Wed, Jun 27, 2018 at 9:45 AM, Nitin Pawar
>> wrote:
>>
>> > here are the details
>> >
>> > query:
>> >
>> > select Account Account,Name Name,CONCAT(DateString , string2)
>> > Merged_String from dfs.tmp.`/nitin/` t1
>> >
>> > There is no custom UDF in this query;
>> >
>> >
>> >
>> > On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar
>> > wrote:
>> >
>> > > Hi Vitalii,
>> > >
>> > > Thanks for the description.
>> > > I will try to get the log for this but as this is happening in
>> production
>> > > and drill log gets overwritten in our case very fast.
>> > >
>> > > Also I will get the query for the same while getting the log
>> > >
>> > > I will get the details and update the thread
>> > >
>> > > Thanks,
>> > > Nitin
>> > >
>> > > On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka <>
>> > > vitalii.dira...@gmail.com> wrote:
>> > >
>> > >> Hi Nitin,
>> > >>
>> > >> It happens in the process of reallocation of the size of buffers in
>> the
>> > >> memory.
>> > >> It isn't a User Exception, so it looks like a bug, if you get it in
>> some
>> > >> existed plugin.
>> > >> But to say you exactly, please describe your case. What kind of query
>> > did
>> > >> you perform, any UDF's, which data source?
>> > >> Also logs can help.
>> > >>
>> > >> Thanks.
>> > >>
>> > >> Kind regards
>> > >> Vitalii
>> > >>
>> > >>
>> > >> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar
>> >
>> > >> wrote:
>> > >>
>> > >> > Hi,
>> > >> >
>> > >> > Can someone help me understand below error? and how do I not let
>> this
>> > >> > happen ??
>> > >> >
>> > >> > SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged
>> buffer.
>> > >> >
>> > >> > Fragment 0:0
>> > >> >
>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
>> > >> > ip-10-0-103-63.ec2.internal:31010]
>> > >> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove
>> > >> unmanaged
>> > >> > buffer.
>> > >> >
>> > >> > Fragment 0:0
>> > >> >
>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
>> > >> > ip-10-0-103-63.ec2.internal:31010]
>> > >> >
>> > >> > --
>> > >> > Nitin Pawar
>> > >> >
>> > >>
>> > >
>> > >
>> > >
>> > > --
>> > > Nitin Pawar
>> > >
>> >
>> >
>> >
>> > --
>> > Nitin Pawar
>> >
>>
>
>
>
> --
> Nitin Pawar
>



--
Nitin Pawar

Re: Drill error

[Vitalii Diravka]

Thank you, Nitin

Kind regards
Vitalii


On Thu, Jun 28, 2018 at 6:04 PM Nitin Pawar  wrote:

> created https://issues.apache.org/jira/browse/DRILL-6551
>
> On Thu, Jun 28, 2018 at 7:43 PM, Vitalii Diravka <
> vitalii.dira...@gmail.com>
> wrote:
>
> > Hi Nitin,
> >
> > This is definitely the regression. Could you please post the Jira ticket
> > with details of your case, data and workaround [1].
> > It will help for developers to solve this issue.
> >
> > [1] https://issues.apache.org/jira/projects/DRILL
> > Thanks.
> >
> > Kind regards
> > Vitalii
> >
> >
> > On Thu, Jun 28, 2018 at 10:50 AM Nitin Pawar 
> > wrote:
> >
> > > I was able to fix this issue by doing concat(string1, ' ', string2)
> > instead
> > > of concat(string1, string2)
> > > Not sure how adding a separator helps but it solved the problem
> > >
> > > On Thu, Jun 28, 2018 at 11:55 AM, Nitin Pawar  >
> > > wrote:
> > >
> > > > Could this cause an issue if one of the field in concat function has
> > > large
> > > > text ?
> > > >
> > > > On Thu, Jun 28, 2018 at 11:10 AM, Nitin Pawar <
> nitinpawar...@gmail.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Khurram,
> > > >>
> > > >> This is a parquet table.
> > > >> all the columns in the table are string columns (even date column is
> > > >> stored as string)
> > > >>
> > > >> I am currently using drill 1.13.0. Same query used to work fine on
> > drill
> > > >> 1.8.0
> > > >>
> > > >> Thanks,
> > > >> Nitin
> > > >>
> > > >> On Thu, Jun 28, 2018 at 12:17 AM, Khurram Faraaz 
> > > >> wrote:
> > > >>
> > > >>> Hi Nitin,
> > > >>>
> > > >>> Can you please share the description of the table (i.e. column
> > types) ?
> > > >>> Is this a parquet table or JSON ?
> > > >>> Also please share the version of Drill and the drillbit.log
> > > >>>
> > > >>> Thanks,
> > > >>> Khurram
> > > >>>
> > > >>> On Wed, Jun 27, 2018 at 9:45 AM, Nitin Pawar <
> > nitinpawar...@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>> > here are the details
> > > >>> >
> > > >>> > query:
> > > >>> >
> > > >>> > select Account Account,Name Name,CONCAT(DateString , string2)
> > > >>> > Merged_String from dfs.tmp.`/nitin/` t1
> > > >>> >
> > > >>> > There is no custom UDF in this query;
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar <
> > > nitinpawar...@gmail.com>
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Hi Vitalii,
> > > >>> > >
> > > >>> > > Thanks for the description.
> > > >>> > > I will try to get the log for this but as this is happening in
> > > >>> production
> > > >>> > > and drill log gets overwritten in our case very fast.
> > > >>> > >
> > > >>> > > Also I will get the query for the same while getting the log
> > > >>> > >
> > > >>> > > I will get the details and update the thread
> > > >>> > >
> > > >>> > > Thanks,
> > > >>> > > Nitin
> > > >>> > >
> > > >>> > > On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka <
> > > >>> > > vitalii.dira...@gmail.com> wrote:
> > > >>> > >
> > > >>> > >> Hi Nitin,
> > > >>> > >>
> > > >>> > >> It happens in the process of reallocation of the size of
> buffers
> > > in
> > > >>> the
> > > >>> > >> memory.
> > > >>> > >> It isn't a User Exception, so it looks like a bug, if you get
> it
> > > in
> > > >>> some
> > > >>> > >> existed plugin.
> > > >>> > >> But to say you exactly, please describe your case. What kind
> of
> > > >>> query
> > > >>> > did
> > > >>> > >> you perform, any UDF's, which data source?
> > > >>> > >> Also logs can help.
> > > >>> > >>
> > > >>> > >> Thanks.
> > > >>> > >>
> > > >>> > >> Kind regards
> > > >>> > >> Vitalii
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar <
> > > >>> nitinpawar...@gmail.com>
> > > >>> > >> wrote:
> > > >>> > >>
> > > >>> > >> > Hi,
> > > >>> > >> >
> > > >>> > >> > Can someone help me understand below error? and how do I not
> > let
> > > >>> this
> > > >>> > >> > happen ??
> > > >>> > >> >
> > > >>> > >> > SYSTEM ERROR: IllegalStateException: Tried to remove
> unmanaged
> > > >>> buffer.
> > > >>> > >> >
> > > >>> > >> > Fragment 0:0
> > > >>> > >> >
> > > >>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> > > >>> > >> > ip-10-0-103-63.ec2.internal:31010]
> > > >>> > >> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to
> > remove
> > > >>> > >> unmanaged
> > > >>> > >> > buffer.
> > > >>> > >> >
> > > >>> > >> > Fragment 0:0
> > > >>> > >> >
> > > >>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> > > >>> > >> > ip-10-0-103-63.ec2.internal:31010]
> > > >>> > >> >
> > > >>> > >> > --
> > > >>> > >> > Nitin Pawar
> > > >>> > >> >
> > > >>> > >>
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > --
> > > >>> > > Nitin Pawar
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > Nitin Pawar
> > > >>> >
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Nitin Pawar
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Nitin Pawar
> > > >
> > >
> > >
> > >
> > > --
> > > Nitin Pawar
> 

Re: Drill error

[Nitin Pawar]

created https://issues.apache.org/jira/browse/DRILL-6551

On Thu, Jun 28, 2018 at 7:43 PM, Vitalii Diravka 
wrote:

> Hi Nitin,
>
> This is definitely the regression. Could you please post the Jira ticket
> with details of your case, data and workaround [1].
> It will help for developers to solve this issue.
>
> [1] https://issues.apache.org/jira/projects/DRILL
> Thanks.
>
> Kind regards
> Vitalii
>
>
> On Thu, Jun 28, 2018 at 10:50 AM Nitin Pawar 
> wrote:
>
> > I was able to fix this issue by doing concat(string1, ' ', string2)
> instead
> > of concat(string1, string2)
> > Not sure how adding a separator helps but it solved the problem
> >
> > On Thu, Jun 28, 2018 at 11:55 AM, Nitin Pawar 
> > wrote:
> >
> > > Could this cause an issue if one of the field in concat function has
> > large
> > > text ?
> > >
> > > On Thu, Jun 28, 2018 at 11:10 AM, Nitin Pawar  >
> > > wrote:
> > >
> > >> Hi Khurram,
> > >>
> > >> This is a parquet table.
> > >> all the columns in the table are string columns (even date column is
> > >> stored as string)
> > >>
> > >> I am currently using drill 1.13.0. Same query used to work fine on
> drill
> > >> 1.8.0
> > >>
> > >> Thanks,
> > >> Nitin
> > >>
> > >> On Thu, Jun 28, 2018 at 12:17 AM, Khurram Faraaz 
> > >> wrote:
> > >>
> > >>> Hi Nitin,
> > >>>
> > >>> Can you please share the description of the table (i.e. column
> types) ?
> > >>> Is this a parquet table or JSON ?
> > >>> Also please share the version of Drill and the drillbit.log
> > >>>
> > >>> Thanks,
> > >>> Khurram
> > >>>
> > >>> On Wed, Jun 27, 2018 at 9:45 AM, Nitin Pawar <
> nitinpawar...@gmail.com>
> > >>> wrote:
> > >>>
> > >>> > here are the details
> > >>> >
> > >>> > query:
> > >>> >
> > >>> > select Account Account,Name Name,CONCAT(DateString , string2)
> > >>> > Merged_String from dfs.tmp.`/nitin/` t1
> > >>> >
> > >>> > There is no custom UDF in this query;
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar <
> > nitinpawar...@gmail.com>
> > >>> > wrote:
> > >>> >
> > >>> > > Hi Vitalii,
> > >>> > >
> > >>> > > Thanks for the description.
> > >>> > > I will try to get the log for this but as this is happening in
> > >>> production
> > >>> > > and drill log gets overwritten in our case very fast.
> > >>> > >
> > >>> > > Also I will get the query for the same while getting the log
> > >>> > >
> > >>> > > I will get the details and update the thread
> > >>> > >
> > >>> > > Thanks,
> > >>> > > Nitin
> > >>> > >
> > >>> > > On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka <
> > >>> > > vitalii.dira...@gmail.com> wrote:
> > >>> > >
> > >>> > >> Hi Nitin,
> > >>> > >>
> > >>> > >> It happens in the process of reallocation of the size of buffers
> > in
> > >>> the
> > >>> > >> memory.
> > >>> > >> It isn't a User Exception, so it looks like a bug, if you get it
> > in
> > >>> some
> > >>> > >> existed plugin.
> > >>> > >> But to say you exactly, please describe your case. What kind of
> > >>> query
> > >>> > did
> > >>> > >> you perform, any UDF's, which data source?
> > >>> > >> Also logs can help.
> > >>> > >>
> > >>> > >> Thanks.
> > >>> > >>
> > >>> > >> Kind regards
> > >>> > >> Vitalii
> > >>> > >>
> > >>> > >>
> > >>> > >> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar <
> > >>> nitinpawar...@gmail.com>
> > >>> > >> wrote:
> > >>> > >>
> > >>> > >> > Hi,
> > >>> > >> >
> > >>> > >> > Can someone help me understand below error? and how do I not
> let
> > >>> this
> > >>> > >> > happen ??
> > >>> > >> >
> > >>> > >> > SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged
> > >>> buffer.
> > >>> > >> >
> > >>> > >> > Fragment 0:0
> > >>> > >> >
> > >>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> > >>> > >> > ip-10-0-103-63.ec2.internal:31010]
> > >>> > >> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to
> remove
> > >>> > >> unmanaged
> > >>> > >> > buffer.
> > >>> > >> >
> > >>> > >> > Fragment 0:0
> > >>> > >> >
> > >>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> > >>> > >> > ip-10-0-103-63.ec2.internal:31010]
> > >>> > >> >
> > >>> > >> > --
> > >>> > >> > Nitin Pawar
> > >>> > >> >
> > >>> > >>
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> > > --
> > >>> > > Nitin Pawar
> > >>> > >
> > >>> >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Nitin Pawar
> > >>> >
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Nitin Pawar
> > >>
> > >
> > >
> > >
> > > --
> > > Nitin Pawar
> > >
> >
> >
> >
> > --
> > Nitin Pawar
> >
>



-- 
Nitin Pawar

Re: Drill error

[Vitalii Diravka]

Hi Nitin,

This is definitely the regression. Could you please post the Jira ticket
with details of your case, data and workaround [1].
It will help for developers to solve this issue.

[1] https://issues.apache.org/jira/projects/DRILL
Thanks.

Kind regards
Vitalii


On Thu, Jun 28, 2018 at 10:50 AM Nitin Pawar 
wrote:

> I was able to fix this issue by doing concat(string1, ' ', string2) instead
> of concat(string1, string2)
> Not sure how adding a separator helps but it solved the problem
>
> On Thu, Jun 28, 2018 at 11:55 AM, Nitin Pawar 
> wrote:
>
> > Could this cause an issue if one of the field in concat function has
> large
> > text ?
> >
> > On Thu, Jun 28, 2018 at 11:10 AM, Nitin Pawar 
> > wrote:
> >
> >> Hi Khurram,
> >>
> >> This is a parquet table.
> >> all the columns in the table are string columns (even date column is
> >> stored as string)
> >>
> >> I am currently using drill 1.13.0. Same query used to work fine on drill
> >> 1.8.0
> >>
> >> Thanks,
> >> Nitin
> >>
> >> On Thu, Jun 28, 2018 at 12:17 AM, Khurram Faraaz 
> >> wrote:
> >>
> >>> Hi Nitin,
> >>>
> >>> Can you please share the description of the table (i.e. column types) ?
> >>> Is this a parquet table or JSON ?
> >>> Also please share the version of Drill and the drillbit.log
> >>>
> >>> Thanks,
> >>> Khurram
> >>>
> >>> On Wed, Jun 27, 2018 at 9:45 AM, Nitin Pawar 
> >>> wrote:
> >>>
> >>> > here are the details
> >>> >
> >>> > query:
> >>> >
> >>> > select Account Account,Name Name,CONCAT(DateString , string2)
> >>> > Merged_String from dfs.tmp.`/nitin/` t1
> >>> >
> >>> > There is no custom UDF in this query;
> >>> >
> >>> >
> >>> >
> >>> > On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar <
> nitinpawar...@gmail.com>
> >>> > wrote:
> >>> >
> >>> > > Hi Vitalii,
> >>> > >
> >>> > > Thanks for the description.
> >>> > > I will try to get the log for this but as this is happening in
> >>> production
> >>> > > and drill log gets overwritten in our case very fast.
> >>> > >
> >>> > > Also I will get the query for the same while getting the log
> >>> > >
> >>> > > I will get the details and update the thread
> >>> > >
> >>> > > Thanks,
> >>> > > Nitin
> >>> > >
> >>> > > On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka <
> >>> > > vitalii.dira...@gmail.com> wrote:
> >>> > >
> >>> > >> Hi Nitin,
> >>> > >>
> >>> > >> It happens in the process of reallocation of the size of buffers
> in
> >>> the
> >>> > >> memory.
> >>> > >> It isn't a User Exception, so it looks like a bug, if you get it
> in
> >>> some
> >>> > >> existed plugin.
> >>> > >> But to say you exactly, please describe your case. What kind of
> >>> query
> >>> > did
> >>> > >> you perform, any UDF's, which data source?
> >>> > >> Also logs can help.
> >>> > >>
> >>> > >> Thanks.
> >>> > >>
> >>> > >> Kind regards
> >>> > >> Vitalii
> >>> > >>
> >>> > >>
> >>> > >> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar <
> >>> nitinpawar...@gmail.com>
> >>> > >> wrote:
> >>> > >>
> >>> > >> > Hi,
> >>> > >> >
> >>> > >> > Can someone help me understand below error? and how do I not let
> >>> this
> >>> > >> > happen ??
> >>> > >> >
> >>> > >> > SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged
> >>> buffer.
> >>> > >> >
> >>> > >> > Fragment 0:0
> >>> > >> >
> >>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> >>> > >> > ip-10-0-103-63.ec2.internal:31010]
> >>> > >> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove
> >>> > >> unmanaged
> >>> > >> > buffer.
> >>> > >> >
> >>> > >> > Fragment 0:0
> >>> > >> >
> >>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> >>> > >> > ip-10-0-103-63.ec2.internal:31010]
> >>> > >> >
> >>> > >> > --
> >>> > >> > Nitin Pawar
> >>> > >> >
> >>> > >>
> >>> > >
> >>> > >
> >>> > >
> >>> > > --
> >>> > > Nitin Pawar
> >>> > >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Nitin Pawar
> >>> >
> >>>
> >>
> >>
> >>
> >> --
> >> Nitin Pawar
> >>
> >
> >
> >
> > --
> > Nitin Pawar
> >
>
>
>
> --
> Nitin Pawar
>

Re: Drill error

[Nitin Pawar]

I was able to fix this issue by doing concat(string1, ' ', string2) instead
of concat(string1, string2)
Not sure how adding a separator helps but it solved the problem

On Thu, Jun 28, 2018 at 11:55 AM, Nitin Pawar 
wrote:

> Could this cause an issue if one of the field in concat function has large
> text ?
>
> On Thu, Jun 28, 2018 at 11:10 AM, Nitin Pawar 
> wrote:
>
>> Hi Khurram,
>>
>> This is a parquet table.
>> all the columns in the table are string columns (even date column is
>> stored as string)
>>
>> I am currently using drill 1.13.0. Same query used to work fine on drill
>> 1.8.0
>>
>> Thanks,
>> Nitin
>>
>> On Thu, Jun 28, 2018 at 12:17 AM, Khurram Faraaz 
>> wrote:
>>
>>> Hi Nitin,
>>>
>>> Can you please share the description of the table (i.e. column types) ?
>>> Is this a parquet table or JSON ?
>>> Also please share the version of Drill and the drillbit.log
>>>
>>> Thanks,
>>> Khurram
>>>
>>> On Wed, Jun 27, 2018 at 9:45 AM, Nitin Pawar 
>>> wrote:
>>>
>>> > here are the details
>>> >
>>> > query:
>>> >
>>> > select Account Account,Name Name,CONCAT(DateString , string2)
>>> > Merged_String from dfs.tmp.`/nitin/` t1
>>> >
>>> > There is no custom UDF in this query;
>>> >
>>> >
>>> >
>>> > On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar 
>>> > wrote:
>>> >
>>> > > Hi Vitalii,
>>> > >
>>> > > Thanks for the description.
>>> > > I will try to get the log for this but as this is happening in
>>> production
>>> > > and drill log gets overwritten in our case very fast.
>>> > >
>>> > > Also I will get the query for the same while getting the log
>>> > >
>>> > > I will get the details and update the thread
>>> > >
>>> > > Thanks,
>>> > > Nitin
>>> > >
>>> > > On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka <
>>> > > vitalii.dira...@gmail.com> wrote:
>>> > >
>>> > >> Hi Nitin,
>>> > >>
>>> > >> It happens in the process of reallocation of the size of buffers in
>>> the
>>> > >> memory.
>>> > >> It isn't a User Exception, so it looks like a bug, if you get it in
>>> some
>>> > >> existed plugin.
>>> > >> But to say you exactly, please describe your case. What kind of
>>> query
>>> > did
>>> > >> you perform, any UDF's, which data source?
>>> > >> Also logs can help.
>>> > >>
>>> > >> Thanks.
>>> > >>
>>> > >> Kind regards
>>> > >> Vitalii
>>> > >>
>>> > >>
>>> > >> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar <
>>> nitinpawar...@gmail.com>
>>> > >> wrote:
>>> > >>
>>> > >> > Hi,
>>> > >> >
>>> > >> > Can someone help me understand below error? and how do I not let
>>> this
>>> > >> > happen ??
>>> > >> >
>>> > >> > SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged
>>> buffer.
>>> > >> >
>>> > >> > Fragment 0:0
>>> > >> >
>>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
>>> > >> > ip-10-0-103-63.ec2.internal:31010]
>>> > >> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove
>>> > >> unmanaged
>>> > >> > buffer.
>>> > >> >
>>> > >> > Fragment 0:0
>>> > >> >
>>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
>>> > >> > ip-10-0-103-63.ec2.internal:31010]
>>> > >> >
>>> > >> > --
>>> > >> > Nitin Pawar
>>> > >> >
>>> > >>
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Nitin Pawar
>>> > >
>>> >
>>> >
>>> >
>>> > --
>>> > Nitin Pawar
>>> >
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
>
> --
> Nitin Pawar
>



-- 
Nitin Pawar

Re: Drill error

[Nitin Pawar]

Could this cause an issue if one of the field in concat function has large
text ?

On Thu, Jun 28, 2018 at 11:10 AM, Nitin Pawar 
wrote:

> Hi Khurram,
>
> This is a parquet table.
> all the columns in the table are string columns (even date column is
> stored as string)
>
> I am currently using drill 1.13.0. Same query used to work fine on drill
> 1.8.0
>
> Thanks,
> Nitin
>
> On Thu, Jun 28, 2018 at 12:17 AM, Khurram Faraaz  wrote:
>
>> Hi Nitin,
>>
>> Can you please share the description of the table (i.e. column types) ?
>> Is this a parquet table or JSON ?
>> Also please share the version of Drill and the drillbit.log
>>
>> Thanks,
>> Khurram
>>
>> On Wed, Jun 27, 2018 at 9:45 AM, Nitin Pawar 
>> wrote:
>>
>> > here are the details
>> >
>> > query:
>> >
>> > select Account Account,Name Name,CONCAT(DateString , string2)
>> > Merged_String from dfs.tmp.`/nitin/` t1
>> >
>> > There is no custom UDF in this query;
>> >
>> >
>> >
>> > On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar 
>> > wrote:
>> >
>> > > Hi Vitalii,
>> > >
>> > > Thanks for the description.
>> > > I will try to get the log for this but as this is happening in
>> production
>> > > and drill log gets overwritten in our case very fast.
>> > >
>> > > Also I will get the query for the same while getting the log
>> > >
>> > > I will get the details and update the thread
>> > >
>> > > Thanks,
>> > > Nitin
>> > >
>> > > On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka <
>> > > vitalii.dira...@gmail.com> wrote:
>> > >
>> > >> Hi Nitin,
>> > >>
>> > >> It happens in the process of reallocation of the size of buffers in
>> the
>> > >> memory.
>> > >> It isn't a User Exception, so it looks like a bug, if you get it in
>> some
>> > >> existed plugin.
>> > >> But to say you exactly, please describe your case. What kind of query
>> > did
>> > >> you perform, any UDF's, which data source?
>> > >> Also logs can help.
>> > >>
>> > >> Thanks.
>> > >>
>> > >> Kind regards
>> > >> Vitalii
>> > >>
>> > >>
>> > >> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar > >
>> > >> wrote:
>> > >>
>> > >> > Hi,
>> > >> >
>> > >> > Can someone help me understand below error? and how do I not let
>> this
>> > >> > happen ??
>> > >> >
>> > >> > SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged
>> buffer.
>> > >> >
>> > >> > Fragment 0:0
>> > >> >
>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
>> > >> > ip-10-0-103-63.ec2.internal:31010]
>> > >> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove
>> > >> unmanaged
>> > >> > buffer.
>> > >> >
>> > >> > Fragment 0:0
>> > >> >
>> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
>> > >> > ip-10-0-103-63.ec2.internal:31010]
>> > >> >
>> > >> > --
>> > >> > Nitin Pawar
>> > >> >
>> > >>
>> > >
>> > >
>> > >
>> > > --
>> > > Nitin Pawar
>> > >
>> >
>> >
>> >
>> > --
>> > Nitin Pawar
>> >
>>
>
>
>
> --
> Nitin Pawar
>



-- 
Nitin Pawar

Re: Drill error

[Nitin Pawar]

Hi Khurram,

This is a parquet table.
all the columns in the table are string columns (even date column is stored
as string)

I am currently using drill 1.13.0. Same query used to work fine on drill
1.8.0

Thanks,
Nitin

On Thu, Jun 28, 2018 at 12:17 AM, Khurram Faraaz  wrote:

> Hi Nitin,
>
> Can you please share the description of the table (i.e. column types) ?
> Is this a parquet table or JSON ?
> Also please share the version of Drill and the drillbit.log
>
> Thanks,
> Khurram
>
> On Wed, Jun 27, 2018 at 9:45 AM, Nitin Pawar 
> wrote:
>
> > here are the details
> >
> > query:
> >
> > select Account Account,Name Name,CONCAT(DateString , string2)
> > Merged_String from dfs.tmp.`/nitin/` t1
> >
> > There is no custom UDF in this query;
> >
> >
> >
> > On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar 
> > wrote:
> >
> > > Hi Vitalii,
> > >
> > > Thanks for the description.
> > > I will try to get the log for this but as this is happening in
> production
> > > and drill log gets overwritten in our case very fast.
> > >
> > > Also I will get the query for the same while getting the log
> > >
> > > I will get the details and update the thread
> > >
> > > Thanks,
> > > Nitin
> > >
> > > On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka <
> > > vitalii.dira...@gmail.com> wrote:
> > >
> > >> Hi Nitin,
> > >>
> > >> It happens in the process of reallocation of the size of buffers in
> the
> > >> memory.
> > >> It isn't a User Exception, so it looks like a bug, if you get it in
> some
> > >> existed plugin.
> > >> But to say you exactly, please describe your case. What kind of query
> > did
> > >> you perform, any UDF's, which data source?
> > >> Also logs can help.
> > >>
> > >> Thanks.
> > >>
> > >> Kind regards
> > >> Vitalii
> > >>
> > >>
> > >> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar 
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > Can someone help me understand below error? and how do I not let
> this
> > >> > happen ??
> > >> >
> > >> > SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged
> buffer.
> > >> >
> > >> > Fragment 0:0
> > >> >
> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> > >> > ip-10-0-103-63.ec2.internal:31010]
> > >> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove
> > >> unmanaged
> > >> > buffer.
> > >> >
> > >> > Fragment 0:0
> > >> >
> > >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> > >> > ip-10-0-103-63.ec2.internal:31010]
> > >> >
> > >> > --
> > >> > Nitin Pawar
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Nitin Pawar
> > >
> >
> >
> >
> > --
> > Nitin Pawar
> >
>



-- 
Nitin Pawar

Re: Drill error

[Khurram Faraaz]

Hi Nitin,

Can you please share the description of the table (i.e. column types) ?
Is this a parquet table or JSON ?
Also please share the version of Drill and the drillbit.log

Thanks,
Khurram

On Wed, Jun 27, 2018 at 9:45 AM, Nitin Pawar 
wrote:

> here are the details
>
> query:
>
> select Account Account,Name Name,CONCAT(DateString , string2)
> Merged_String from dfs.tmp.`/nitin/` t1
>
> There is no custom UDF in this query;
>
>
>
> On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar 
> wrote:
>
> > Hi Vitalii,
> >
> > Thanks for the description.
> > I will try to get the log for this but as this is happening in production
> > and drill log gets overwritten in our case very fast.
> >
> > Also I will get the query for the same while getting the log
> >
> > I will get the details and update the thread
> >
> > Thanks,
> > Nitin
> >
> > On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka <
> > vitalii.dira...@gmail.com> wrote:
> >
> >> Hi Nitin,
> >>
> >> It happens in the process of reallocation of the size of buffers in the
> >> memory.
> >> It isn't a User Exception, so it looks like a bug, if you get it in some
> >> existed plugin.
> >> But to say you exactly, please describe your case. What kind of query
> did
> >> you perform, any UDF's, which data source?
> >> Also logs can help.
> >>
> >> Thanks.
> >>
> >> Kind regards
> >> Vitalii
> >>
> >>
> >> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar 
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > Can someone help me understand below error? and how do I not let this
> >> > happen ??
> >> >
> >> > SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged buffer.
> >> >
> >> > Fragment 0:0
> >> >
> >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> >> > ip-10-0-103-63.ec2.internal:31010]
> >> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove
> >> unmanaged
> >> > buffer.
> >> >
> >> > Fragment 0:0
> >> >
> >> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> >> > ip-10-0-103-63.ec2.internal:31010]
> >> >
> >> > --
> >> > Nitin Pawar
> >> >
> >>
> >
> >
> >
> > --
> > Nitin Pawar
> >
>
>
>
> --
> Nitin Pawar
>

Re: Drill error

[Nitin Pawar]

here are the details

query:

select Account Account,Name Name,CONCAT(DateString , string2)
Merged_String from dfs.tmp.`/nitin/` t1

There is no custom UDF in this query;



On Wed, Jun 27, 2018 at 2:22 PM, Nitin Pawar 
wrote:

> Hi Vitalii,
>
> Thanks for the description.
> I will try to get the log for this but as this is happening in production
> and drill log gets overwritten in our case very fast.
>
> Also I will get the query for the same while getting the log
>
> I will get the details and update the thread
>
> Thanks,
> Nitin
>
> On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka <
> vitalii.dira...@gmail.com> wrote:
>
>> Hi Nitin,
>>
>> It happens in the process of reallocation of the size of buffers in the
>> memory.
>> It isn't a User Exception, so it looks like a bug, if you get it in some
>> existed plugin.
>> But to say you exactly, please describe your case. What kind of query did
>> you perform, any UDF's, which data source?
>> Also logs can help.
>>
>> Thanks.
>>
>> Kind regards
>> Vitalii
>>
>>
>> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar 
>> wrote:
>>
>> > Hi,
>> >
>> > Can someone help me understand below error? and how do I not let this
>> > happen ??
>> >
>> > SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged buffer.
>> >
>> > Fragment 0:0
>> >
>> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
>> > ip-10-0-103-63.ec2.internal:31010]
>> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove
>> unmanaged
>> > buffer.
>> >
>> > Fragment 0:0
>> >
>> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
>> > ip-10-0-103-63.ec2.internal:31010]
>> >
>> > --
>> > Nitin Pawar
>> >
>>
>
>
>
> --
> Nitin Pawar
>



-- 
Nitin Pawar

Re: Drill error

[Nitin Pawar]

Hi Vitalii,

Thanks for the description.
I will try to get the log for this but as this is happening in production
and drill log gets overwritten in our case very fast.

Also I will get the query for the same while getting the log

I will get the details and update the thread

Thanks,
Nitin

On Tue, Jun 26, 2018 at 7:32 PM, Vitalii Diravka 
wrote:

> Hi Nitin,
>
> It happens in the process of reallocation of the size of buffers in the
> memory.
> It isn't a User Exception, so it looks like a bug, if you get it in some
> existed plugin.
> But to say you exactly, please describe your case. What kind of query did
> you perform, any UDF's, which data source?
> Also logs can help.
>
> Thanks.
>
> Kind regards
> Vitalii
>
>
> On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar 
> wrote:
>
> > Hi,
> >
> > Can someone help me understand below error? and how do I not let this
> > happen ??
> >
> > SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged buffer.
> >
> > Fragment 0:0
> >
> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> > ip-10-0-103-63.ec2.internal:31010]
> > Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged
> > buffer.
> >
> > Fragment 0:0
> >
> > [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> > ip-10-0-103-63.ec2.internal:31010]
> >
> > --
> > Nitin Pawar
> >
>



-- 
Nitin Pawar

Re: Drill error

[Vitalii Diravka]

Hi Nitin,

It happens in the process of reallocation of the size of buffers in the
memory.
It isn't a User Exception, so it looks like a bug, if you get it in some
existed plugin.
But to say you exactly, please describe your case. What kind of query did
you perform, any UDF's, which data source?
Also logs can help.

Thanks.

Kind regards
Vitalii


On Tue, Jun 26, 2018 at 1:27 PM Nitin Pawar  wrote:

> Hi,
>
> Can someone help me understand below error? and how do I not let this
> happen ??
>
> SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged buffer.
>
> Fragment 0:0
>
> [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> ip-10-0-103-63.ec2.internal:31010]
> Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged
> buffer.
>
> Fragment 0:0
>
> [Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
> ip-10-0-103-63.ec2.internal:31010]
>
> --
> Nitin Pawar
>

Drill error

[Nitin Pawar]

Hi,

Can someone help me understand below error? and how do I not let this
happen ??

SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged buffer.

Fragment 0:0

[Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
ip-10-0-103-63.ec2.internal:31010]
Caused By: SYSTEM ERROR: IllegalStateException: Tried to remove unmanaged
buffer.

Fragment 0:0

[Error Id: bcd510f6-75ee-49a7-b723-7b35d8575623 on
ip-10-0-103-63.ec2.internal:31010]

-- 
Nitin Pawar

RE: Apache Drill- Error setting up remote intermediate fragment execution

[Surneni Tilak]

Hi Sorabh,

Thanks for your quick response. It resolved my problem .

Best regards, 
_
Tilak Surneni 

-Original Message-
From: Sorabh Hamirwasia [mailto:shamirwa...@mapr.com] 
Sent: Wednesday, April 11, 2018 6:09 PM
To: user@drill.apache.org
Subject: Re: Apache Drill- Error setting up remote intermediate fragment 
execution

Sent too soon.


Hi Surneni,

This looks to be an issue with your network setup. The address or hostname 
known to a Drillbit about another Drillbit is not resolved correctly while 
trying to make connection.


Nodes with failure: drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt


Can you please verify if above 2 nodes can ping each other using the hostname?


If not may be you can try to update the /etc/hosts file to provide a mapping 
between hostname and corresponding IP address.


with org.apache.drill.exec.rpc.RpcException: Command failed while establishing 
connection.  Failure type CONNECTION..
2018-04-11 07:26:17,705 [BitServer-6] ERROR o.a.d.e.rpc.ConnectionMultiListener 
- Failed to establish connection
java.util.concurrent.ExecutionException: 
java.nio.channels.UnresolvedAddressException
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) 
~[netty-common-4.0.48.Final.jar:4.0.48.Final]


Thanks,
Sorabh



From: Sorabh Hamirwasia
Sent: Wednesday, April 11, 2018 9:04 AM
To: user@drill.apache.org
Subject: Re: Apache Drill- Error setting up remote intermediate fragment 
execution


Hi Surneni,

This looks to be an issue with your network setup. The address or hostname 
known to a Drillbit about another Drillbit is not resolved correctly while 
trying to make connection.


Thanks,
Sorabh



From: Surneni Tilak <surneni.ti...@ericsson.com>
Sent: Wednesday, April 11, 2018 8:30 AM
To: user@drill.apache.org
Subject: FW: Apache Drill- Error setting up remote intermediate fragment 
execution

Hi Team,

I have successfully launched  a Drill cluster on 3 docker containers with Zoo 
keeper  Quorum and accessing data from a remote node on which Drill is not 
installed . I am able to list all the drill bits from each node and  run  
queries on a single drill bit , But I am facing issue when I am trying to run a 
query which distributes the query to the other drill bits by creating the 
intermediate fragments.  I have gone through the documentation but It couldn't 
help. I got the below error message in Drill shell  and the detailed  error log 
mentioned below. Please help me on resolving this.

Error in Drill Shell:

Error: CONNECTION ERROR: Error setting up remote intermediate fragment execution

Nodes with failures drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt

[Error Id: 5d701a0d-988b-4845-83f6-b029ad6825a8 on 
drillmaster-76d6885c59-wpksz:31010] (state=,code=0)


Error Log:





2018-04-11 07:26:17,391 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,401 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 
1 threads. Time: 2ms total, 2.798853ms avg, 2ms max.
2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute d 1 out of 1 using 
1 threads. Earliest start: 0.699000 ?s, Latest start: 0.699000 ?s, Average 
start: 0.699000 ?s .
2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pr uning, pruning 
class: 
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.p.l.partition.PruneScanRule - No partition columns a re projected from 
the scan..continue. Total pruning elapsed time: 0 m

Re: Apache Drill- Error setting up remote intermediate fragment execution

[Sorabh Hamirwasia]

Sent too soon.


Hi Surneni,

This looks to be an issue with your network setup. The address or hostname 
known to a Drillbit about another Drillbit is not resolved correctly while 
trying to make connection.


Nodes with failure: drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt


Can you please verify if above 2 nodes can ping each other using the hostname?


If not may be you can try to update the /etc/hosts file to provide a mapping 
between hostname and corresponding IP address.


with org.apache.drill.exec.rpc.RpcException: Command failed while establishing 
connection.  Failure type CONNECTION..
2018-04-11 07:26:17,705 [BitServer-6] ERROR o.a.d.e.rpc.ConnectionMultiListener 
- Failed to establish connection
java.util.concurrent.ExecutionException: 
java.nio.channels.UnresolvedAddressException
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) 
~[netty-common-4.0.48.Final.jar:4.0.48.Final]


Thanks,
Sorabh



From: Sorabh Hamirwasia
Sent: Wednesday, April 11, 2018 9:04 AM
To: user@drill.apache.org
Subject: Re: Apache Drill- Error setting up remote intermediate fragment 
execution


Hi Surneni,

This looks to be an issue with your network setup. The address or hostname 
known to a Drillbit about another Drillbit is not resolved correctly while 
trying to make connection.


Thanks,
Sorabh



From: Surneni Tilak <surneni.ti...@ericsson.com>
Sent: Wednesday, April 11, 2018 8:30 AM
To: user@drill.apache.org
Subject: FW: Apache Drill- Error setting up remote intermediate fragment 
execution

Hi Team,

I have successfully launched  a Drill cluster on 3 docker containers with Zoo 
keeper  Quorum and accessing data from a remote node on which Drill is not 
installed . I am able to list all the drill bits from each node and  run  
queries on a single drill bit , But I am facing issue when I am trying to run a 
query which distributes the query to the other drill bits by creating the 
intermediate fragments.  I have gone through the documentation but It couldn't 
help. I got the below error message in Drill shell  and the detailed  error log 
mentioned below. Please help me on resolving this.

Error in Drill Shell:

Error: CONNECTION ERROR: Error setting up remote intermediate fragment execution

Nodes with failures drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt

[Error Id: 5d701a0d-988b-4845-83f6-b029ad6825a8 on 
drillmaster-76d6885c59-wpksz:31010] (state=,code=0)


Error Log:





2018-04-11 07:26:17,391 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,401 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute
d 1 out of 1 using 1 threads. Time: 2ms total, 2.798853ms avg, 2ms max.
2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute
d 1 out of 1 using 1 threads. Earliest start: 0.699000 ?s, Latest start: 
0.699000 ?s, Average start: 0.699000 ?s .
2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pr
uning, pruning class: 
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.p.l.partition.PruneScanRule - No partition columns a
re projected from the scan..continue. Total pruning elapsed time: 0 ms
2018-04-11 07:26:17,611 [BitServer-3] ERROR o.a.d.e.rpc.ConnectionMultiListener 
- Failed to establish connection
java.util.concurrent.ExecutionException: 
java.nio.channels.UnresolvedAddressException
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) 
~[netty-common-4.0.48.Final.jar:4.0.48

Re: Apache Drill- Error setting up remote intermediate fragment execution

[Sorabh Hamirwasia]

Hi Surneni,

This looks to be an issue with your network setup. The address or hostname 
known to a Drillbit about another Drillbit is not resolved correctly while 
trying to make connection.


Thanks,
Sorabh



From: Surneni Tilak <surneni.ti...@ericsson.com>
Sent: Wednesday, April 11, 2018 8:30 AM
To: user@drill.apache.org
Subject: FW: Apache Drill- Error setting up remote intermediate fragment 
execution

Hi Team,

I have successfully launched  a Drill cluster on 3 docker containers with Zoo 
keeper  Quorum and accessing data from a remote node on which Drill is not 
installed . I am able to list all the drill bits from each node and  run  
queries on a single drill bit , But I am facing issue when I am trying to run a 
query which distributes the query to the other drill bits by creating the 
intermediate fragments.  I have gone through the documentation but It couldn't 
help. I got the below error message in Drill shell  and the detailed  error log 
mentioned below. Please help me on resolving this.

Error in Drill Shell:

Error: CONNECTION ERROR: Error setting up remote intermediate fragment execution

Nodes with failures drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt

[Error Id: 5d701a0d-988b-4845-83f6-b029ad6825a8 on 
drillmaster-76d6885c59-wpksz:31010] (state=,code=0)


Error Log:





2018-04-11 07:26:17,391 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,401 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute
d 1 out of 1 using 1 threads. Time: 2ms total, 2.798853ms avg, 2ms max.
2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute
d 1 out of 1 using 1 threads. Earliest start: 0.699000 ?s, Latest start: 
0.699000 ?s, Average start: 0.699000 ?s .
2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pr
uning, pruning class: 
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.p.l.partition.PruneScanRule - No partition columns a
re projected from the scan..continue. Total pruning elapsed time: 0 ms
2018-04-11 07:26:17,611 [BitServer-3] ERROR o.a.d.e.rpc.ConnectionMultiListener 
- Failed to establish connection
java.util.concurrent.ExecutionException: 
java.nio.channels.UnresolvedAddressException
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) 
~[netty-common-4.0.48.Final.jar:4.0.48.Final]
at 
org.apache.drill.exec.rpc.ConnectionMultiListener$ConnectionHandler.operationComplete(ConnectionMultiListener.java:89)
 [drill-
rpc-1.12.0.jar:1.12.0]
at 
org.apache.drill.exec.rpc.ConnectionMultiListener$ConnectionHandler.operationComplete(ConnectionMultiListener.java:76)
 [drill-
rpc-1.12.0.jar:1.12.0]
at 
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
 [netty-common-4.0.48.Final.jar:4.0.48.Final]
at 
io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
 [netty-common-4.0.48.Final.jar:4.0.48.Fina
l]
at 
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
 [netty-common-4.0.48.Final.jar:4.0.48.Final]
at 
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122) 
[netty-common-4.0.48.Final.jar:4.0.48.Final]
at 
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:241)
 [netty-transport-4.0.48.Final.j
ar:4.0.48.Final]
at 
io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1226)
 [netty-transport-4.0.48.Final.ja
r:4.0.48

FW: Apache Drill- Error setting up remote intermediate fragment execution

[Surneni Tilak]

Hi Team,

I have successfully launched  a Drill cluster on 3 docker containers with Zoo 
keeper  Quorum and accessing data from a remote node on which Drill is not 
installed . I am able to list all the drill bits from each node and  run  
queries on a single drill bit , But I am facing issue when I am trying to run a 
query which distributes the query to the other drill bits by creating the 
intermediate fragments.  I have gone through the documentation but It couldn't 
help. I got the below error message in Drill shell  and the detailed  error log 
mentioned below. Please help me on resolving this.

Error in Drill Shell:

Error: CONNECTION ERROR: Error setting up remote intermediate fragment execution

Nodes with failures drillslave-7dccb67657-9qv4l, drillslave-7dccb67657-rhfgt

[Error Id: 5d701a0d-988b-4845-83f6-b029ad6825a8 on 
drillmaster-76d6885c59-wpksz:31010] (state=,code=0)


Error Log:





2018-04-11 07:26:17,391 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,392 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,401 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.exec.store.dfs.FileSelection - FileSelection.getStatus
es() took 0 ms, numFiles: 1
2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute
d 1 out of 1 using 1 threads. Time: 2ms total, 2.798853ms avg, 2ms max.
2018-04-11 07:26:17,404 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.s.schedule.BlockMapBuilder - Get block maps: Execute
d 1 out of 1 using 1 threads. Earliest start: 0.699000 ?s, Latest start: 
0.699000 ?s, Average start: 0.699000 ?s .
2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.p.l.partition.PruneScanRule - Beginning partition pr
uning, pruning class: 
org.apache.drill.exec.planner.logical.partition.PruneScanRule$DirPruneScanFilterOnScanRule
2018-04-11 07:26:17,408 [25324766-53a8-f988-20b4-65bbe0aa6cf9:foreman] INFO  
o.a.d.e.p.l.partition.PruneScanRule - No partition columns a
re projected from the scan..continue. Total pruning elapsed time: 0 ms
2018-04-11 07:26:17,611 [BitServer-3] ERROR o.a.d.e.rpc.ConnectionMultiListener 
- Failed to establish connection
java.util.concurrent.ExecutionException: 
java.nio.channels.UnresolvedAddressException
at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54) 
~[netty-common-4.0.48.Final.jar:4.0.48.Final]
at 
org.apache.drill.exec.rpc.ConnectionMultiListener$ConnectionHandler.operationComplete(ConnectionMultiListener.java:89)
 [drill-
rpc-1.12.0.jar:1.12.0]
at 
org.apache.drill.exec.rpc.ConnectionMultiListener$ConnectionHandler.operationComplete(ConnectionMultiListener.java:76)
 [drill-
rpc-1.12.0.jar:1.12.0]
at 
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
 [netty-common-4.0.48.Final.jar:4.0.48.Final]
at 
io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481)
 [netty-common-4.0.48.Final.jar:4.0.48.Fina
l]
at 
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
 [netty-common-4.0.48.Final.jar:4.0.48.Final]
at 
io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122) 
[netty-common-4.0.48.Final.jar:4.0.48.Final]
at 
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.connect(AbstractNioChannel.java:241)
 [netty-transport-4.0.48.Final.j
ar:4.0.48.Final]
at 
io.netty.channel.DefaultChannelPipeline$HeadContext.connect(DefaultChannelPipeline.java:1226)
 [netty-transport-4.0.48.Final.ja
r:4.0.48.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.invokeConnect(AbstractChannelHandlerContext.java:539)
 [netty-transport-4.0.48.F
inal.jar:4.0.48.Final]
at 
io.netty.channel.AbstractChannelHandlerContext.connect(AbstractChannelHandlerContext.java:524)
 [netty-transport-4.0.48.Final.j
ar:4.0.48.Final]
at 
io.netty.channel.ChannelOutboundHandlerAdapter.connect(ChannelOutboundHandlerAdapter.java:47)
 [netty-transport-4.0.48.Final.ja

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Akshay Joshi]

Hi Arjun,

I tried this and still getting the same error.

Thanks,
Akshay
On Wed, 1 Nov 2017 at 12:30 PM, Arjun kr <arjun...@outlook.com> wrote:

>
> Noticed that you are specifying property
> 'drill.exec.sys.store.provider.local.path' inside 'drill.exec' section. Can
> you try correcting it ? if it does not work , Please create JIRA with
> details as Kunal suggested.
>
> Shared entry :
>
> 3. drill-override.conf file
> drill.exec: {
>   cluster-id: "drillbits1",
>   drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
> apache-drill-1.11.0",
>   zk.connect: "<>:<"
> }
>
> Correct it as given below.
>
> drill.exec: {
>   cluster-id: "drillbits1",
>   sys.store.provider.local.path="/u01/bdcsce/drill/
> apache-drill-1.11.0",
>   zk.connect: "<>:<"
> }
>
> Thanks,
>
> Arjun
>
> ____
> From: Arjun kr <arjun...@outlook.com>
> Sent: Wednesday, November 1, 2017 11:20 AM
> To: user@drill.apache.org
> Subject: Re: Apache drill : Error while creating storage plugin for Oracle
> DB
>
> Hi Akshay,
>
>
> Just wondering if the issue is only with Oracle storage plugin. Are you
> able to create storage plugin other than oracle , say dfs or cp ?
>
>
> Thanks,
>
>
> Arjun
>
> 
> From: Kunal Khatua <kkha...@mapr.com>
> Sent: Wednesday, November 1, 2017 11:12 AM
> To: user@drill.apache.org
> Subject: RE: Apache drill : Error while creating storage plugin for Oracle
> DB
>
> Could you file a JIRA for this? I see the potential sources where the bug
> originated and there doesn’t seem to be sufficient logging, so it gets hard
> to narrow it down.
>
> -Original Message-
> From: Akshay Joshi [mailto:joshiakshay0...@gmail.com]
> Sent: Tuesday, October 31, 2017 10:04 PM
> To: user@drill.apache.org
> Subject: Re: Apache drill : Error while creating storage plugin for Oracle
> DB
>
> Hi Arjun/Kunal,
>
> Just to add to my previous mail I am using *Apache Drill version 1.11.0.*
>
> Thanks,
> Akshay
>
> On Wed, Nov 1, 2017 at 10:29 AM, Akshay Joshi <joshiakshay0...@gmail.com>
> wrote:
>
> > Hi Arjun/Kunal,
> >
> > Error message is not getting logged in drillbit.out  as well once
> "create"
> > button is clicked. None of the log files is getting updated in drill UI.
> >
> > Tried with curl command and getting the same error *(unable to create/
> > update storage) *without much stacktrace about error. PFB more details
> > about curl command and o/p. HTTPS is not enabled.
> > Just to add I am able to connect my oracle DB using sqldeveloper client.
> >
> > *COMMAND*
> > curl -v -X POST -H "Content-Type: application/json" -d
> > '{"name":"oracle1",
> > "config": {"type": "jdbc", "enabled": true,"driver": "oracle.dbc.driver.
> > OracleDriver",
> > "url":"jdbc:oracle:thin:/@ > server>:/" }}' http://<>:
> > 8047/storage/query.json
> >
> > *CURL O/P*
> > * About to connect() to <> port 8047 (#0)
> > *   Trying <>... connected
> > * Connected to <> (<>) port 8047
> > (#0)
> > > POST /storage/query.json HTTP/1.1
> > > User-Agent: curl/7.19.7 <>
> > > Host: <>:8047
> > > Accept: */*
> > > Content-Type: application/json
> > > Content-Length: 179
> > >
> > < HTTP/1.1 200 OK
> > < Content-Type: application/json
> > < Content-Length: 59
> > < Server: Jetty(9.1.5.v20140505)
> > <
> > {
> >   "result" : "error (unable to create/ update storage)"
> > * Connection #0 to host <> left intact
> > * Closing connection #0
> > }
> >
> > Kindly let me know if any other thoughts.
> >
> > Thanks,
> > Akshay
> >
> >
> > On Wed, Nov 1, 2017 at 12:20 AM, Arjun kr <arjun...@outlook.com> wrote:
> >
> >>
> >> Can you try with CURL as well ?
> >>
> >> curl -v -X POST -H "Content-Type: application/json" -d
> >> '{"name":"oracle1", "config": {"type": "jdbc", "enabled": true,"driver":
> >> "oracle.jdbc.driver.OracleDriver",
> >> "url":"jdbc:oracle:thin:/@ >> server>:/" }}' http://<
> >> drill_server>:8047/storage/oracle1.json
> >>
> >> Pleas

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Arjun kr]

Noticed that you are specifying property 
'drill.exec.sys.store.provider.local.path' inside 'drill.exec' section. Can you 
try correcting it ? if it does not work , Please create JIRA with details as 
Kunal suggested.

Shared entry :

3. drill-override.conf file
drill.exec: {
  cluster-id: "drillbits1",
  drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
apache-drill-1.11.0",
  zk.connect: "<>:<"
}

Correct it as given below.

drill.exec: {
  cluster-id: "drillbits1",
  sys.store.provider.local.path="/u01/bdcsce/drill/
apache-drill-1.11.0",
  zk.connect: "<>:<"
}

Thanks,

Arjun


From: Arjun kr <arjun...@outlook.com>
Sent: Wednesday, November 1, 2017 11:20 AM
To: user@drill.apache.org
Subject: Re: Apache drill : Error while creating storage plugin for Oracle DB

Hi Akshay,


Just wondering if the issue is only with Oracle storage plugin. Are you able to 
create storage plugin other than oracle , say dfs or cp ?


Thanks,


Arjun


From: Kunal Khatua <kkha...@mapr.com>
Sent: Wednesday, November 1, 2017 11:12 AM
To: user@drill.apache.org
Subject: RE: Apache drill : Error while creating storage plugin for Oracle DB

Could you file a JIRA for this? I see the potential sources where the bug 
originated and there doesn’t seem to be sufficient logging, so it gets hard to 
narrow it down.

-Original Message-
From: Akshay Joshi [mailto:joshiakshay0...@gmail.com]
Sent: Tuesday, October 31, 2017 10:04 PM
To: user@drill.apache.org
Subject: Re: Apache drill : Error while creating storage plugin for Oracle DB

Hi Arjun/Kunal,

Just to add to my previous mail I am using *Apache Drill version 1.11.0.*

Thanks,
Akshay

On Wed, Nov 1, 2017 at 10:29 AM, Akshay Joshi <joshiakshay0...@gmail.com>
wrote:

> Hi Arjun/Kunal,
>
> Error message is not getting logged in drillbit.out  as well once "create"
> button is clicked. None of the log files is getting updated in drill UI.
>
> Tried with curl command and getting the same error *(unable to create/
> update storage) *without much stacktrace about error. PFB more details
> about curl command and o/p. HTTPS is not enabled.
> Just to add I am able to connect my oracle DB using sqldeveloper client.
>
> *COMMAND*
> curl -v -X POST -H "Content-Type: application/json" -d
> '{"name":"oracle1",
> "config": {"type": "jdbc", "enabled": true,"driver": "oracle.dbc.driver.
> OracleDriver",
> "url":"jdbc:oracle:thin:/@ server>:/" }}' http://<>:
> 8047/storage/query.json
>
> *CURL O/P*
> * About to connect() to <> port 8047 (#0)
> *   Trying <>... connected
> * Connected to <> (<>) port 8047
> (#0)
> > POST /storage/query.json HTTP/1.1
> > User-Agent: curl/7.19.7 <>
> > Host: <>:8047
> > Accept: */*
> > Content-Type: application/json
> > Content-Length: 179
> >
> < HTTP/1.1 200 OK
> < Content-Type: application/json
> < Content-Length: 59
> < Server: Jetty(9.1.5.v20140505)
> <
> {
>   "result" : "error (unable to create/ update storage)"
> * Connection #0 to host <> left intact
> * Closing connection #0
> }
>
> Kindly let me know if any other thoughts.
>
> Thanks,
> Akshay
>
>
> On Wed, Nov 1, 2017 at 12:20 AM, Arjun kr <arjun...@outlook.com> wrote:
>
>>
>> Can you try with CURL as well ?
>>
>> curl -v -X POST -H "Content-Type: application/json" -d
>> '{"name":"oracle1", "config": {"type": "jdbc", "enabled": true,"driver":
>> "oracle.jdbc.driver.OracleDriver",
>> "url":"jdbc:oracle:thin:/@> server>:/" }}' http://<
>> drill_server>:8047/storage/oracle1.json
>>
>> Please refer to this blog for curl command if you have HTTPS enabled.
>>
>> http://www.openkb.info/2016/02/how-to-use-rest-api-to-submit-queries.
>> html
>>
>> Also assume that you are able to connect with this connection URL
>> using any other jdbc application.
>>
>> Thanks,
>>
>> Arjun
>> 
>> From: Kunal Khatua <kkha...@mapr.com>
>> Sent: Tuesday, October 31, 2017 11:57 PM
>> To: user@drill.apache.org
>> Subject: RE: Apache drill : Error while creating storage plugin for
>> Oracle DB
>>
>> There are other logs that might be reporting the error.
>>
>> Look at the other logs in the Drill UI ... one of them that can carry
>> more information would b

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Akshay Joshi]

Hi Arjun/Kunal,

Yes I am able to create storage plugins other than oracle.
I have created plugin for Alluxio. dfs and cp were default plugins.
As well I am able to trigger queries using these plugins.

Thanks,
Akshay

On Wed, 1 Nov 2017 at 11:20 AM, Arjun kr <arjun...@outlook.com> wrote:

> Hi Akshay,
>
>
> Just wondering if the issue is only with Oracle storage plugin. Are you
> able to create storage plugin other than oracle , say dfs or cp ?
>
>
> Thanks,
>
>
> Arjun
>
> 
> From: Kunal Khatua <kkha...@mapr.com>
> Sent: Wednesday, November 1, 2017 11:12 AM
> To: user@drill.apache.org
> Subject: RE: Apache drill : Error while creating storage plugin for Oracle
> DB
>
> Could you file a JIRA for this? I see the potential sources where the bug
> originated and there doesn’t seem to be sufficient logging, so it gets hard
> to narrow it down.
>
> -Original Message-
> From: Akshay Joshi [mailto:joshiakshay0...@gmail.com]
> Sent: Tuesday, October 31, 2017 10:04 PM
> To: user@drill.apache.org
> Subject: Re: Apache drill : Error while creating storage plugin for Oracle
> DB
>
> Hi Arjun/Kunal,
>
> Just to add to my previous mail I am using *Apache Drill version 1.11.0.*
>
> Thanks,
> Akshay
>
> On Wed, Nov 1, 2017 at 10:29 AM, Akshay Joshi <joshiakshay0...@gmail.com>
> wrote:
>
> > Hi Arjun/Kunal,
> >
> > Error message is not getting logged in drillbit.out  as well once
> "create"
> > button is clicked. None of the log files is getting updated in drill UI.
> >
> > Tried with curl command and getting the same error *(unable to create/
> > update storage) *without much stacktrace about error. PFB more details
> > about curl command and o/p. HTTPS is not enabled.
> > Just to add I am able to connect my oracle DB using sqldeveloper client.
> >
> > *COMMAND*
> > curl -v -X POST -H "Content-Type: application/json" -d
> > '{"name":"oracle1",
> > "config": {"type": "jdbc", "enabled": true,"driver": "oracle.dbc.driver.
> > OracleDriver",
> > "url":"jdbc:oracle:thin:/@ > server>:/" }}' http://<>:
> > 8047/storage/query.json
> >
> > *CURL O/P*
> > * About to connect() to <> port 8047 (#0)
> > *   Trying <>... connected
> > * Connected to <> (<>) port 8047
> > (#0)
> > > POST /storage/query.json HTTP/1.1
> > > User-Agent: curl/7.19.7 <>
> > > Host: <>:8047
> > > Accept: */*
> > > Content-Type: application/json
> > > Content-Length: 179
> > >
> > < HTTP/1.1 200 OK
> > < Content-Type: application/json
> > < Content-Length: 59
> > < Server: Jetty(9.1.5.v20140505)
> > <
> > {
> >   "result" : "error (unable to create/ update storage)"
> > * Connection #0 to host <> left intact
> > * Closing connection #0
> > }
> >
> > Kindly let me know if any other thoughts.
> >
> > Thanks,
> > Akshay
> >
> >
> > On Wed, Nov 1, 2017 at 12:20 AM, Arjun kr <arjun...@outlook.com> wrote:
> >
> >>
> >> Can you try with CURL as well ?
> >>
> >> curl -v -X POST -H "Content-Type: application/json" -d
> >> '{"name":"oracle1", "config": {"type": "jdbc", "enabled": true,"driver":
> >> "oracle.jdbc.driver.OracleDriver",
> >> "url":"jdbc:oracle:thin:/@ >> server>:/" }}' http://<
> >> drill_server>:8047/storage/oracle1.json
> >>
> >> Please refer to this blog for curl command if you have HTTPS enabled.
> >>
> >> http://www.openkb.info/2016/02/how-to-use-rest-api-to-submit-queries.
> >> html
> >>
> >> Also assume that you are able to connect with this connection URL
> >> using any other jdbc application.
> >>
> >> Thanks,
> >>
> >> Arjun
> >> 
> >> From: Kunal Khatua <kkha...@mapr.com>
> >> Sent: Tuesday, October 31, 2017 11:57 PM
> >> To: user@drill.apache.org
> >> Subject: RE: Apache drill : Error while creating storage plugin for
> >> Oracle DB
> >>
> >> There are other logs that might be reporting the error.
> >>
> >> Look at the other logs in the Drill UI ... one of them that can carry
> >> more information would

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Arjun kr]

Hi Akshay,


Just wondering if the issue is only with Oracle storage plugin. Are you able to 
create storage plugin other than oracle , say dfs or cp ?


Thanks,


Arjun


From: Kunal Khatua <kkha...@mapr.com>
Sent: Wednesday, November 1, 2017 11:12 AM
To: user@drill.apache.org
Subject: RE: Apache drill : Error while creating storage plugin for Oracle DB

Could you file a JIRA for this? I see the potential sources where the bug 
originated and there doesn’t seem to be sufficient logging, so it gets hard to 
narrow it down.

-Original Message-
From: Akshay Joshi [mailto:joshiakshay0...@gmail.com]
Sent: Tuesday, October 31, 2017 10:04 PM
To: user@drill.apache.org
Subject: Re: Apache drill : Error while creating storage plugin for Oracle DB

Hi Arjun/Kunal,

Just to add to my previous mail I am using *Apache Drill version 1.11.0.*

Thanks,
Akshay

On Wed, Nov 1, 2017 at 10:29 AM, Akshay Joshi <joshiakshay0...@gmail.com>
wrote:

> Hi Arjun/Kunal,
>
> Error message is not getting logged in drillbit.out  as well once "create"
> button is clicked. None of the log files is getting updated in drill UI.
>
> Tried with curl command and getting the same error *(unable to create/
> update storage) *without much stacktrace about error. PFB more details
> about curl command and o/p. HTTPS is not enabled.
> Just to add I am able to connect my oracle DB using sqldeveloper client.
>
> *COMMAND*
> curl -v -X POST -H "Content-Type: application/json" -d
> '{"name":"oracle1",
> "config": {"type": "jdbc", "enabled": true,"driver": "oracle.dbc.driver.
> OracleDriver",
> "url":"jdbc:oracle:thin:/@ server>:/" }}' http://<>:
> 8047/storage/query.json
>
> *CURL O/P*
> * About to connect() to <> port 8047 (#0)
> *   Trying <>... connected
> * Connected to <> (<>) port 8047
> (#0)
> > POST /storage/query.json HTTP/1.1
> > User-Agent: curl/7.19.7 <>
> > Host: <>:8047
> > Accept: */*
> > Content-Type: application/json
> > Content-Length: 179
> >
> < HTTP/1.1 200 OK
> < Content-Type: application/json
> < Content-Length: 59
> < Server: Jetty(9.1.5.v20140505)
> <
> {
>   "result" : "error (unable to create/ update storage)"
> * Connection #0 to host <> left intact
> * Closing connection #0
> }
>
> Kindly let me know if any other thoughts.
>
> Thanks,
> Akshay
>
>
> On Wed, Nov 1, 2017 at 12:20 AM, Arjun kr <arjun...@outlook.com> wrote:
>
>>
>> Can you try with CURL as well ?
>>
>> curl -v -X POST -H "Content-Type: application/json" -d
>> '{"name":"oracle1", "config": {"type": "jdbc", "enabled": true,"driver":
>> "oracle.jdbc.driver.OracleDriver",
>> "url":"jdbc:oracle:thin:/@> server>:/" }}' http://<
>> drill_server>:8047/storage/oracle1.json
>>
>> Please refer to this blog for curl command if you have HTTPS enabled.
>>
>> http://www.openkb.info/2016/02/how-to-use-rest-api-to-submit-queries.
>> html
>>
>> Also assume that you are able to connect with this connection URL
>> using any other jdbc application.
>>
>> Thanks,
>>
>> Arjun
>> 
>> From: Kunal Khatua <kkha...@mapr.com>
>> Sent: Tuesday, October 31, 2017 11:57 PM
>> To: user@drill.apache.org
>> Subject: RE: Apache drill : Error while creating storage plugin for
>> Oracle DB
>>
>> There are other logs that might be reporting the error.
>>
>> Look at the other logs in the Drill UI ... one of them that can carry
>> more information would be drillbit.out
>>
>> -Original Message-
>> From: Akshay Joshi [mailto:joshiakshay0...@gmail.com]
>> Sent: Tuesday, October 31, 2017 11:06 AM
>> To: user@drill.apache.org
>> Subject: Re: Apache drill : Error while creating storage plugin for
>> Oracle DB
>>
>> Hi Arjun,
>>
>> No error message getting logged in drillbit.log once I click on "create"
>> button.
>>
>> Thanks,
>> Akshay
>>
>> On Tue, Oct 31, 2017 at 11:19 PM, Arjun kr <arjun...@outlook.com> wrote:
>>
>> >
>> > Do you see any specific error message in drillbit.log ?
>> >
>> > Thanks,
>> >
>> > Arjun
>> >
>> > ____
>> > From: Akshay Joshi <joshiakshay0...@gmail.com>
>> > Sent:

RE: Apache drill : Error while creating storage plugin for Oracle DB

[Kunal Khatua]

Could you file a JIRA for this? I see the potential sources where the bug 
originated and there doesn’t seem to be sufficient logging, so it gets hard to 
narrow it down. 

-Original Message-
From: Akshay Joshi [mailto:joshiakshay0...@gmail.com] 
Sent: Tuesday, October 31, 2017 10:04 PM
To: user@drill.apache.org
Subject: Re: Apache drill : Error while creating storage plugin for Oracle DB

Hi Arjun/Kunal,

Just to add to my previous mail I am using *Apache Drill version 1.11.0.*

Thanks,
Akshay

On Wed, Nov 1, 2017 at 10:29 AM, Akshay Joshi <joshiakshay0...@gmail.com>
wrote:

> Hi Arjun/Kunal,
>
> Error message is not getting logged in drillbit.out  as well once "create"
> button is clicked. None of the log files is getting updated in drill UI.
>
> Tried with curl command and getting the same error *(unable to create/ 
> update storage) *without much stacktrace about error. PFB more details 
> about curl command and o/p. HTTPS is not enabled.
> Just to add I am able to connect my oracle DB using sqldeveloper client.
>
> *COMMAND*
> curl -v -X POST -H "Content-Type: application/json" -d 
> '{"name":"oracle1",
> "config": {"type": "jdbc", "enabled": true,"driver": "oracle.dbc.driver.
> OracleDriver",
> "url":"jdbc:oracle:thin:/@ server>:/" }}' http://<>:
> 8047/storage/query.json
>
> *CURL O/P*
> * About to connect() to <> port 8047 (#0)
> *   Trying <>... connected
> * Connected to <> (<>) port 8047 
> (#0)
> > POST /storage/query.json HTTP/1.1
> > User-Agent: curl/7.19.7 <>
> > Host: <>:8047
> > Accept: */*
> > Content-Type: application/json
> > Content-Length: 179
> >
> < HTTP/1.1 200 OK
> < Content-Type: application/json
> < Content-Length: 59
> < Server: Jetty(9.1.5.v20140505)
> <
> {
>   "result" : "error (unable to create/ update storage)"
> * Connection #0 to host <> left intact
> * Closing connection #0
> }
>
> Kindly let me know if any other thoughts.
>
> Thanks,
> Akshay
>
>
> On Wed, Nov 1, 2017 at 12:20 AM, Arjun kr <arjun...@outlook.com> wrote:
>
>>
>> Can you try with CURL as well ?
>>
>> curl -v -X POST -H "Content-Type: application/json" -d 
>> '{"name":"oracle1", "config": {"type": "jdbc", "enabled": true,"driver":
>> "oracle.jdbc.driver.OracleDriver", 
>> "url":"jdbc:oracle:thin:/@> server>:/" }}' http://<
>> drill_server>:8047/storage/oracle1.json
>>
>> Please refer to this blog for curl command if you have HTTPS enabled.
>>
>> http://www.openkb.info/2016/02/how-to-use-rest-api-to-submit-queries.
>> html
>>
>> Also assume that you are able to connect with this connection URL 
>> using any other jdbc application.
>>
>> Thanks,
>>
>> Arjun
>> 
>> From: Kunal Khatua <kkha...@mapr.com>
>> Sent: Tuesday, October 31, 2017 11:57 PM
>> To: user@drill.apache.org
>> Subject: RE: Apache drill : Error while creating storage plugin for 
>> Oracle DB
>>
>> There are other logs that might be reporting the error.
>>
>> Look at the other logs in the Drill UI ... one of them that can carry 
>> more information would be drillbit.out
>>
>> -Original Message-
>> From: Akshay Joshi [mailto:joshiakshay0...@gmail.com]
>> Sent: Tuesday, October 31, 2017 11:06 AM
>> To: user@drill.apache.org
>> Subject: Re: Apache drill : Error while creating storage plugin for 
>> Oracle DB
>>
>> Hi Arjun,
>>
>> No error message getting logged in drillbit.log once I click on "create"
>> button.
>>
>> Thanks,
>> Akshay
>>
>> On Tue, Oct 31, 2017 at 11:19 PM, Arjun kr <arjun...@outlook.com> wrote:
>>
>> >
>> > Do you see any specific error message in drillbit.log ?
>> >
>> > Thanks,
>> >
>> > Arjun
>> >
>> > ________
>> > From: Akshay Joshi <joshiakshay0...@gmail.com>
>> > Sent: Tuesday, October 31, 2017 11:10 PM
>> > To: user@drill.apache.org
>> > Subject: Re: Apache drill : Error while creating storage plugin for 
>> > Oracle DB
>> >
>> > Hi Arjun,
>> >
>> > Yes. I copied jdbc driver (ojdbc7.jar) to all the drill nodes and 
>> > restarted the drillbits on all nodes.
>> >
&

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Akshay Joshi]

Hi Arjun/Kunal,

Just to add to my previous mail I am using *Apache Drill version 1.11.0.*

Thanks,
Akshay

On Wed, Nov 1, 2017 at 10:29 AM, Akshay Joshi <joshiakshay0...@gmail.com>
wrote:

> Hi Arjun/Kunal,
>
> Error message is not getting logged in drillbit.out  as well once "create"
> button is clicked. None of the log files is getting updated in drill UI.
>
> Tried with curl command and getting the same error *(unable to create/
> update storage) *without much stacktrace about error. PFB more details
> about curl command and o/p. HTTPS is not enabled.
> Just to add I am able to connect my oracle DB using sqldeveloper client.
>
> *COMMAND*
> curl -v -X POST -H "Content-Type: application/json" -d '{"name":"oracle1",
> "config": {"type": "jdbc", "enabled": true,"driver": "oracle.dbc.driver.
> OracleDriver",
> "url":"jdbc:oracle:thin:/@ server>:/" }}' http://<>:
> 8047/storage/query.json
>
> *CURL O/P*
> * About to connect() to <> port 8047 (#0)
> *   Trying <>... connected
> * Connected to <> (<>) port 8047 (#0)
> > POST /storage/query.json HTTP/1.1
> > User-Agent: curl/7.19.7 <>
> > Host: <>:8047
> > Accept: */*
> > Content-Type: application/json
> > Content-Length: 179
> >
> < HTTP/1.1 200 OK
> < Content-Type: application/json
> < Content-Length: 59
> < Server: Jetty(9.1.5.v20140505)
> <
> {
>   "result" : "error (unable to create/ update storage)"
> * Connection #0 to host <> left intact
> * Closing connection #0
> }
>
> Kindly let me know if any other thoughts.
>
> Thanks,
> Akshay
>
>
> On Wed, Nov 1, 2017 at 12:20 AM, Arjun kr <arjun...@outlook.com> wrote:
>
>>
>> Can you try with CURL as well ?
>>
>> curl -v -X POST -H "Content-Type: application/json" -d
>> '{"name":"oracle1", "config": {"type": "jdbc", "enabled": true,"driver":
>> "oracle.jdbc.driver.OracleDriver", 
>> "url":"jdbc:oracle:thin:/@> server>:/" }}' http://<
>> drill_server>:8047/storage/oracle1.json
>>
>> Please refer to this blog for curl command if you have HTTPS enabled.
>>
>> http://www.openkb.info/2016/02/how-to-use-rest-api-to-submit-queries.html
>>
>> Also assume that you are able to connect with this connection URL using
>> any other jdbc application.
>>
>> Thanks,
>>
>> Arjun
>> 
>> From: Kunal Khatua <kkha...@mapr.com>
>> Sent: Tuesday, October 31, 2017 11:57 PM
>> To: user@drill.apache.org
>> Subject: RE: Apache drill : Error while creating storage plugin for
>> Oracle DB
>>
>> There are other logs that might be reporting the error.
>>
>> Look at the other logs in the Drill UI ... one of them that can carry
>> more information would be drillbit.out
>>
>> -Original Message-
>> From: Akshay Joshi [mailto:joshiakshay0...@gmail.com]
>> Sent: Tuesday, October 31, 2017 11:06 AM
>> To: user@drill.apache.org
>> Subject: Re: Apache drill : Error while creating storage plugin for
>> Oracle DB
>>
>> Hi Arjun,
>>
>> No error message getting logged in drillbit.log once I click on "create"
>> button.
>>
>> Thanks,
>> Akshay
>>
>> On Tue, Oct 31, 2017 at 11:19 PM, Arjun kr <arjun...@outlook.com> wrote:
>>
>> >
>> > Do you see any specific error message in drillbit.log ?
>> >
>> > Thanks,
>> >
>> > Arjun
>> >
>> > ________
>> > From: Akshay Joshi <joshiakshay0...@gmail.com>
>> > Sent: Tuesday, October 31, 2017 11:10 PM
>> > To: user@drill.apache.org
>> > Subject: Re: Apache drill : Error while creating storage plugin for
>> > Oracle DB
>> >
>> > Hi Arjun,
>> >
>> > Yes. I copied jdbc driver (ojdbc7.jar) to all the drill nodes and
>> > restarted the drillbits on all nodes.
>> >
>> > Thanks and Regards,
>> > Akshay
>> > On Tue, 31 Oct 2017 at 10:30 PM, Arjun kr <arjun...@outlook.com> wrote:
>> >
>> > > Hi Akshay,
>> > >
>> > >
>> > > Did you copy jdbc driver to all the drill nodes and restarted
>> drillbits ?
>> > >
>> > >
>> > > Thanks,
>&g

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Akshay Joshi]

Hi Arjun/Kunal,

Error message is not getting logged in drillbit.out  as well once "create"
button is clicked. None of the log files is getting updated in drill UI.

Tried with curl command and getting the same error *(unable to create/
update storage) *without much stacktrace about error. PFB more details
about curl command and o/p. HTTPS is not enabled.
Just to add I am able to connect my oracle DB using sqldeveloper client.

*COMMAND*
curl -v -X POST -H "Content-Type: application/json" -d '{"name":"oracle1",
"config": {"type": "jdbc", "enabled": true,"driver":
"oracle.dbc.driver.OracleDriver",
"url":"jdbc:oracle:thin:/@:/"
}}' http://<>:8047/storage/query.json

*CURL O/P*
* About to connect() to <> port 8047 (#0)
*   Trying <>... connected
* Connected to <> (<>) port 8047 (#0)
> POST /storage/query.json HTTP/1.1
> User-Agent: curl/7.19.7 <>
> Host: <>:8047
> Accept: */*
> Content-Type: application/json
> Content-Length: 179
>
< HTTP/1.1 200 OK
< Content-Type: application/json
< Content-Length: 59
< Server: Jetty(9.1.5.v20140505)
<
{
  "result" : "error (unable to create/ update storage)"
* Connection #0 to host <> left intact
* Closing connection #0
}

Kindly let me know if any other thoughts.

Thanks,
Akshay


On Wed, Nov 1, 2017 at 12:20 AM, Arjun kr <arjun...@outlook.com> wrote:

>
> Can you try with CURL as well ?
>
> curl -v -X POST -H "Content-Type: application/json" -d '{"name":"oracle1",
> "config": {"type": "jdbc", "enabled": true,"driver": 
> "oracle.jdbc.driver.OracleDriver",
> "url":"jdbc:oracle:thin:/@ server>:/" }}' http://<
> drill_server>:8047/storage/oracle1.json
>
> Please refer to this blog for curl command if you have HTTPS enabled.
>
> http://www.openkb.info/2016/02/how-to-use-rest-api-to-submit-queries.html
>
> Also assume that you are able to connect with this connection URL using
> any other jdbc application.
>
> Thanks,
>
> Arjun
> 
> From: Kunal Khatua <kkha...@mapr.com>
> Sent: Tuesday, October 31, 2017 11:57 PM
> To: user@drill.apache.org
> Subject: RE: Apache drill : Error while creating storage plugin for Oracle
> DB
>
> There are other logs that might be reporting the error.
>
> Look at the other logs in the Drill UI ... one of them that can carry more
> information would be drillbit.out
>
> -Original Message-
> From: Akshay Joshi [mailto:joshiakshay0...@gmail.com]
> Sent: Tuesday, October 31, 2017 11:06 AM
> To: user@drill.apache.org
> Subject: Re: Apache drill : Error while creating storage plugin for Oracle
> DB
>
> Hi Arjun,
>
> No error message getting logged in drillbit.log once I click on "create"
> button.
>
> Thanks,
> Akshay
>
> On Tue, Oct 31, 2017 at 11:19 PM, Arjun kr <arjun...@outlook.com> wrote:
>
> >
> > Do you see any specific error message in drillbit.log ?
> >
> > Thanks,
> >
> > Arjun
> >
> > 
> > From: Akshay Joshi <joshiakshay0...@gmail.com>
> > Sent: Tuesday, October 31, 2017 11:10 PM
> > To: user@drill.apache.org
> > Subject: Re: Apache drill : Error while creating storage plugin for
> > Oracle DB
> >
> > Hi Arjun,
> >
> > Yes. I copied jdbc driver (ojdbc7.jar) to all the drill nodes and
> > restarted the drillbits on all nodes.
> >
> > Thanks and Regards,
> > Akshay
> > On Tue, 31 Oct 2017 at 10:30 PM, Arjun kr <arjun...@outlook.com> wrote:
> >
> > > Hi Akshay,
> > >
> > >
> > > Did you copy jdbc driver to all the drill nodes and restarted
> drillbits ?
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Arjun
> > >
> > >
> > > 
> > > From: Akshay Joshi <joshiakshay0...@gmail.com>
> > > Sent: Tuesday, October 31, 2017 9:46 PM
> > > To: user@drill.apache.org
> > > Subject: Apache drill : Error while creating storage plugin for
> > > Oracle DB
> > >
> > > Hi Team,
> > >
> > > I have configured drill on 4 node cluster and getting error while
> > creating
> > > storage plugin for Oracle DB.
> > >
> > > Error : Please retry: error (unable to create/ update storage)
> > >
> > > Oracle DB Version : *12.1.0.2.0*
> > >
> > > Following steps followed to create storage plugin:
> > > 1. copied ojdbc7.jar to jars/3rdparty folder 2. Storage plugin
> > > configuration {
> > >   "type": "jdbc",
> > >   "driver": "oracle.jdbc.driver.OracleDriver",
> > >   "url":
> > > "jdbc:oracle:thin:<>/<>@<>:<>/
> > ORCL",
> > >   "enabled": true
> > > }
> > > 3. drill-override.conf file
> > > drill.exec: {
> > >   cluster-id: "drillbits1",
> > >   drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
> > > apache-drill-1.11.0",
> > >   zk.connect: "<>:<"
> > > }
> > >
> > > Thanks and Regards,
> > > Akshay
> > >
> >
>

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Arjun kr]

Can you try with CURL as well ?

curl -v -X POST -H "Content-Type: application/json" -d '{"name":"oracle1", 
"config": {"type": "jdbc", "enabled": true,"driver": 
"oracle.jdbc.driver.OracleDriver", 
"url":"jdbc:oracle:thin:/@:/" }}' 
http://< drill_server>:8047/storage/oracle1.json

Please refer to this blog for curl command if you have HTTPS enabled.

http://www.openkb.info/2016/02/how-to-use-rest-api-to-submit-queries.html

Also assume that you are able to connect with this connection URL using any 
other jdbc application.

Thanks,

Arjun

From: Kunal Khatua <kkha...@mapr.com>
Sent: Tuesday, October 31, 2017 11:57 PM
To: user@drill.apache.org
Subject: RE: Apache drill : Error while creating storage plugin for Oracle DB

There are other logs that might be reporting the error.

Look at the other logs in the Drill UI ... one of them that can carry more 
information would be drillbit.out

-Original Message-
From: Akshay Joshi [mailto:joshiakshay0...@gmail.com]
Sent: Tuesday, October 31, 2017 11:06 AM
To: user@drill.apache.org
Subject: Re: Apache drill : Error while creating storage plugin for Oracle DB

Hi Arjun,

No error message getting logged in drillbit.log once I click on "create"
button.

Thanks,
Akshay

On Tue, Oct 31, 2017 at 11:19 PM, Arjun kr <arjun...@outlook.com> wrote:

>
> Do you see any specific error message in drillbit.log ?
>
> Thanks,
>
> Arjun
>
> ________
> From: Akshay Joshi <joshiakshay0...@gmail.com>
> Sent: Tuesday, October 31, 2017 11:10 PM
> To: user@drill.apache.org
> Subject: Re: Apache drill : Error while creating storage plugin for
> Oracle DB
>
> Hi Arjun,
>
> Yes. I copied jdbc driver (ojdbc7.jar) to all the drill nodes and
> restarted the drillbits on all nodes.
>
> Thanks and Regards,
> Akshay
> On Tue, 31 Oct 2017 at 10:30 PM, Arjun kr <arjun...@outlook.com> wrote:
>
> > Hi Akshay,
> >
> >
> > Did you copy jdbc driver to all the drill nodes and restarted drillbits ?
> >
> >
> > Thanks,
> >
> >
> > Arjun
> >
> >
> > 
> > From: Akshay Joshi <joshiakshay0...@gmail.com>
> > Sent: Tuesday, October 31, 2017 9:46 PM
> > To: user@drill.apache.org
> > Subject: Apache drill : Error while creating storage plugin for
> > Oracle DB
> >
> > Hi Team,
> >
> > I have configured drill on 4 node cluster and getting error while
> creating
> > storage plugin for Oracle DB.
> >
> > Error : Please retry: error (unable to create/ update storage)
> >
> > Oracle DB Version : *12.1.0.2.0*
> >
> > Following steps followed to create storage plugin:
> > 1. copied ojdbc7.jar to jars/3rdparty folder 2. Storage plugin
> > configuration {
> >   "type": "jdbc",
> >   "driver": "oracle.jdbc.driver.OracleDriver",
> >   "url":
> > "jdbc:oracle:thin:<>/<>@<>:<>/
> ORCL",
> >   "enabled": true
> > }
> > 3. drill-override.conf file
> > drill.exec: {
> >   cluster-id: "drillbits1",
> >   drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
> > apache-drill-1.11.0",
> >   zk.connect: "<>:<"
> > }
> >
> > Thanks and Regards,
> > Akshay
> >
>

RE: Apache drill : Error while creating storage plugin for Oracle DB

[Kunal Khatua]

There are other logs that might be reporting the error. 

Look at the other logs in the Drill UI ... one of them that can carry more 
information would be drillbit.out 

-Original Message-
From: Akshay Joshi [mailto:joshiakshay0...@gmail.com] 
Sent: Tuesday, October 31, 2017 11:06 AM
To: user@drill.apache.org
Subject: Re: Apache drill : Error while creating storage plugin for Oracle DB

Hi Arjun,

No error message getting logged in drillbit.log once I click on "create"
button.

Thanks,
Akshay

On Tue, Oct 31, 2017 at 11:19 PM, Arjun kr <arjun...@outlook.com> wrote:

>
> Do you see any specific error message in drillbit.log ?
>
> Thanks,
>
> Arjun
>
> 
> From: Akshay Joshi <joshiakshay0...@gmail.com>
> Sent: Tuesday, October 31, 2017 11:10 PM
> To: user@drill.apache.org
> Subject: Re: Apache drill : Error while creating storage plugin for 
> Oracle DB
>
> Hi Arjun,
>
> Yes. I copied jdbc driver (ojdbc7.jar) to all the drill nodes and 
> restarted the drillbits on all nodes.
>
> Thanks and Regards,
> Akshay
> On Tue, 31 Oct 2017 at 10:30 PM, Arjun kr <arjun...@outlook.com> wrote:
>
> > Hi Akshay,
> >
> >
> > Did you copy jdbc driver to all the drill nodes and restarted drillbits ?
> >
> >
> > Thanks,
> >
> >
> > Arjun
> >
> >
> > ________
> > From: Akshay Joshi <joshiakshay0...@gmail.com>
> > Sent: Tuesday, October 31, 2017 9:46 PM
> > To: user@drill.apache.org
> > Subject: Apache drill : Error while creating storage plugin for 
> > Oracle DB
> >
> > Hi Team,
> >
> > I have configured drill on 4 node cluster and getting error while
> creating
> > storage plugin for Oracle DB.
> >
> > Error : Please retry: error (unable to create/ update storage)
> >
> > Oracle DB Version : *12.1.0.2.0*
> >
> > Following steps followed to create storage plugin:
> > 1. copied ojdbc7.jar to jars/3rdparty folder 2. Storage plugin 
> > configuration {
> >   "type": "jdbc",
> >   "driver": "oracle.jdbc.driver.OracleDriver",
> >   "url": 
> > "jdbc:oracle:thin:<>/<>@<>:<>/
> ORCL",
> >   "enabled": true
> > }
> > 3. drill-override.conf file
> > drill.exec: {
> >   cluster-id: "drillbits1",
> >   drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
> > apache-drill-1.11.0",
> >   zk.connect: "<>:<"
> > }
> >
> > Thanks and Regards,
> > Akshay
> >
>

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Akshay Joshi]

Hi Arjun,

No error message getting logged in drillbit.log once I click on "create"
button.

Thanks,
Akshay

On Tue, Oct 31, 2017 at 11:19 PM, Arjun kr <arjun...@outlook.com> wrote:

>
> Do you see any specific error message in drillbit.log ?
>
> Thanks,
>
> Arjun
>
> 
> From: Akshay Joshi <joshiakshay0...@gmail.com>
> Sent: Tuesday, October 31, 2017 11:10 PM
> To: user@drill.apache.org
> Subject: Re: Apache drill : Error while creating storage plugin for Oracle
> DB
>
> Hi Arjun,
>
> Yes. I copied jdbc driver (ojdbc7.jar) to all the drill nodes and restarted
> the drillbits on all nodes.
>
> Thanks and Regards,
> Akshay
> On Tue, 31 Oct 2017 at 10:30 PM, Arjun kr <arjun...@outlook.com> wrote:
>
> > Hi Akshay,
> >
> >
> > Did you copy jdbc driver to all the drill nodes and restarted drillbits ?
> >
> >
> > Thanks,
> >
> >
> > Arjun
> >
> >
> > ________
> > From: Akshay Joshi <joshiakshay0...@gmail.com>
> > Sent: Tuesday, October 31, 2017 9:46 PM
> > To: user@drill.apache.org
> > Subject: Apache drill : Error while creating storage plugin for Oracle DB
> >
> > Hi Team,
> >
> > I have configured drill on 4 node cluster and getting error while
> creating
> > storage plugin for Oracle DB.
> >
> > Error : Please retry: error (unable to create/ update storage)
> >
> > Oracle DB Version : *12.1.0.2.0*
> >
> > Following steps followed to create storage plugin:
> > 1. copied ojdbc7.jar to jars/3rdparty folder
> > 2. Storage plugin configuration
> > {
> >   "type": "jdbc",
> >   "driver": "oracle.jdbc.driver.OracleDriver",
> >   "url": "jdbc:oracle:thin:<>/<>@<>:<>/
> ORCL",
> >   "enabled": true
> > }
> > 3. drill-override.conf file
> > drill.exec: {
> >   cluster-id: "drillbits1",
> >   drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
> > apache-drill-1.11.0",
> >   zk.connect: "<>:<"
> > }
> >
> > Thanks and Regards,
> > Akshay
> >
>

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Arjun kr]

Do you see any specific error message in drillbit.log ?

Thanks,

Arjun


From: Akshay Joshi <joshiakshay0...@gmail.com>
Sent: Tuesday, October 31, 2017 11:10 PM
To: user@drill.apache.org
Subject: Re: Apache drill : Error while creating storage plugin for Oracle DB

Hi Arjun,

Yes. I copied jdbc driver (ojdbc7.jar) to all the drill nodes and restarted
the drillbits on all nodes.

Thanks and Regards,
Akshay
On Tue, 31 Oct 2017 at 10:30 PM, Arjun kr <arjun...@outlook.com> wrote:

> Hi Akshay,
>
>
> Did you copy jdbc driver to all the drill nodes and restarted drillbits ?
>
>
> Thanks,
>
>
> Arjun
>
>
> 
> From: Akshay Joshi <joshiakshay0...@gmail.com>
> Sent: Tuesday, October 31, 2017 9:46 PM
> To: user@drill.apache.org
> Subject: Apache drill : Error while creating storage plugin for Oracle DB
>
> Hi Team,
>
> I have configured drill on 4 node cluster and getting error while creating
> storage plugin for Oracle DB.
>
> Error : Please retry: error (unable to create/ update storage)
>
> Oracle DB Version : *12.1.0.2.0*
>
> Following steps followed to create storage plugin:
> 1. copied ojdbc7.jar to jars/3rdparty folder
> 2. Storage plugin configuration
> {
>   "type": "jdbc",
>   "driver": "oracle.jdbc.driver.OracleDriver",
>   "url": "jdbc:oracle:thin:<>/<>@<>:<>/ORCL",
>   "enabled": true
> }
> 3. drill-override.conf file
> drill.exec: {
>   cluster-id: "drillbits1",
>   drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
> apache-drill-1.11.0",
>   zk.connect: "<>:<"
> }
>
> Thanks and Regards,
> Akshay
>

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Akshay Joshi]

Hi Arjun,

Yes. I copied jdbc driver (ojdbc7.jar) to all the drill nodes and restarted
the drillbits on all nodes.

Thanks and Regards,
Akshay
On Tue, 31 Oct 2017 at 10:30 PM, Arjun kr <arjun...@outlook.com> wrote:

> Hi Akshay,
>
>
> Did you copy jdbc driver to all the drill nodes and restarted drillbits ?
>
>
> Thanks,
>
>
> Arjun
>
>
> 
> From: Akshay Joshi <joshiakshay0...@gmail.com>
> Sent: Tuesday, October 31, 2017 9:46 PM
> To: user@drill.apache.org
> Subject: Apache drill : Error while creating storage plugin for Oracle DB
>
> Hi Team,
>
> I have configured drill on 4 node cluster and getting error while creating
> storage plugin for Oracle DB.
>
> Error : Please retry: error (unable to create/ update storage)
>
> Oracle DB Version : *12.1.0.2.0*
>
> Following steps followed to create storage plugin:
> 1. copied ojdbc7.jar to jars/3rdparty folder
> 2. Storage plugin configuration
> {
>   "type": "jdbc",
>   "driver": "oracle.jdbc.driver.OracleDriver",
>   "url": "jdbc:oracle:thin:<>/<>@<>:<>/ORCL",
>   "enabled": true
> }
> 3. drill-override.conf file
> drill.exec: {
>   cluster-id: "drillbits1",
>   drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
> apache-drill-1.11.0",
>   zk.connect: "<>:<"
> }
>
> Thanks and Regards,
> Akshay
>

Re: Apache drill : Error while creating storage plugin for Oracle DB

[Arjun kr]

Hi Akshay,


Did you copy jdbc driver to all the drill nodes and restarted drillbits ?


Thanks,


Arjun



From: Akshay Joshi <joshiakshay0...@gmail.com>
Sent: Tuesday, October 31, 2017 9:46 PM
To: user@drill.apache.org
Subject: Apache drill : Error while creating storage plugin for Oracle DB

Hi Team,

I have configured drill on 4 node cluster and getting error while creating
storage plugin for Oracle DB.

Error : Please retry: error (unable to create/ update storage)

Oracle DB Version : *12.1.0.2.0*

Following steps followed to create storage plugin:
1. copied ojdbc7.jar to jars/3rdparty folder
2. Storage plugin configuration
{
  "type": "jdbc",
  "driver": "oracle.jdbc.driver.OracleDriver",
  "url": "jdbc:oracle:thin:<>/<>@<>:<>/ORCL",
  "enabled": true
}
3. drill-override.conf file
drill.exec: {
  cluster-id: "drillbits1",
  drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
apache-drill-1.11.0",
  zk.connect: "<>:<"
}

Thanks and Regards,
Akshay

Apache drill : Error while creating storage plugin for Oracle DB

[Akshay Joshi]

Hi Team,

I have configured drill on 4 node cluster and getting error while creating
storage plugin for Oracle DB.

Error : Please retry: error (unable to create/ update storage)

Oracle DB Version : *12.1.0.2.0*

Following steps followed to create storage plugin:
1. copied ojdbc7.jar to jars/3rdparty folder
2. Storage plugin configuration
{
  "type": "jdbc",
  "driver": "oracle.jdbc.driver.OracleDriver",
  "url": "jdbc:oracle:thin:<>/<>@<>:<>/ORCL",
  "enabled": true
}
3. drill-override.conf file
drill.exec: {
  cluster-id: "drillbits1",
  drill.exec.sys.store.provider.local.path="/u01/bdcsce/drill/
apache-drill-1.11.0",
  zk.connect: "<>:<"
}

Thanks and Regards,
Akshay

RE: drill error connecting to Hbase

[Shai Shapira]

We re-install drill, with newer version ( 1.11) and played a bit in with the 
configuration using the Web access and made it work.

Thanks a lot for your help!!

Thanks,
Shai

-Original Message-
From: Dor Ben Dov 
Sent: Sunday, August 06, 2017 2:09 PM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

Hi Kunal,

I am assisting Shai with the drill, I followed you instructions but once I am 
running maven with the profile of cloudera aka 'cdh' I am receiving this
[dor@dor-fedora64 drill]$ mvn -U -DskipTests clean install -Pcdh [INFO] 
Scanning for projects...
Downloading: 
https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/apache/14/apache-14.pom
Downloading: http://conjars.org/repo/org/apache/apache/14/apache-14.pom
Downloading: http://repository.mapr.com/maven/org/apache/apache/14/apache-14.pom
Downloading: http://repo.dremio.com/release/org/apache/apache/14/apache-14.pom
Downloading: 
http://repository.mapr.com/nexus/content/repositories/drill/org/apache/apache/14/apache-14.pom
Downloading: 
https://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.apache.drill:drill-root:1.11.0: Could 
not transfer artifact org.apache:apache:pom:14 from/to cloudera 
(https://repository.cloudera.com/artifactory/cloudera-repos/): 
repository.cloudera.com: Name or service not known and 'parent.relativePath' 
points at wrong local POM @ line 15, column 11  @ [ERROR] The build could not 
read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.apache.drill:drill-root:1.11.0 
(/home/dor/Downloads/drill/pom.xml) has 1 error
[ERROR] Non-resolvable parent POM for org.apache.drill:drill-root:1.11.0: 
Could not transfer artifact org.apache:apache:pom:14 from/to cloudera 
(https://repository.cloudera.com/artifactory/cloudera-repos/): 
repository.cloudera.com: Name or service not known and 'parent.relativePath' 
points at wrong local POM @ line 15, column 11: Unknown host 
repository.cloudera.com: Name or service not known -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] 
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
[dor@dor-fedora64 drill]$


** I am using fedora 26 **

Regards,
Dor

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com]
Sent: יום ה 03 אוגוסט 2017 20:53
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

The failure appears to be coming from this:

Caused by: java.lang.IllegalAccessError: Class 
org/apache/hadoop/hbase/zookeeper/MetaTableLocator illegally accessing "package 
private" member of class com/google/common/base/Stopwatch


Scrolling up a bit, I noticed that during the startup, there is an error here:
2017-08-03 11:04:58,957 [main] WARN  o.a.drill.exec.util.GuavaPatcher - Unable 
to patch Guava classes.
javassist.CannotCompileException: by java.lang.LinkageError: 
com.google.common.base.Stopwatch

When you build your Drill package, you can specify a profile. 

https://github.com/apache/drill/blob/master/pom.xml#L953

You can choose the platform you need Drill for and build with that. This tells 
Maven to apply specific versions of some dependencies that will work.

e.g.
mvn -U -DskipTests clean install -P

If there is an issue, let us know the specs of the platform you are building 
against. It is possible that there might have been upgrades to the dependencies 
within these platforms that are not being reflected in the pon.xml.



-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com]
Sent: Thursday, August 03, 2017 1:13 AM
To: user@drill.apache.org; Kunal Khatua <kkha...@mapr.com>
Subject: RE: drill error connecting to Hbase

Attached the relevant part from the sqlline.log, Hope it helps


Thanks,
Shai


-Original Message-
From: Shai Shapira
Sent: Thursday, August 03, 2017 11:04 AM
To: kkha...@mapr.com
Cc: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

Hi,

My versions are:
Hbase - 1.2.0   
Hive - 1.1.0 

I'll send the complete stack trace.

Is Drill is so version sensitive?
Can I build a solution for production based on Drill? Or should I stick to what 
is coming with the Cloudera/Hortonworks distribution?

Thanks,
Shai


hbase shell
17/08/03 10:54:05 INFO Configuration.deprecation: hadoop.native.lib is 
deprecated. Instead, use io.native.lib.available HBase Shell; enter 
'help' for list of supported commands.
Type "exit" to leave the HBase Shell Version 1.2.0-cdh5.8.2, rUnknown, 
Sun Sep 11 11:52:54 PDT 2016


hive 

RE: drill error connecting to Hbase

[Dor Ben Dov]

Hi Kunal,

I am assisting Shai with the drill, I followed you instructions but once I am 
running maven with the profile of cloudera aka 'cdh' I am receiving this
[dor@dor-fedora64 drill]$ mvn -U -DskipTests clean install -Pcdh
[INFO] Scanning for projects...
Downloading: 
https://repository.cloudera.com/artifactory/cloudera-repos/org/apache/apache/14/apache-14.pom
Downloading: http://conjars.org/repo/org/apache/apache/14/apache-14.pom
Downloading: http://repository.mapr.com/maven/org/apache/apache/14/apache-14.pom
Downloading: http://repo.dremio.com/release/org/apache/apache/14/apache-14.pom
Downloading: 
http://repository.mapr.com/nexus/content/repositories/drill/org/apache/apache/14/apache-14.pom
Downloading: 
https://repo.maven.apache.org/maven2/org/apache/apache/14/apache-14.pom
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.apache.drill:drill-root:1.11.0: Could 
not transfer artifact org.apache:apache:pom:14 from/to cloudera 
(https://repository.cloudera.com/artifactory/cloudera-repos/): 
repository.cloudera.com: Name or service not known and 'parent.relativePath' 
points at wrong local POM @ line 15, column 11
 @ 
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project org.apache.drill:drill-root:1.11.0 
(/home/dor/Downloads/drill/pom.xml) has 1 error
[ERROR] Non-resolvable parent POM for org.apache.drill:drill-root:1.11.0: 
Could not transfer artifact org.apache:apache:pom:14 from/to cloudera 
(https://repository.cloudera.com/artifactory/cloudera-repos/): 
repository.cloudera.com: Name or service not known and 'parent.relativePath' 
points at wrong local POM @ line 15, column 11: Unknown host 
repository.cloudera.com: Name or service not known -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] 
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException
[dor@dor-fedora64 drill]$


** I am using fedora 26 **

Regards,
Dor

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com] 
Sent: יום ה 03 אוגוסט 2017 20:53
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

The failure appears to be coming from this:

Caused by: java.lang.IllegalAccessError: Class 
org/apache/hadoop/hbase/zookeeper/MetaTableLocator illegally accessing "package 
private" member of class com/google/common/base/Stopwatch


Scrolling up a bit, I noticed that during the startup, there is an error here:
2017-08-03 11:04:58,957 [main] WARN  o.a.drill.exec.util.GuavaPatcher - Unable 
to patch Guava classes.
javassist.CannotCompileException: by java.lang.LinkageError: 
com.google.common.base.Stopwatch

When you build your Drill package, you can specify a profile. 

https://github.com/apache/drill/blob/master/pom.xml#L953

You can choose the platform you need Drill for and build with that. This tells 
Maven to apply specific versions of some dependencies that will work.

e.g.
mvn -U -DskipTests clean install -P

If there is an issue, let us know the specs of the platform you are building 
against. It is possible that there might have been upgrades to the dependencies 
within these platforms that are not being reflected in the pon.xml.



-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Thursday, August 03, 2017 1:13 AM
To: user@drill.apache.org; Kunal Khatua <kkha...@mapr.com>
Subject: RE: drill error connecting to Hbase

Attached the relevant part from the sqlline.log, Hope it helps


Thanks,
Shai


-Original Message-
From: Shai Shapira 
Sent: Thursday, August 03, 2017 11:04 AM
To: kkha...@mapr.com
Cc: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

Hi,

My versions are:
Hbase - 1.2.0   
Hive - 1.1.0 

I'll send the complete stack trace.

Is Drill is so version sensitive?
Can I build a solution for production based on Drill? Or should I stick to what 
is coming with the Cloudera/Hortonworks distribution?

Thanks,
Shai


hbase shell
17/08/03 10:54:05 INFO Configuration.deprecation: hadoop.native.lib is 
deprecated. Instead, use io.native.lib.available HBase Shell; enter 
'help' for list of supported commands.
Type "exit" to leave the HBase Shell Version 1.2.0-cdh5.8.2, rUnknown, 
Sun Sep 11 11:52:54 PDT 2016


hive shell

Logging initialized using configuration in 
jar:file:/opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p1980.2070/jars/hive-common-1.1.0-cdh5.8.2.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com]

RE: drill error connecting to Hbase

[Shai Shapira]

Attached the relevant part from the sqlline.log, 
Hope it helps


Thanks,
Shai


-Original Message-
From: Shai Shapira 
Sent: Thursday, August 03, 2017 11:04 AM
To: kkha...@mapr.com
Cc: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

Hi,

My versions are:
Hbase - 1.2.0   
Hive - 1.1.0 

I'll send the complete stack trace.

Is Drill is so version sensitive?
Can I build a solution for production based on Drill? Or should I stick to what 
is coming with the Cloudera/Hortonworks distribution?

Thanks,
Shai


hbase shell
17/08/03 10:54:05 INFO Configuration.deprecation: hadoop.native.lib is 
deprecated. Instead, use io.native.lib.available HBase Shell; enter 
'help' for list of supported commands.
Type "exit" to leave the HBase Shell Version 1.2.0-cdh5.8.2, rUnknown, 
Sun Sep 11 11:52:54 PDT 2016


hive shell

Logging initialized using configuration in 
jar:file:/opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p1980.2070/jars/hive-common-1.1.0-cdh5.8.2.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com]
Sent: Wednesday, July 26, 2017 8:58 PM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

The bundled projects (HBase, Hive) in CDH have their own versions. I'm 
wondering if that is what is the difference.

Drill has been tested with HBase 1.1.1 and Hive 1.2.1 . For higher versions, as 
long as APIs have not changed, things should be backward compatible. 

Also, the error message you see in the SQLLine session... there is a complete 
stack trace in the Drill logs. Can you share that stack trace as well?


-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com]
Sent: Wednesday, July 26, 2017 5:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

It is CDH 5.8.2

I believe it is reliable versions, isn't it?

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com]
Sent: Monday, July 24, 2017 8:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

This means that the connectivity with ZK appears to be working. 

What are the HBase, ZK and Hadoop versions that you are working with? I presume 
that the student table is otherwise accessible.

-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com]
Sent: Sunday, July 23, 2017 2:58 AM
To: user@drill.apache.org
Cc: Shai Shapira <shai.shap...@amdocs.com>
Subject: RE: drill error connecting to Hbase

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai




illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)


Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira <shai.shap...@amdocs.com>
Subject: drill error connecting to Hbase



Shai Shapira
*  shai.shap...@amdocs.com<mailto:shai.shap...@amdocs.com>
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

This message and the information

RE: drill error connecting to Hbase

[Shai Shapira]

Hi,

My versions are:
Hbase - 1.2.0   
Hive - 1.1.0 

I'll send the complete stack trace.

Is Drill is so version sensitive?
Can I build a solution for production based on Drill? Or should I stick to what 
is coming with the Cloudera/Hortonworks distribution?

Thanks,
Shai


hbase shell
17/08/03 10:54:05 INFO Configuration.deprecation: hadoop.native.lib is 
deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help' for list of supported commands.
Type "exit" to leave the HBase Shell
Version 1.2.0-cdh5.8.2, rUnknown, Sun Sep 11 11:52:54 PDT 2016


hive shell

Logging initialized using configuration in 
jar:file:/opt/cloudera/parcels/CDH-5.8.2-1.cdh5.8.2.p1980.2070/jars/hive-common-1.1.0-cdh5.8.2.jar!/hive-log4j.properties
WARNING: Hive CLI is deprecated and migration to Beeline is recommended.

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com] 
Sent: Wednesday, July 26, 2017 8:58 PM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

The bundled projects (HBase, Hive) in CDH have their own versions. I'm 
wondering if that is what is the difference.

Drill has been tested with HBase 1.1.1 and Hive 1.2.1 . For higher versions, as 
long as APIs have not changed, things should be backward compatible. 

Also, the error message you see in the SQLLine session... there is a complete 
stack trace in the Drill logs. Can you share that stack trace as well?


-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Wednesday, July 26, 2017 5:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

It is CDH 5.8.2

I believe it is reliable versions, isn't it?

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com] 
Sent: Monday, July 24, 2017 8:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

This means that the connectivity with ZK appears to be working. 

What are the HBase, ZK and Hadoop versions that you are working with? I presume 
that the student table is otherwise accessible.

-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Sunday, July 23, 2017 2:58 AM
To: user@drill.apache.org
Cc: Shai Shapira <shai.shap...@amdocs.com>
Subject: RE: drill error connecting to Hbase

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai




illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)


Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira <shai.shap...@amdocs.com>
Subject: drill error connecting to Hbase



Shai Shapira
*  shai.shap...@amdocs.com<mailto:shai.shap...@amdocs.com>
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

RE: drill error connecting to Hbase

[Kunal Khatua]

The bundled projects (HBase, Hive) in CDH have their own versions. I'm 
wondering if that is what is the difference.

Drill has been tested with HBase 1.1.1 and Hive 1.2.1 . For higher versions, as 
long as APIs have not changed, things should be backward compatible. 

Also, the error message you see in the SQLLine session... there is a complete 
stack trace in the Drill logs. Can you share that stack trace as well?


-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Wednesday, July 26, 2017 5:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

It is CDH 5.8.2

I believe it is reliable versions, isn't it?

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com] 
Sent: Monday, July 24, 2017 8:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

This means that the connectivity with ZK appears to be working. 

What are the HBase, ZK and Hadoop versions that you are working with? I presume 
that the student table is otherwise accessible.

-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Sunday, July 23, 2017 2:58 AM
To: user@drill.apache.org
Cc: Shai Shapira <shai.shap...@amdocs.com>
Subject: RE: drill error connecting to Hbase

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai




illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)


Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira <shai.shap...@amdocs.com>
Subject: drill error connecting to Hbase



Shai Shapira
*  shai.shap...@amdocs.com<mailto:shai.shap...@amdocs.com>
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

RE: drill error connecting to Hbase

[Shai Shapira]

It is CDH 5.8.2

I believe it is reliable versions, isn't it?

Thanks,
Shai

-Original Message-
From: Kunal Khatua [mailto:kkha...@mapr.com] 
Sent: Monday, July 24, 2017 8:50 AM
To: user@drill.apache.org
Subject: RE: drill error connecting to Hbase

This means that the connectivity with ZK appears to be working. 

What are the HBase, ZK and Hadoop versions that you are working with? I presume 
that the student table is otherwise accessible.

-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Sunday, July 23, 2017 2:58 AM
To: user@drill.apache.org
Cc: Shai Shapira <shai.shap...@amdocs.com>
Subject: RE: drill error connecting to Hbase

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai




illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)


Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira <shai.shap...@amdocs.com>
Subject: drill error connecting to Hbase



Shai Shapira
*  shai.shap...@amdocs.com<mailto:shai.shap...@amdocs.com>
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>
This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

RE: drill error connecting to Hbase

[Kunal Khatua]

This means that the connectivity with ZK appears to be working. 

What are the HBase, ZK and Hadoop versions that you are working with? I presume 
that the student table is otherwise accessible.

-Original Message-
From: Shai Shapira [mailto:shai.shap...@amdocs.com] 
Sent: Sunday, July 23, 2017 2:58 AM
To: user@drill.apache.org
Cc: Shai Shapira <shai.shap...@amdocs.com>
Subject: RE: drill error connecting to Hbase

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai




illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)


Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira <shai.shap...@amdocs.com>
Subject: drill error connecting to Hbase



Shai Shapira
*  shai.shap...@amdocs.com<mailto:shai.shap...@amdocs.com>
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

RE: drill error connecting to Hbase

[Shai Shapira]

Hi,

I installed Drill and started to work with it, my goal is to use it to connect 
to Hbase.
I checked it a bit locally, csv files, Json files, works great.
When I am trying to connect to Hbase, I am getting error.

It seems that it is connecting to the Hbase/ZK, but fails somehow there.
The errors when trying to select from non-exist table ( stud ) and when 
accessing an existing table ( students ) are different.
For existing table, the error is in the zookeeper.MetaTableLocator.

Any ideas?

Thanks,
Shai




illin4620 STABDB05 54 > drill
Jul 20, 2017 6:17:02 PM org.glassfish.jersey.server.ApplicationHandler 
initialize
INFO: Initiating Jersey application, version Jersey: 2.8 2014-04-29 01:25:26...
apache drill 1.10.0
"just drill it"
0: jdbc:drill:zk=local> use hbase;
+---++
|  ok   |  summary   |
+---++
| true  | Default schema changed to [hbase]  |
+---++
1 row selected (0.895 seconds)
0: jdbc:drill:zk=local> select * from students ;
Error: SYSTEM ERROR: IllegalAccessError: tried to access method 
com.google.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator


[Error Id: 71a4a041-4f5d-4e68-9400-78c2faeac6f9 on illin4620:31010] 
(state=,code=0)
0: jdbc:drill:zk=local> select * from stud ;
Error: DATA_READ ERROR: Failure while loading table stud in database hbase.

Message:  stud
SQL Query null

[Error Id: f0a6591d-9068-4490-95c0-b0aea41365b4 on illin4620:31010] 
(state=,code=0)


Thanks,
Shai

From: Shai Shapira
Sent: Sunday, July 23, 2017 12:49 PM
To: Shai Shapira <shai.shap...@amdocs.com>
Subject: drill error connecting to Hbase



Shai Shapira
*  shai.shap...@amdocs.com<mailto:shai.shap...@amdocs.com>
* +972 9 776 4171

This message and the information contained herein is proprietary and 
confidential and subject to the Amdocs policy statement,

you may review at https://www.amdocs.com/about/email-disclaimer 
<https://www.amdocs.com/about/email-disclaimer>

Re: Apache Drill Error

[Robin Moffatt]

Thanks, that fixed the problem for me on Mac with Drill 1.8. I had the same
error as OP (Unable to locate an executable at
"/System/Library/Frameworks/JavaVM.framework/Versions/A/bin/java" (-1))

$ diff drill-config.sh drill-config.sh.bak
395c395
< #export JAVA_HOME
---
> export JAVA_HOME

On 21 September 2016 at 18:08, Padma Penumarthy 
wrote:

> Trying commenting out the following line in
> distribution/target/apache-drill-1.9.0-SNAPSHOT/apache-
> drill-1.9.0-SNAPSHOT/bin/drill-config.sh
>
> export JAVA_HOME
>
> Thanks,
> Padma
>
>
> > On Sep 20, 2016, at 7:17 PM, Rajasimman Selvaraj 
> wrote:
> >
> > Hi ,
> >   I am getting the following error while starting the drill in Mac OS
> for the first time.
> > Unable to locate an executable at "/System/Library/Frameworks/
> JavaVM.framework/Versions/A/bin/java" (-1)
> >
> > I have the java JDK installed in my machine as listed below.
> > RajaSimmans-MBP:apache-drill-1.8.0 Hadoop$ java -version
> > java version "1.8.0_101"
> > Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> > Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
> > RajaSimmans-MBP:apache-drill-1.8.0 Hadoop$
> >
> > Please help me to get the drill start and running in my Mac OS.
> >
> > Regards,
> > S.RajaSimman.
>
>

Re: Apache Drill Error

[Padma Penumarthy]

Sorry, I meant Try (not Trying)

> On Sep 21, 2016, at 10:08 AM, Padma Penumarthy  
> wrote:
> 
> Trying commenting out the following line in 
> distribution/target/apache-drill-1.9.0-SNAPSHOT/apache-drill-1.9.0-SNAPSHOT/bin/drill-config.sh
> 
> export JAVA_HOME
> 
> Thanks,
> Padma
> 
> 
>> On Sep 20, 2016, at 7:17 PM, Rajasimman Selvaraj > > wrote:
>> 
>> Hi ,
>>   I am getting the following error while starting the drill in Mac OS for 
>> the first time.
>> Unable to locate an executable at 
>> "/System/Library/Frameworks/JavaVM.framework/Versions/A/bin/java" (-1)
>> 
>> I have the java JDK installed in my machine as listed below.
>> RajaSimmans-MBP:apache-drill-1.8.0 Hadoop$ java -version
>> java version "1.8.0_101"
>> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
>> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
>> RajaSimmans-MBP:apache-drill-1.8.0 Hadoop$ 
>> 
>> Please help me to get the drill start and running in my Mac OS.
>> 
>> Regards,
>> S.RajaSimman.
> 

Re: Apache Drill Error

[Padma Penumarthy]

Trying commenting out the following line in 
distribution/target/apache-drill-1.9.0-SNAPSHOT/apache-drill-1.9.0-SNAPSHOT/bin/drill-config.sh

export JAVA_HOME

Thanks,
Padma


> On Sep 20, 2016, at 7:17 PM, Rajasimman Selvaraj  
> wrote:
> 
> Hi ,
>   I am getting the following error while starting the drill in Mac OS for the 
> first time.
> Unable to locate an executable at 
> "/System/Library/Frameworks/JavaVM.framework/Versions/A/bin/java" (-1)
> 
> I have the java JDK installed in my machine as listed below.
> RajaSimmans-MBP:apache-drill-1.8.0 Hadoop$ java -version
> java version "1.8.0_101"
> Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
> RajaSimmans-MBP:apache-drill-1.8.0 Hadoop$ 
> 
> Please help me to get the drill start and running in my Mac OS.
> 
> Regards,
> S.RajaSimman.

Apache Drill Error

[Rajasimman Selvaraj]

Hi ,
   I am getting the following error while starting the drill in Mac OS for the 
first time.
Unable to locate an executable at 
"/System/Library/Frameworks/JavaVM.framework/Versions/A/bin/java" (-1)

I have the java JDK installed in my machine as listed below.
RajaSimmans-MBP:apache-drill-1.8.0 Hadoop$ java -version
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
RajaSimmans-MBP:apache-drill-1.8.0 Hadoop$ 

Please help me to get the drill start and running in my Mac OS.

Regards,
S.RajaSimman.

Re: IllegalStateException: Memory was leaked by query - Drill error on wide table, but OK on a narrow but longer table.

[Edmon Begoli]

I don't know if it is related, but our /tmp files were filled up.

I will try to search, but our machines got hosed up from the problem
(queries were running over the last weekend), so I am not sure if I can
recover or if I can easily replicate.

Will try.

On Saturday, March 26, 2016, Abdel Hakim Deneche 
wrote:

> Hey Edmon,
>
> Can you search the logs for the errorId displayed with this error ? I've
> seen a similar issue when external sort fails to spill to disk (no space
> left on disk), it would leak memory and display this error message instead
> of the original issue (problem spilling to disk).
>
> I will open a JIRA to fix the error message. Waiting on you to confirm it's
> indeed the same issue.
>
> Thanks
>
> On Mon, Mar 21, 2016 at 1:57 PM, Edmon Begoli  > wrote:
>
> > We are converting some raw CMS data from csv to parquet using Drill, and
> > using partitioning as we go.
> >
> > Query 1 runs OK on a narrower file:
> >
> > size:
> > 13G Mar 16 18:20 out_revenuej_lds_100_201412.csv (Month 12 file)
> > 13G Mar 16 16:37 out_claimsj_lds_100_2014_q1.csv (Quarter 1 file)
> >
> > q1 has 198 columns;
> > month 12 has 32 columns.
> >
> > Both are partitioned on the same unique ID resulting in 14 buckets.
> > Resulting parquet for month 12:  2.6G parquet/outpatient_revenue_12
> >
> > It fails on out_claimsj_lds_100_2014_q1.csv with 198 columns.
> >
> > Error:
> > 
> >
> > *Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query.
> > Memory leaked: (8530784)*
> > *Allocator(op:0:0:6:ExternalSort) 2000/8530784/357974944/357913941
> > (res/actual/peak/limit)*
> >
> >
> > *Fragment 0:0*
> >
> > *[Error Id: 0606ff19-1c3a-4611-a3d4-1d28d9b3bd60 on
> cyclone-backend:31010]
> > (state=,code=0)*
> >
> >
> > Environment
> > -
> > (upped memory to MAX RAM - 4 GB to try to make it pass)
> >
> >
> > drill-env.sh
> > DRILL_MAX_DIRECT_MEMORY="60G"
> > DRILL_HEAP="48G"
> >
> > export DRILL_JAVA_OPTS="-Xms$DRILL_HEAP -Xmx$DRILL_HEAP
> > -XX:MaxDirectMemorySize=$DRILL
> > _MAX_DIRECT_MEMORY -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=1G
> > -Ddrill.exec.enab
> > le-epoll=true"
> >
> > # Class unloading is disabled by default in Java 7
> > #
> >
> >
> http://hg.openjdk.java.net/jdk7u/jdk7u60/hotspot/file/tip/src/share/vm/runtime/globa
> > ls.hpp#l1622
> > export SERVER_GC_OPTS="-XX:+CMSClassUnloadingEnabled -XX:+UseG1GC "
> >
> > Query 1
> > ---
> >
> > -- Success
> >
> > --12
> > CREATE TABLE outpatient_revenue_12
> > PARTITION BY (UID_CLASS) AS SELECT CAST( SUBSTR(columns[0],1,2) as INT)
> as
> >  UID_CLASS, CASE WHEN columns[0] = '' THEN NULL ELSE CAST(columns[0] as
> > DOUBLE) END  as DSYSRTKY, CASE WHEN columns[1] = '' THEN NULL ELSE
> > CAST(columns[1] as DOUBLE) END  as CLAIMNO, CASE WHEN columns[2] = ''
> THEN
> > NULL ELSE CAST(columns[2] as DOUBLE) END  as CLM_LN, CASE WHEN
> columns[3] =
> > '' THEN NULL ELSE TO_DATE(columns[3], 'MMdd') END  as THRU_DT,
> > NULLIF(columns[4],'') as CLM_TYPE, NULLIF(columns[5],'') as REV_CNTR,
> CASE
> > WHEN columns[6] = '' THEN NULL ELSE TO_DATE(columns[6], 'MMdd') END
> as
> > REV_DT, NULLIF(columns[7],'') as APCHIPPS, NULLIF(columns[8],'') as
> > HCPCS_CD, NULLIF(columns[9],'') as MDFR_CD1, NULLIF(columns[10],'') as
> > MDFR_CD2, NULLIF(columns[11],'') as PMTMTHD, NULLIF(columns[12],'') as
> > DSCNTIND, NULLIF(columns[13],'') as PACKGIND, NULLIF(columns[14],'') as
> > OTAF_1, NULLIF(columns[15],'') as IDENDC, CASE WHEN columns[16] = '' THEN
> > NULL ELSE CAST(columns[16] as DOUBLE) END  as REV_UNIT, CASE WHEN
> > columns[17] = '' THEN NULL ELSE CAST(columns[17] as DOUBLE) END  as
> > REV_RATE, CASE WHEN columns[18] = '' THEN NULL ELSE CAST(columns[18] as
> > DOUBLE) END  as REVBLOOD, CASE WHEN columns[19] = '' THEN NULL ELSE
> > CAST(columns[19] as DOUBLE) END  as REVDCTBL, CASE WHEN columns[20] = ''
> > THEN NULL ELSE CAST(columns[20] as DOUBLE) END  as WAGEADJ, CASE WHEN
> > columns[21] = '' THEN NULL ELSE CAST(columns[21] as DOUBLE) END  as
> > RDCDCOIN, CASE WHEN columns[22] = '' THEN NULL ELSE CAST(columns[22] as
> > DOUBLE) END  as REV_MSP1, CASE WHEN columns[23] = '' THEN NULL ELSE
> > CAST(columns[23] as DOUBLE) END  as REV_MSP2, CASE WHEN columns[24] = ''
> > THEN NULL ELSE CAST(columns[24] as DOUBLE) END  as RPRVDPMT, CASE WHEN
> > columns[25] = '' THEN NULL ELSE CAST(columns[25] as DOUBLE) END  as
> > RBENEPMT, CASE WHEN columns[26] = '' THEN NULL ELSE CAST(columns[26] as
> > DOUBLE) END  as PTNTRESP, CASE WHEN columns[27] = '' THEN NULL ELSE
> > CAST(columns[27] as DOUBLE) END  as REVPMT, CASE WHEN columns[28] = ''
> THEN
> > NULL ELSE CAST(columns[28] as DOUBLE) END  as REV_CHRG, CASE WHEN
> > columns[29] = '' THEN NULL ELSE CAST(columns[29] as DOUBLE) END  as
> > REV_NCVR, NULLIF(columns[30],'') as REVSTIND, NULLIF(columns[31],'') as
> > REV_CNTR_PRICNG_IND_CD
> > FROM
> >
> 

Re: IllegalStateException: Memory was leaked by query - Drill error on wide table, but OK on a narrow but longer table.

[Abdel Hakim Deneche]

Hey Edmon,

Can you search the logs for the errorId displayed with this error ? I've
seen a similar issue when external sort fails to spill to disk (no space
left on disk), it would leak memory and display this error message instead
of the original issue (problem spilling to disk).

I will open a JIRA to fix the error message. Waiting on you to confirm it's
indeed the same issue.

Thanks

On Mon, Mar 21, 2016 at 1:57 PM, Edmon Begoli  wrote:

> We are converting some raw CMS data from csv to parquet using Drill, and
> using partitioning as we go.
>
> Query 1 runs OK on a narrower file:
>
> size:
> 13G Mar 16 18:20 out_revenuej_lds_100_201412.csv (Month 12 file)
> 13G Mar 16 16:37 out_claimsj_lds_100_2014_q1.csv (Quarter 1 file)
>
> q1 has 198 columns;
> month 12 has 32 columns.
>
> Both are partitioned on the same unique ID resulting in 14 buckets.
> Resulting parquet for month 12:  2.6G parquet/outpatient_revenue_12
>
> It fails on out_claimsj_lds_100_2014_q1.csv with 198 columns.
>
> Error:
> 
>
> *Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query.
> Memory leaked: (8530784)*
> *Allocator(op:0:0:6:ExternalSort) 2000/8530784/357974944/357913941
> (res/actual/peak/limit)*
>
>
> *Fragment 0:0*
>
> *[Error Id: 0606ff19-1c3a-4611-a3d4-1d28d9b3bd60 on cyclone-backend:31010]
> (state=,code=0)*
>
>
> Environment
> -
> (upped memory to MAX RAM - 4 GB to try to make it pass)
>
>
> drill-env.sh
> DRILL_MAX_DIRECT_MEMORY="60G"
> DRILL_HEAP="48G"
>
> export DRILL_JAVA_OPTS="-Xms$DRILL_HEAP -Xmx$DRILL_HEAP
> -XX:MaxDirectMemorySize=$DRILL
> _MAX_DIRECT_MEMORY -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=1G
> -Ddrill.exec.enab
> le-epoll=true"
>
> # Class unloading is disabled by default in Java 7
> #
>
> http://hg.openjdk.java.net/jdk7u/jdk7u60/hotspot/file/tip/src/share/vm/runtime/globa
> ls.hpp#l1622
> export SERVER_GC_OPTS="-XX:+CMSClassUnloadingEnabled -XX:+UseG1GC "
>
> Query 1
> ---
>
> -- Success
>
> --12
> CREATE TABLE outpatient_revenue_12
> PARTITION BY (UID_CLASS) AS SELECT CAST( SUBSTR(columns[0],1,2) as INT) as
>  UID_CLASS, CASE WHEN columns[0] = '' THEN NULL ELSE CAST(columns[0] as
> DOUBLE) END  as DSYSRTKY, CASE WHEN columns[1] = '' THEN NULL ELSE
> CAST(columns[1] as DOUBLE) END  as CLAIMNO, CASE WHEN columns[2] = '' THEN
> NULL ELSE CAST(columns[2] as DOUBLE) END  as CLM_LN, CASE WHEN columns[3] =
> '' THEN NULL ELSE TO_DATE(columns[3], 'MMdd') END  as THRU_DT,
> NULLIF(columns[4],'') as CLM_TYPE, NULLIF(columns[5],'') as REV_CNTR, CASE
> WHEN columns[6] = '' THEN NULL ELSE TO_DATE(columns[6], 'MMdd') END  as
> REV_DT, NULLIF(columns[7],'') as APCHIPPS, NULLIF(columns[8],'') as
> HCPCS_CD, NULLIF(columns[9],'') as MDFR_CD1, NULLIF(columns[10],'') as
> MDFR_CD2, NULLIF(columns[11],'') as PMTMTHD, NULLIF(columns[12],'') as
> DSCNTIND, NULLIF(columns[13],'') as PACKGIND, NULLIF(columns[14],'') as
> OTAF_1, NULLIF(columns[15],'') as IDENDC, CASE WHEN columns[16] = '' THEN
> NULL ELSE CAST(columns[16] as DOUBLE) END  as REV_UNIT, CASE WHEN
> columns[17] = '' THEN NULL ELSE CAST(columns[17] as DOUBLE) END  as
> REV_RATE, CASE WHEN columns[18] = '' THEN NULL ELSE CAST(columns[18] as
> DOUBLE) END  as REVBLOOD, CASE WHEN columns[19] = '' THEN NULL ELSE
> CAST(columns[19] as DOUBLE) END  as REVDCTBL, CASE WHEN columns[20] = ''
> THEN NULL ELSE CAST(columns[20] as DOUBLE) END  as WAGEADJ, CASE WHEN
> columns[21] = '' THEN NULL ELSE CAST(columns[21] as DOUBLE) END  as
> RDCDCOIN, CASE WHEN columns[22] = '' THEN NULL ELSE CAST(columns[22] as
> DOUBLE) END  as REV_MSP1, CASE WHEN columns[23] = '' THEN NULL ELSE
> CAST(columns[23] as DOUBLE) END  as REV_MSP2, CASE WHEN columns[24] = ''
> THEN NULL ELSE CAST(columns[24] as DOUBLE) END  as RPRVDPMT, CASE WHEN
> columns[25] = '' THEN NULL ELSE CAST(columns[25] as DOUBLE) END  as
> RBENEPMT, CASE WHEN columns[26] = '' THEN NULL ELSE CAST(columns[26] as
> DOUBLE) END  as PTNTRESP, CASE WHEN columns[27] = '' THEN NULL ELSE
> CAST(columns[27] as DOUBLE) END  as REVPMT, CASE WHEN columns[28] = '' THEN
> NULL ELSE CAST(columns[28] as DOUBLE) END  as REV_CHRG, CASE WHEN
> columns[29] = '' THEN NULL ELSE CAST(columns[29] as DOUBLE) END  as
> REV_NCVR, NULLIF(columns[30],'') as REVSTIND, NULLIF(columns[31],'') as
> REV_CNTR_PRICNG_IND_CD
> FROM
> dfs.`default`.`/data/cms/2014_outpatient/out_revenuej_lds_100_201412.csv`;
>
>
> Query 2
> ---
>
> -- Failed
>
> -- Q1
> CREATE TABLE base_outpatient_q1
> PARTITION BY (UID_CLASS)
> AS
> SELECT CAST( SUBSTR(columns[0],1,2) as INT) as  UID_CLASS, CASE WHEN
> columns[0] = '' THEN NULL ELSE CAST(columns[0] as DOUBLE) END  as
> `DSYSRTKY`, CASE WHEN columns[1] = '' THEN NULL ELSE CAST(columns[1] as
> DOUBLE) END  as `CLAIMNO`, NULLIF(columns[2],'') as `PROVIDER`, CASE WHEN
> columns[3] = '' THEN NULL ELSE TO_DATE(columns[3], 'MMdd') END  as
> `THRU_DT`, NULLIF(columns[4],'') as `RIC_CD`, NULLIF(columns[5],'') as
> `CLM_TYPE`, 

IllegalStateException: Memory was leaked by query - Drill error on wide table, but OK on a narrow but longer table.

[Edmon Begoli]

We are converting some raw CMS data from csv to parquet using Drill, and
using partitioning as we go.

Query 1 runs OK on a narrower file:

size:
13G Mar 16 18:20 out_revenuej_lds_100_201412.csv (Month 12 file)
13G Mar 16 16:37 out_claimsj_lds_100_2014_q1.csv (Quarter 1 file)

q1 has 198 columns;
month 12 has 32 columns.

Both are partitioned on the same unique ID resulting in 14 buckets.
Resulting parquet for month 12:  2.6G parquet/outpatient_revenue_12

It fails on out_claimsj_lds_100_2014_q1.csv with 198 columns.

Error:


*Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query.
Memory leaked: (8530784)*
*Allocator(op:0:0:6:ExternalSort) 2000/8530784/357974944/357913941
(res/actual/peak/limit)*


*Fragment 0:0*

*[Error Id: 0606ff19-1c3a-4611-a3d4-1d28d9b3bd60 on cyclone-backend:31010]
(state=,code=0)*


Environment
-
(upped memory to MAX RAM - 4 GB to try to make it pass)


drill-env.sh
DRILL_MAX_DIRECT_MEMORY="60G"
DRILL_HEAP="48G"

export DRILL_JAVA_OPTS="-Xms$DRILL_HEAP -Xmx$DRILL_HEAP
-XX:MaxDirectMemorySize=$DRILL
_MAX_DIRECT_MEMORY -XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=1G
-Ddrill.exec.enab
le-epoll=true"

# Class unloading is disabled by default in Java 7
#
http://hg.openjdk.java.net/jdk7u/jdk7u60/hotspot/file/tip/src/share/vm/runtime/globa
ls.hpp#l1622
export SERVER_GC_OPTS="-XX:+CMSClassUnloadingEnabled -XX:+UseG1GC "

Query 1
---

-- Success

--12
CREATE TABLE outpatient_revenue_12
PARTITION BY (UID_CLASS) AS SELECT CAST( SUBSTR(columns[0],1,2) as INT) as
 UID_CLASS, CASE WHEN columns[0] = '' THEN NULL ELSE CAST(columns[0] as
DOUBLE) END  as DSYSRTKY, CASE WHEN columns[1] = '' THEN NULL ELSE
CAST(columns[1] as DOUBLE) END  as CLAIMNO, CASE WHEN columns[2] = '' THEN
NULL ELSE CAST(columns[2] as DOUBLE) END  as CLM_LN, CASE WHEN columns[3] =
'' THEN NULL ELSE TO_DATE(columns[3], 'MMdd') END  as THRU_DT,
NULLIF(columns[4],'') as CLM_TYPE, NULLIF(columns[5],'') as REV_CNTR, CASE
WHEN columns[6] = '' THEN NULL ELSE TO_DATE(columns[6], 'MMdd') END  as
REV_DT, NULLIF(columns[7],'') as APCHIPPS, NULLIF(columns[8],'') as
HCPCS_CD, NULLIF(columns[9],'') as MDFR_CD1, NULLIF(columns[10],'') as
MDFR_CD2, NULLIF(columns[11],'') as PMTMTHD, NULLIF(columns[12],'') as
DSCNTIND, NULLIF(columns[13],'') as PACKGIND, NULLIF(columns[14],'') as
OTAF_1, NULLIF(columns[15],'') as IDENDC, CASE WHEN columns[16] = '' THEN
NULL ELSE CAST(columns[16] as DOUBLE) END  as REV_UNIT, CASE WHEN
columns[17] = '' THEN NULL ELSE CAST(columns[17] as DOUBLE) END  as
REV_RATE, CASE WHEN columns[18] = '' THEN NULL ELSE CAST(columns[18] as
DOUBLE) END  as REVBLOOD, CASE WHEN columns[19] = '' THEN NULL ELSE
CAST(columns[19] as DOUBLE) END  as REVDCTBL, CASE WHEN columns[20] = ''
THEN NULL ELSE CAST(columns[20] as DOUBLE) END  as WAGEADJ, CASE WHEN
columns[21] = '' THEN NULL ELSE CAST(columns[21] as DOUBLE) END  as
RDCDCOIN, CASE WHEN columns[22] = '' THEN NULL ELSE CAST(columns[22] as
DOUBLE) END  as REV_MSP1, CASE WHEN columns[23] = '' THEN NULL ELSE
CAST(columns[23] as DOUBLE) END  as REV_MSP2, CASE WHEN columns[24] = ''
THEN NULL ELSE CAST(columns[24] as DOUBLE) END  as RPRVDPMT, CASE WHEN
columns[25] = '' THEN NULL ELSE CAST(columns[25] as DOUBLE) END  as
RBENEPMT, CASE WHEN columns[26] = '' THEN NULL ELSE CAST(columns[26] as
DOUBLE) END  as PTNTRESP, CASE WHEN columns[27] = '' THEN NULL ELSE
CAST(columns[27] as DOUBLE) END  as REVPMT, CASE WHEN columns[28] = '' THEN
NULL ELSE CAST(columns[28] as DOUBLE) END  as REV_CHRG, CASE WHEN
columns[29] = '' THEN NULL ELSE CAST(columns[29] as DOUBLE) END  as
REV_NCVR, NULLIF(columns[30],'') as REVSTIND, NULLIF(columns[31],'') as
REV_CNTR_PRICNG_IND_CD
FROM
dfs.`default`.`/data/cms/2014_outpatient/out_revenuej_lds_100_201412.csv`;


Query 2
---

-- Failed

-- Q1
CREATE TABLE base_outpatient_q1
PARTITION BY (UID_CLASS)
AS
SELECT CAST( SUBSTR(columns[0],1,2) as INT) as  UID_CLASS, CASE WHEN
columns[0] = '' THEN NULL ELSE CAST(columns[0] as DOUBLE) END  as
`DSYSRTKY`, CASE WHEN columns[1] = '' THEN NULL ELSE CAST(columns[1] as
DOUBLE) END  as `CLAIMNO`, NULLIF(columns[2],'') as `PROVIDER`, CASE WHEN
columns[3] = '' THEN NULL ELSE TO_DATE(columns[3], 'MMdd') END  as
`THRU_DT`, NULLIF(columns[4],'') as `RIC_CD`, NULLIF(columns[5],'') as
`CLM_TYPE`, NULLIF(columns[6],'') as `QUERY_CD`, NULLIF(columns[7],'') as
`FAC_TYPE`, NULLIF(columns[8],'') as `TYPESRVC`, NULLIF(columns[9],'') as
`FREQ_CD`, NULLIF(columns[10],'') as `FI_NUM`, NULLIF(columns[11],'') as
`NOPAY_CD`, CASE WHEN columns[12] = '' THEN NULL ELSE CAST(columns[12] as
DOUBLE) END  as `PMT_AMT`, CASE WHEN columns[13] = '' THEN NULL ELSE
CAST(columns[13] as DOUBLE) END  as `PRPAYAMT`, NULLIF(columns[14],'') as
`PRPAY_CD`, NULLIF(columns[15],'') as `PRSTATE`, NULLIF(columns[16],'') as
`ORGNPINM`, NULLIF(columns[17],'') as `AT_UPIN`, NULLIF(columns[18],'') as
`AT_NPI`, NULLIF(columns[19],'') as `OP_UPIN`, NULLIF(columns[20],'') as
`OP_NPI`, NULLIF(columns[21],'') 

RE: Drill error with large sort

[Paul Friedman]

Thanks for this, these parameters fixed it!

---Paul


-Original Message-
From: Abdel Hakim Deneche [mailto:adene...@maprtech.com]
Sent: Thursday, February 25, 2016 5:32 PM
To: user <user@drill.apache.org>
Subject: Re: Drill error with large sort

Not so short answer:

In Drill 1.5 (I assume you are using 1.5) we have an improved allocator that
better tracks how much memory each operator is using. In your case it seems
that the date has very wide columns that are causing Sort to choke on the
very first batch of data (1024 records taking up 224MB!!!) because it's way
more than it's memory limit (around 178MB in your particular case).
Drill uses a fancy equation to compute this limit and increasing the
aforementioned option will increase the sort limit. More details here:

http://drill.apache.org/docs/configuring-drill-memory/

On Thu, Feb 25, 2016 at 5:26 PM, Abdel Hakim Deneche <adene...@maprtech.com>
wrote:

> Short answer:
>
> increase the value of planner.memory.max_query_memory_per_node, by
> default it's set to 2GB, try setting to 4 or even 8GB. This should get
> the query to pass.
>
> On Thu, Feb 25, 2016 at 5:24 PM, Jeff Maass <jma...@cccis.com> wrote:
>
>>
>> If you are open to changing the query:
>>   # try removing the functions on the 5th column
>>   # is there any way you could further limit the query?
>>   # does the query finish if u add a limit / top clause?
>>   # what do the logs say?
>>
>> 
>> From: Paul Friedman <paul.fried...@streetlightdata.com>
>> Sent: Thursday, February 25, 2016 7:07:12 PM
>> To: user@drill.apache.org
>> Subject: Drill error with large sort
>>
>> I’ve got a query reading from a large directory of parquet files (41
>> GB) and I’m consistently getting this error:
>>
>>
>>
>> Error: RESOURCE ERROR: One or more nodes ran out of memory while
>> executing the query.
>>
>>
>>
>> Unable to allocate sv2 for 1023 records, and not enough batchGroups
>> to spill.
>>
>> batchGroups.size 0
>>
>> spilledBatchGroups.size 0
>>
>> allocated memory 224287987
>>
>> allocator limit 178956970
>>
>> Fragment 0:0
>>
>>
>>
>> [Error Id: 878d604c-4656-4a5a-8b46-ff38a6ae020d on
>> chai.dev.streetlightdata.com:31010] (state=,code=0)
>>
>>
>>
>> Direct memory is set to 48GB and heap is 8GB.
>>
>>
>>
>> The query is:
>>
>>
>>
>> select probe_id, provider_id, is_moving, mode,
>> cast(convert_to(points,
>> 'JSON') as varchar(1))
>>
>> from dfs.`/home/paul/data`
>>
>> where
>>
>> start_lat between 24.4873780449008 and 60.0108911181433 and
>>
>> start_lon between -139.065890469841 and -52.8305074899881 and
>>
>> provider_id = '343' and
>>
>> mod(abs(hash(probe_id)),  100) = 0
>>
>> order by probe_id, start_time;
>>
>>
>>
>> I’m also using the “example” drill-override configuration.
>>
>>
>>
>> Any help would be appreciated.
>>
>>
>>
>> Thanks.
>>
>>
>>
>> ---Paul
>>
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <http://www.mapr.com/training?utm_source=Email_medium=Signature
> m_campaign=Free%20available>
>



-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email_medium=Signature_campaign=Free%20available>

Re: Drill error with large sort

[Abdel Hakim Deneche]

Not so short answer:

In Drill 1.5 (I assume you are using 1.5) we have an improved allocator
that better tracks how much memory each operator is using. In your case it
seems that the date has very wide columns that are causing Sort to choke on
the very first batch of data (1024 records taking up 224MB!!!) because it's
way more than it's memory limit (around 178MB in your particular case).
Drill uses a fancy equation to compute this limit and increasing the
aforementioned option will increase the sort limit. More details here:

http://drill.apache.org/docs/configuring-drill-memory/

On Thu, Feb 25, 2016 at 5:26 PM, Abdel Hakim Deneche <adene...@maprtech.com>
wrote:

> Short answer:
>
> increase the value of planner.memory.max_query_memory_per_node, by default
> it's set to 2GB, try setting to 4 or even 8GB. This should get the query to
> pass.
>
> On Thu, Feb 25, 2016 at 5:24 PM, Jeff Maass <jma...@cccis.com> wrote:
>
>>
>> If you are open to changing the query:
>>   # try removing the functions on the 5th column
>>   # is there any way you could further limit the query?
>>   # does the query finish if u add a limit / top clause?
>>   # what do the logs say?
>>
>> 
>> From: Paul Friedman <paul.fried...@streetlightdata.com>
>> Sent: Thursday, February 25, 2016 7:07:12 PM
>> To: user@drill.apache.org
>> Subject: Drill error with large sort
>>
>> I’ve got a query reading from a large directory of parquet files (41 GB)
>> and I’m consistently getting this error:
>>
>>
>>
>> Error: RESOURCE ERROR: One or more nodes ran out of memory while executing
>> the query.
>>
>>
>>
>> Unable to allocate sv2 for 1023 records, and not enough batchGroups to
>> spill.
>>
>> batchGroups.size 0
>>
>> spilledBatchGroups.size 0
>>
>> allocated memory 224287987
>>
>> allocator limit 178956970
>>
>> Fragment 0:0
>>
>>
>>
>> [Error Id: 878d604c-4656-4a5a-8b46-ff38a6ae020d on
>> chai.dev.streetlightdata.com:31010] (state=,code=0)
>>
>>
>>
>> Direct memory is set to 48GB and heap is 8GB.
>>
>>
>>
>> The query is:
>>
>>
>>
>> select probe_id, provider_id, is_moving, mode,  cast(convert_to(points,
>> 'JSON') as varchar(1))
>>
>> from dfs.`/home/paul/data`
>>
>> where
>>
>> start_lat between 24.4873780449008 and 60.0108911181433 and
>>
>> start_lon between -139.065890469841 and -52.8305074899881 and
>>
>> provider_id = '343' and
>>
>> mod(abs(hash(probe_id)),  100) = 0
>>
>> order by probe_id, start_time;
>>
>>
>>
>> I’m also using the “example” drill-override configuration.
>>
>>
>>
>> Any help would be appreciated.
>>
>>
>>
>> Thanks.
>>
>>
>>
>> ---Paul
>>
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
>   <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <http://www.mapr.com/training?utm_source=Email_medium=Signature_campaign=Free%20available>
>



-- 

Abdelhakim Deneche

Software Engineer

  <http://www.mapr.com/>


Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email_medium=Signature_campaign=Free%20available>

Re: Drill error with large sort

[Jeff Maass]

If you are open to changing the query:
  # try removing the functions on the 5th column
  # is there any way you could further limit the query?
  # does the query finish if u add a limit / top clause?
  # what do the logs say?


From: Paul Friedman <paul.fried...@streetlightdata.com>
Sent: Thursday, February 25, 2016 7:07:12 PM
To: user@drill.apache.org
Subject: Drill error with large sort

I’ve got a query reading from a large directory of parquet files (41 GB)
and I’m consistently getting this error:



Error: RESOURCE ERROR: One or more nodes ran out of memory while executing
the query.



Unable to allocate sv2 for 1023 records, and not enough batchGroups to
spill.

batchGroups.size 0

spilledBatchGroups.size 0

allocated memory 224287987

allocator limit 178956970

Fragment 0:0



[Error Id: 878d604c-4656-4a5a-8b46-ff38a6ae020d on
chai.dev.streetlightdata.com:31010] (state=,code=0)



Direct memory is set to 48GB and heap is 8GB.



The query is:



select probe_id, provider_id, is_moving, mode,  cast(convert_to(points,
'JSON') as varchar(1))

from dfs.`/home/paul/data`

where

start_lat between 24.4873780449008 and 60.0108911181433 and

start_lon between -139.065890469841 and -52.8305074899881 and

provider_id = '343' and

mod(abs(hash(probe_id)),  100) = 0

order by probe_id, start_time;



I’m also using the “example” drill-override configuration.



Any help would be appreciated.



Thanks.



---Paul

Drill error with large sort

[Paul Friedman]

I’ve got a query reading from a large directory of parquet files (41 GB)
and I’m consistently getting this error:



Error: RESOURCE ERROR: One or more nodes ran out of memory while executing
the query.



Unable to allocate sv2 for 1023 records, and not enough batchGroups to
spill.

batchGroups.size 0

spilledBatchGroups.size 0

allocated memory 224287987

allocator limit 178956970

Fragment 0:0



[Error Id: 878d604c-4656-4a5a-8b46-ff38a6ae020d on
chai.dev.streetlightdata.com:31010] (state=,code=0)



Direct memory is set to 48GB and heap is 8GB.



The query is:



select probe_id, provider_id, is_moving, mode,  cast(convert_to(points,
'JSON') as varchar(1))

from dfs.`/home/paul/data`

where

start_lat between 24.4873780449008 and 60.0108911181433 and

start_lon between -139.065890469841 and -52.8305074899881 and

provider_id = '343' and

mod(abs(hash(probe_id)),  100) = 0

order by probe_id, start_time;



I’m also using the “example” drill-override configuration.



Any help would be appreciated.



Thanks.



---Paul