Re: [1.9.0] : UserException: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0 and then SYSTEM ERROR: IOException: Failed to shutdown streamer

2017-12-06 Thread Anup Tiwari 
Hi All,

As asked in trail mail can someone explain how to handle :- *UserException:
SYSTEM ERROR: IllegalReferenceCountException*

*: refCnt: 0?*
As above error doesn't explain what and where the real problem is? But if
we execute same query in hive for which we get above error then it works.

Regards,
*Anup Tiwari*

On Mon, Dec 12, 2016 at 5:07 PM, Anup Tiwari 
wrote:

> Hi Aman,
>
> Sorry for delayed response, since we are executing this query on our
> ~150GB logs and as i have mentioned in trail mail, by executing "removed
> conditions alone" CTAS got executed successfully, so i don't know which
> sample data i should share(since i don't know pattern)?
>
> Can you tell me in which scenarios we throw "
> IllegalReferenceCountException" and how to handle this in different
> scenarios?
>
> Regards,
> *Anup Tiwari*
>
> On Thu, Dec 8, 2016 at 10:55 PM, Aman Sinha  wrote:
>
>> Hi Anup,
>> since your original query was working on 1.6 and failed in 1.9,  could you
>> pls file a JIRA for this ?  It sounds like a regression related to
>> evaluation of a Project expression (based on the stack trace).  Since
>> there
>> are several CASE exprs, quite likely something related to its evaluation.
>> It would be great if you can provide some sample data for someone to
>> debug.
>> Thanks.
>>
>> On Thu, Dec 8, 2016 at 12:50 AM, Anup Tiwari 
>> wrote:
>>
>> > Hi,
>> >
>> > I have removed few conditions from my query then it just worked fine.
>> >
>> > Also can someone tell me in which scenarios we throw "
>> > *IllegalReferenceCountException*" and how to handle it in different
>> > scenarios ?
>> >
>> > As i got this in another query and by removing some conditions it worked
>> > for me but when i execute that removed conditions alone in CTAS , it got
>> > executed successfully.
>> >
>> > Regards,
>> > *Anup Tiwari*
>> >
>> > On Wed, Dec 7, 2016 at 12:22 PM, Anup Tiwari > >
>> > wrote:
>> >
>> > > Hi Team,
>> > >
>> > > I am getting below 2 error in my one of the query which was working
>> fine
>> > > on 1.6, Please help me out in this:-
>> > >
>> > > 1. UserException: SYSTEM ERROR: IllegalReferenceCountException:
>> refCnt:
>> > 0
>> > > 2. SYSTEM ERROR: IOException: Failed to shutdown streamer
>> > >
>> > > Please find below query and its stack trace :-
>> > >
>> > > *Query :-*
>> > >
>> > > create table a_tt3_reg_login as
>> > > select sessionid,
>> > >
>> > > count(distinct (case when ((( event = 'e.a' and ajaxUrl like
>> > > '%/ab/pL%t=r%' ) or ( (Base64Conv(Response) like '%st%tr%' and
>> > > Base64Conv(Response) not like '%error%') and ajaxUrl like
>> '%/sign/ter%'
>> > ))
>> > > OR ( event = 'e.a' and ajaxUrl like '%/player/ter/ter.htm%' and
>> > > Base64Conv(Response) like '%st%tr%ter%tr%')  OR (id = '/ter/thyou.htm'
>> > and
>> > > url = '/pla/natlob.htm')) then sessionid end) )  as  regs,
>> > >
>> > > count(distinct (case when ( ajaxUrl like '%/signup/poLo%t=log%' and
>> event
>> > > = 'e.a' ) or ( event = 'e.a' and ajaxUrl like
>> '%j_spring_security_check%'
>> > > and Base64Conv(Response)  like '%st%tr%') then sessionid end) ) as
>> login
>> > ,
>> > >
>> > > count(distinct (case when ((ajaxUrl like
>> '/pl%/loadResponsePage.htm%fD=
>> > true=yes%'
>> > > or ajaxUrl like '/pl%/loadResponsePage.htm%fD=true=YES%') OR
>> > (ajaxUrl
>> > > like 'loadSuccessPage.do%fD=true=yes%' or ajaxUrl like
>> > > 'loadSuccessPage.do%fD=true=YES%'))  then sessionid end) ) as fd
>> ,
>> > >
>> > > count(distinct (case when ((ajaxUrl like
>> '/pl%/loadResponsePage.htm%fD=
>> > false=yes%'
>> > > or ajaxUrl like '/pl%/loadResponsePage.htm%fD=false=YES%') OR
>> > > (ajaxUrl like 'loadSuccessPage.do%fD=false=yes%' or ajaxUrl like
>> > > 'loadSuccessPage.do%fD=false=YES%')) then sessionid end) ) as rd
>> > >
>> > > from
>> > > tt2
>> > > group by sessionid;
>> > > Error: SYSTEM ERROR: IllegalReferenceCountException: refCnt: 0
>> > >
>> > > Fragment 14:19
>> > >
>> > > [Error Id: e4659753-f8d0-403c-9eec-0ff6f2e30dd9 on namenode:31010]
>> > > (state=,code=0)
>> > >
>> > >
>> > > *Stack Trace From Drillbit.log:-*
>> > >
>> > > [Error Id: e4659753-f8d0-403c-9eec-0ff6f2e30dd9 on namenode:31010]
>> > > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
>> > > IllegalReferenceCountException: refCnt: 0
>> > >
>> > > Fragment 14:19
>> > >
>> > > [Error Id: e4659753-f8d0-403c-9eec-0ff6f2e30dd9 on namenode:31010]
>> > > at org.apache.drill.common.exceptions.UserException$
>> > > Builder.build(UserException.java:543) ~[drill-common-1.9.0.jar:1.9.0]
>> > > at org.apache.drill.exec.work.fragment.FragmentExecutor.
>> > > sendFinalState(FragmentExecutor.java:293)
>> [drill-java-exec-1.9.0.jar:1.
>> > > 9.0]
>> > > at org.apache.drill.exec.work.fragment.FragmentExecutor.
>> > > cleanup(FragmentExecutor.java:160) [drill-java-exec-1.9.0.jar:1.9.0]
>> > > at

Re: Error reading int96 fields

2017-12-06 Thread Rahul Raj 
I have created https://issues.apache.org/jira/browse/DRILL-6016.

You can find the sample dataset at
https://github.com/rajrahul/files/blob/master/result.tar.gz

Regards,
Rahul

On Tue, Dec 5, 2017 at 5:57 PM, Vitalii Diravka 
wrote:

> Hi Rahul,
>
> It looks like a bug.
> Could you please open the jira ticket and provide the query and dataset to
> reproduce the issue?
>
> Thanks
>
> Kind regards
> Vitalii
>
> On Tue, Dec 5, 2017 at 2:01 PM, Rahul Raj  >
> wrote:
>
> > I am getting the error - SYSTEM ERROR : ClassCastException:
> > prg.apache.drill.exec.vector.TimeStampVector cannot be cast to
> > org.apache.drill.exec.vector.VariableWidthVector while trying to read a
> > spark INT96 datetime field on Drill 1.11 in spite of setting the property
> > store.parquet.reader.int96_as_timestamp to  true.
> >
> > I believe this was fixed in drill 1.10(
> > https://issues.apache.org/jira/browse/DRILL-4373). What could be wrong.
> >
> > Regards,
> > Rahul
> >
> > --
> >  This email and any files transmitted with it are confidential and
> > intended solely for the use of the individual or entity to whom it is
> > addressed. If you are not the named addressee then you should not
> > disseminate, distribute or copy this e-mail. Please notify the sender
> > immediately and delete this e-mail from your system.
> >
>

-- 
 This email and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to whom it is 
addressed. If you are not the named addressee then you should not 
disseminate, distribute or copy this e-mail. Please notify the sender 
immediately and delete this e-mail from your system.

Re: Does Drill support Full-text search as in Elasticsearch

2017-12-06 Thread Saurabh Mahapatra 
Hey Ayush,

Just to add to what Aman said: I think the request that I have heard is a
SQL interface to Elasticsearch especially when you are joining result with
another data source like a RDMBS. Is that what you are trying to
accomplish? But the guts of the technology i.e. indexing, search, sharding
etc. is different.

Nice to see a IIT KGP email in the thread. Welcome. Are you using this for
a project at the insti?:)

Saurabh

On Wed, Dec 6, 2017 at 4:09 PM, Aman Sinha  wrote:

> Hi,
> no, currently Drill does not support full-text search and it is unlikely it
> will do so natively in the near future.  Instead, since other products such
> as Elasticsearch are specialized for it, it would be better to add ES
> storage plugins in Drill to push down such full-text search filters into
> ES.  There was some effort in the past by the community to develop that but
> it did not make it into the product.
>
> -Aman
>
> On Tue, Dec 5, 2017 at 5:08 AM, Ayush Pandey  >
> wrote:
>
> > Hello everyone,
> > I am trying to benchmark Apache Drill Vs Elastic Search. The first
> > question that I am trying to answer is: Whether Apache Drill supports
> > Full-Text search as is the case with Elasticsearch.
> >
> > I have dumped data in Drill in Apache Parquet format.
>

Re: Does Drill support Full-text search as in Elasticsearch

2017-12-06 Thread Aman Sinha 
Hi,
no, currently Drill does not support full-text search and it is unlikely it
will do so natively in the near future.  Instead, since other products such
as Elasticsearch are specialized for it, it would be better to add ES
storage plugins in Drill to push down such full-text search filters into
ES.  There was some effort in the past by the community to develop that but
it did not make it into the product.

-Aman

On Tue, Dec 5, 2017 at 5:08 AM, Ayush Pandey 
wrote:

> Hello everyone,
> I am trying to benchmark Apache Drill Vs Elastic Search. The first
> question that I am trying to answer is: Whether Apache Drill supports
> Full-Text search as is the case with Elasticsearch.
>
> I have dumped data in Drill in Apache Parquet format.