Re: Enabling Tez-UI on CDH 5.8

2017-02-24 Thread Manoj Murumkar 
Also, please note that I am using ATS with apache 2.7.3 version.

Jan,

I looked at the post you had made earlier.

we are using Tez 0.8.4 successfully both with Hive and Pig on our
> Cloudera CDH 5.7.1 cluster.

CDH comes with Pig 0.12 which doesn't have tez as a supported engine.
How did you manage to get that working?


On Fri, Feb 24, 2017 at 10:50 AM, Manoj Murumkar 
wrote:

> So, I am able to get the metric posting part working (part where tez posts
> counters to backend). I can see data coming in and without any java
> exceptions in the run logs of tez sessions. I essentially built tez with
> 1.9.13 version of jackson libraries.
>
>  
> org.codehaus.jackson
> jackson-mapper-asl
> 1.9.13
>   
>   
> org.codehaus.jackson
> jackson-core-asl
> 1.9.13
>   
>   
> org.codehaus.jackson
> jackson-jaxrs
> 1.9.13
>   
>   
> org.codehaus.jackson
> jackson-xc
> 1.9.13
>
> Now, when I check the timeline server webUI, I see nulls in pretty much
> everything (see screenshot). Could this be another version incompatibility
> issue?
>
> Thanks,
>
> Manoj
>
>
>
> On Fri, Feb 24, 2017 at 9:43 AM, Edward Capriolo 
> wrote:
>
>> We should really just force tez into the build of hive. It sooo fin
>> stupid this situation. Just drives people at impala / spark. Im brining it
>> up in hive.
>>
>>
>> On Friday, February 24, 2017, Jan Morlock 
>> wrote:
>>
>>> Hi Manoj,
>>>
>>> sorry for the late reply. Your problem appears to be similar to
>>>
>>> https://issues.apache.org/jira/browse/HIVE-8271
>>>
>>> So please check whether you have both jackson-1.8 and jackson-1.9
>>> somewhere in your classpath.
>>>
>>> That being said, you should know, that you can make a lot of friends if
>>> you succeed in getting the Tez UI running on CDH.
>>> I also once tried but didn't pursue further. You can find my post here:
>>>
>>> http://mail-archives.apache.org/mod_mbox/tez-user/201608.mbo
>>> x/%3cCAO25eDBaHYcKaL+pBp2wR0TyXOcaMAsJU+YDvEmFVnYQer6ALg@mai
>>> l.gmail.com%3e
>>>
>>> I hope that helps.
>>>
>>> Cheers
>>> Jan
>>>
>>>
>>>
>>>
>>> 2017-02-17 0:02 GMT+01:00 Manoj Murumkar :
>>>
 Also, hive version that's bundled with CDH5.8 is 1.1.0 which is also
 bundled with same class. Perhaps that's the one in effect? If that's the
 case, is there no way to use timeline server for history with this version?

 On Thu, Feb 16, 2017 at 2:53 PM, Manoj Murumkar <
 manoj.murum...@gmail.com> wrote:

> Btw, here's the environment info:
>
> Tez: 0.8.4
> Timeline server: bundled with apache 2.6.5 version
>
> I have added following libraries (that are used by timeline server) in
> the path, so there's no mis-match, but no luck. I am still getting same
> error.
>
> HADOOP_CLASSPATH=/usr/local/tez/client/lib/old/jackson-core-
> asl-1.9.13.jar:/usr/local/tez/client/lib/old/jackson-mapper-
> asl-1.9.13.jar:
> HADOOP_USER_CLASSPATH_FIRST=true
>
>
>
> On Thu, Feb 16, 2017 at 12:02 PM, Manoj Murumkar <
> manoj.murum...@gmail.com> wrote:
>
>> Hi,
>>
>> Wondering if anyone has successfully enabled Tez-UI (by running ATS
>> from a separate apache installation) on CDH 5.8 cluster. Would appreciate
>> any information on this. We are have trouble with API incompatibility
>> (error pasted below):
>>
>> 2017-02-16 19:40:13,874 [FATAL] [HistoryEventHandlingThread] 
>> |yarn.YarnUncaughtExceptionHandler|: Thread 
>> Thread[HistoryEventHandlingThread,5,main] threw an Error.  Shutting down 
>> now...
>> java.lang.AbstractMethodError: 
>> org.codehaus.jackson.map.AnnotationIntrospector.findSerializer(Lorg/codehaus/jackson/map/introspect/Annotated;)Ljava/lang/Object;
>>  at 
>> org.codehaus.jackson.map.ser.BasicSerializerFactory.findSerializerFromAnnotation(BasicSerializerFactory.java:362)
>>  at 
>> org.codehaus.jackson.map.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:252)
>>  at 
>> org.codehaus.jackson.map.ser.StdSerializerProvider._createUntypedSerializer(StdSerializerProvider.java:782)
>>  at 
>> org.codehaus.jackson.map.ser.StdSerializerProvider._createAndCacheUntypedSerializer(StdSerializerProvider.java:735)
>>  at 
>> org.codehaus.jackson.map.ser.StdSerializerProvider.findValueSerializer(StdSerializerProvider.java:344)
>>  at 
>> org.codehaus.jackson.map.ser.StdSerializerProvider.findTypedValueSerializer(StdSerializerProvider.java:420)
>>  at 
>> org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:601)
>>  at 
>> org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:256)
>>  at 
>>

Re: Enabling Tez-UI on CDH 5.8

2017-02-24 Thread Jan Morlock 
Hi Manoj,

sorry for the late reply. Your problem appears to be similar to

https://issues.apache.org/jira/browse/HIVE-8271

So please check whether you have both jackson-1.8 and jackson-1.9 somewhere
in your classpath.

That being said, you should know, that you can make a lot of friends if you
succeed in getting the Tez UI running on CDH.
I also once tried but didn't pursue further. You can find my post here:

http://mail-archives.apache.org/mod_mbox/tez-user/201608.mbox/%3ccao25edbahyckal+pbp2wr0tyxocamasju+ydvemfvnyqer6...@mail.gmail.com%3e

I hope that helps.

Cheers
Jan




2017-02-17 0:02 GMT+01:00 Manoj Murumkar :

> Also, hive version that's bundled with CDH5.8 is 1.1.0 which is also
> bundled with same class. Perhaps that's the one in effect? If that's the
> case, is there no way to use timeline server for history with this version?
>
> On Thu, Feb 16, 2017 at 2:53 PM, Manoj Murumkar 
> wrote:
>
>> Btw, here's the environment info:
>>
>> Tez: 0.8.4
>> Timeline server: bundled with apache 2.6.5 version
>>
>> I have added following libraries (that are used by timeline server) in
>> the path, so there's no mis-match, but no luck. I am still getting same
>> error.
>>
>> HADOOP_CLASSPATH=/usr/local/tez/client/lib/old/jackson-core-
>> asl-1.9.13.jar:/usr/local/tez/client/lib/old/jackson-mapper-
>> asl-1.9.13.jar:
>> HADOOP_USER_CLASSPATH_FIRST=true
>>
>>
>>
>> On Thu, Feb 16, 2017 at 12:02 PM, Manoj Murumkar <
>> manoj.murum...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Wondering if anyone has successfully enabled Tez-UI (by running ATS from
>>> a separate apache installation) on CDH 5.8 cluster. Would appreciate any
>>> information on this. We are have trouble with API incompatibility (error
>>> pasted below):
>>>
>>> 2017-02-16 19:40:13,874 [FATAL] [HistoryEventHandlingThread] 
>>> |yarn.YarnUncaughtExceptionHandler|: Thread 
>>> Thread[HistoryEventHandlingThread,5,main] threw an Error.  Shutting down 
>>> now...
>>> java.lang.AbstractMethodError: 
>>> org.codehaus.jackson.map.AnnotationIntrospector.findSerializer(Lorg/codehaus/jackson/map/introspect/Annotated;)Ljava/lang/Object;
>>> at 
>>> org.codehaus.jackson.map.ser.BasicSerializerFactory.findSerializerFromAnnotation(BasicSerializerFactory.java:362)
>>> at 
>>> org.codehaus.jackson.map.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:252)
>>> at 
>>> org.codehaus.jackson.map.ser.StdSerializerProvider._createUntypedSerializer(StdSerializerProvider.java:782)
>>> at 
>>> org.codehaus.jackson.map.ser.StdSerializerProvider._createAndCacheUntypedSerializer(StdSerializerProvider.java:735)
>>> at 
>>> org.codehaus.jackson.map.ser.StdSerializerProvider.findValueSerializer(StdSerializerProvider.java:344)
>>> at 
>>> org.codehaus.jackson.map.ser.StdSerializerProvider.findTypedValueSerializer(StdSerializerProvider.java:420)
>>> at 
>>> org.codehaus.jackson.map.ser.StdSerializerProvider._serializeValue(StdSerializerProvider.java:601)
>>> at 
>>> org.codehaus.jackson.map.ser.StdSerializerProvider.serializeValue(StdSerializerProvider.java:256)
>>> at 
>>> org.codehaus.jackson.map.ObjectMapper.writeValue(ObjectMapper.java:1604)
>>> at 
>>> org.codehaus.jackson.jaxrs.JacksonJsonProvider.writeTo(JacksonJsonProvider.java:527)
>>> at 
>>> com.sun.jersey.api.client.RequestWriter.writeRequestEntity(RequestWriter.java:300)
>>> at 
>>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:204)
>>> at 
>>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
>>> at 
>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineJerseyRetryFilter$1.run(TimelineClientImpl.java:226)
>>> at 
>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineClientConnectionRetry.retryOn(TimelineClientImpl.java:162)
>>> at 
>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineJerseyRetryFilter.handle(TimelineClientImpl.java:237)
>>> at com.sun.jersey.api.client.Client.handle(Client.java:648)
>>> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
>>> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>>> at 
>>> com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:563)
>>> at 
>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.doPostingObject(TimelineClientImpl.java:472)
>>> at 
>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.doPosting(TimelineClientImpl.java:321)
>>> at 
>>> org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:301)
>>> at 
>>> org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService.handleEvents(ATSHistoryLoggingService.java:357)
>>> at 
>>>

Re: simple application on tez + llap

2017-02-24 Thread Harish JP 
I do not think we can run arbitrary jobs on LLAP. 
https://cwiki.apache.org/confluence/display/Hive/LLAP. "Only Hive code and 
blessed UDFs are accepted in LLAP."


On 24-Feb-2017, at 4:29 PM, Patcharee Thongtra 
> wrote:

Hi,

I found an example of simple applications like wordcount running on tez - 
https://github.com/apache/tez/tree/master/tez-examples/src/main/java/org/apache/tez/examples.
 However, how to run this on tez+llap? Any suggestions?

BR,

Patcharee

simple application on tez + llap

2017-02-24 Thread Patcharee Thongtra 

Hi,

I found an example of simple applications like wordcount running on tez 
- 
https://github.com/apache/tez/tree/master/tez-examples/src/main/java/org/apache/tez/examples. 
However, how to run this on tez+llap? Any suggestions?


BR,

Patcharee