Re: Re: tez compile
Hi, Now, Set the repo for https://repository.cloudera.com/artifactory/cloudera-repos , and version for 2.3.0-cdh5.1.0. then, run "mvn package -P\!hadoop24 -P\!hadoop26 -DskipTests=true" but, compile to tez-mapreduce module error! error information: /workspace/tez/apache-tez-0.7.0/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/hadoop/mapreduce/JobContextImpl.java:[57,8] org.apache.tez.mapreduce.hadoop.mapreduce.JobContextImpl is not abstract and does not override abstract method userClassesTakesPrecedence() in org.apache.hadoop.mapreduce.JobContext Is it caused by version does not match ? thanks for your help! hanked...@emar.com From: Hitesh Shah Date: 2015-08-20 23:00 To: user Subject: Re: tez compile For the most part, yes - the compile should likely work given that the CDH release is based off Apache Hadoop 2.3. To build Tez, you will need to add the CDH maven repo to the pom.xml at the top-level. Modify hadoop.version to "2.3.0-cdh5.1.0” or something similar based on your requirements. And then run "mvn package -P\!hadoop24 -P\!hadoop26" so that you disable all the features/modules that need a minimum of hadoop-2.4 and hadoop-2.6 respectively. Let us know if you run into any problems. thanks — Hitesh On Aug 20, 2015, at 3:29 AM, hanked...@emar.com wrote: > > > Hi~ all > > Can use cdh-hadoop 2.3 to compile tez 7.0 ? > > Wirtten in the building.txt is support, lowest hadoop version 2.2 ? > > > Now, my cluster with CDH 5.1 version, can use tez ? > > Thanks ! > > > > hanked...@emar.com
Re: Tez UI Kerberos
You could take a separate machine/VM and install ambari-server-2.1.0 on it? But yes, agreed - the Tez UI view does not work with Ambari-1.7.0. — Hitesh On Aug 20, 2015, at 3:38 PM, Gagan Brahmi wrote: > That would work in a Ambari 2.X environment I believe, but an environment > which is managed via Ambari 1.7 it becomes tricky I guess. > > My cluster is a managed by Ambari 1.7. > > On Thu, Aug 20, 2015 at 2:55 PM, Hitesh Shah wrote: > The other approach to this is to have an intermediate proxy that is > authenticated and can make calls to the ATS server. The Tez UI makes 2 kind > of calls to the backend services ( one set to YARN ResourceManager and the > other to Timeline server ) so both of these calls would need to be proxied > through a server/process that is kerberos authenticated. > > A simple option for this would be try using Ambari in standalone mode. In > this mode, the ambari-server acts as an authenticated proxy for the UI. This > involves installing the Ambari Server, setting it up to work in a secure mode > and instantiating the Tez View within it. All your users can then use the Tez > UI within Ambari without needing any kerberos auth. Ambari-server would be > started with kerberos auth and you need to configure that user as a hadoop > proxy user. To be clear, this does *not* require you to set up your Hadoop > cluster using Ambari. > > http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/ch_running_ambari_standalone.html > http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/ch_configuring_views_for_kerberos.html > http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/ch_using_tez_view.html > > thanks > — Hitesh > > > > On Aug 20, 2015, at 2:41 PM, Gagan Brahmi wrote: > > > Thanks Hitesh, but I don't want the UI to be accessed through Kerberos. > > > > Client to Tez UI communication should be without kerberos and Tez UI to ATS > > will be over Kerberos. > > > > Anyone accomplished this before? > > > > > > On Thu, Aug 20, 2015 at 1:57 PM, Hitesh Shah wrote: > > You will first need to do a kinit and then start a new firefox session with > > the following config “network.negotiate-auth.trusted-uris“ set up as needed. > > > > Ref on setting up firefox: > > http://people.redhat.com/mikeb/negotiate/ > > http://docs.oracle.com/cd/E41633_01/pt853pbh1/eng/pt/tsec/task_EnablingKerberosAuthenticationinFirefox-836673.html > > > > — Hitesh > > > > On Aug 20, 2015, at 1:00 PM, Gagan Brahmi wrote: > > > > > Does anyone has an idea how to enable Tez UI in a kerberos enabled > > > environment? > > > > > > I am hosting tez UI on apache and the ATS is secured. I am getting errors > > > where Tez UI is not able to retrieve the data from Timeline server. > > > > > > Couldn't find any documentation which can provide help with this one. > > > > > > > > > Regards, > > > Gagan Brahmi > > > > > >
Re: Tez UI Kerberos
That would work in a Ambari 2.X environment I believe, but an environment which is managed via Ambari 1.7 it becomes tricky I guess. My cluster is a managed by Ambari 1.7. On Thu, Aug 20, 2015 at 2:55 PM, Hitesh Shah wrote: > The other approach to this is to have an intermediate proxy that is > authenticated and can make calls to the ATS server. The Tez UI makes 2 > kind of calls to the backend services ( one set to YARN ResourceManager and > the other to Timeline server ) so both of these calls would need to be > proxied through a server/process that is kerberos authenticated. > > A simple option for this would be try using Ambari in standalone mode. In > this mode, the ambari-server acts as an authenticated proxy for the UI. > This involves installing the Ambari Server, setting it up to work in a > secure mode and instantiating the Tez View within it. All your users can > then use the Tez UI within Ambari without needing any kerberos auth. > Ambari-server would be started with kerberos auth and you need to configure > that user as a hadoop proxy user. To be clear, this does *not* require you > to set up your Hadoop cluster using Ambari. > > > http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/ch_running_ambari_standalone.html > > http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/ch_configuring_views_for_kerberos.html > > http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/ch_using_tez_view.html > > thanks > — Hitesh > > > > On Aug 20, 2015, at 2:41 PM, Gagan Brahmi wrote: > > > Thanks Hitesh, but I don't want the UI to be accessed through Kerberos. > > > > Client to Tez UI communication should be without kerberos and Tez UI to > ATS will be over Kerberos. > > > > Anyone accomplished this before? > > > > > > On Thu, Aug 20, 2015 at 1:57 PM, Hitesh Shah wrote: > > You will first need to do a kinit and then start a new firefox session > with the following config “network.negotiate-auth.trusted-uris“ set up as > needed. > > > > Ref on setting up firefox: > > http://people.redhat.com/mikeb/negotiate/ > > > http://docs.oracle.com/cd/E41633_01/pt853pbh1/eng/pt/tsec/task_EnablingKerberosAuthenticationinFirefox-836673.html > > > > — Hitesh > > > > On Aug 20, 2015, at 1:00 PM, Gagan Brahmi wrote: > > > > > Does anyone has an idea how to enable Tez UI in a kerberos enabled > environment? > > > > > > I am hosting tez UI on apache and the ATS is secured. I am getting > errors where Tez UI is not able to retrieve the data from Timeline server. > > > > > > Couldn't find any documentation which can provide help with this one. > > > > > > > > > Regards, > > > Gagan Brahmi > > > > > >
Re: Tez UI Kerberos
The other approach to this is to have an intermediate proxy that is authenticated and can make calls to the ATS server. The Tez UI makes 2 kind of calls to the backend services ( one set to YARN ResourceManager and the other to Timeline server ) so both of these calls would need to be proxied through a server/process that is kerberos authenticated. A simple option for this would be try using Ambari in standalone mode. In this mode, the ambari-server acts as an authenticated proxy for the UI. This involves installing the Ambari Server, setting it up to work in a secure mode and instantiating the Tez View within it. All your users can then use the Tez UI within Ambari without needing any kerberos auth. Ambari-server would be started with kerberos auth and you need to configure that user as a hadoop proxy user. To be clear, this does *not* require you to set up your Hadoop cluster using Ambari. http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/ch_running_ambari_standalone.html http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/ch_configuring_views_for_kerberos.html http://docs.hortonworks.com/HDPDocuments/Ambari-2.1.0.0/bk_ambari_views_guide/content/ch_using_tez_view.html thanks — Hitesh On Aug 20, 2015, at 2:41 PM, Gagan Brahmi wrote: > Thanks Hitesh, but I don't want the UI to be accessed through Kerberos. > > Client to Tez UI communication should be without kerberos and Tez UI to ATS > will be over Kerberos. > > Anyone accomplished this before? > > > On Thu, Aug 20, 2015 at 1:57 PM, Hitesh Shah wrote: > You will first need to do a kinit and then start a new firefox session with > the following config “network.negotiate-auth.trusted-uris“ set up as needed. > > Ref on setting up firefox: > http://people.redhat.com/mikeb/negotiate/ > http://docs.oracle.com/cd/E41633_01/pt853pbh1/eng/pt/tsec/task_EnablingKerberosAuthenticationinFirefox-836673.html > > — Hitesh > > On Aug 20, 2015, at 1:00 PM, Gagan Brahmi wrote: > > > Does anyone has an idea how to enable Tez UI in a kerberos enabled > > environment? > > > > I am hosting tez UI on apache and the ATS is secured. I am getting errors > > where Tez UI is not able to retrieve the data from Timeline server. > > > > Couldn't find any documentation which can provide help with this one. > > > > > > Regards, > > Gagan Brahmi > >
Re: Tez UI Kerberos
Thanks Hitesh, but I don't want the UI to be accessed through Kerberos. Client to Tez UI communication should be without kerberos and Tez UI to ATS will be over Kerberos. Anyone accomplished this before? On Thu, Aug 20, 2015 at 1:57 PM, Hitesh Shah wrote: > You will first need to do a kinit and then start a new firefox session > with the following config “network.negotiate-auth.trusted-uris“ set up as > needed. > > Ref on setting up firefox: > http://people.redhat.com/mikeb/negotiate/ > > http://docs.oracle.com/cd/E41633_01/pt853pbh1/eng/pt/tsec/task_EnablingKerberosAuthenticationinFirefox-836673.html > > — Hitesh > > On Aug 20, 2015, at 1:00 PM, Gagan Brahmi wrote: > > > Does anyone has an idea how to enable Tez UI in a kerberos enabled > environment? > > > > I am hosting tez UI on apache and the ATS is secured. I am getting > errors where Tez UI is not able to retrieve the data from Timeline server. > > > > Couldn't find any documentation which can provide help with this one. > > > > > > Regards, > > Gagan Brahmi > >
Re: Tez UI Kerberos
You will first need to do a kinit and then start a new firefox session with the following config “network.negotiate-auth.trusted-uris“ set up as needed. Ref on setting up firefox: http://people.redhat.com/mikeb/negotiate/ http://docs.oracle.com/cd/E41633_01/pt853pbh1/eng/pt/tsec/task_EnablingKerberosAuthenticationinFirefox-836673.html — Hitesh On Aug 20, 2015, at 1:00 PM, Gagan Brahmi wrote: > Does anyone has an idea how to enable Tez UI in a kerberos enabled > environment? > > I am hosting tez UI on apache and the ATS is secured. I am getting errors > where Tez UI is not able to retrieve the data from Timeline server. > > Couldn't find any documentation which can provide help with this one. > > > Regards, > Gagan Brahmi
Tez UI Kerberos
Does anyone has an idea how to enable Tez UI in a kerberos enabled environment? I am hosting tez UI on apache and the ATS is secured. I am getting errors where Tez UI is not able to retrieve the data from Timeline server. Couldn't find any documentation which can provide help with this one. Regards, Gagan Brahmi
Re: tez compile
For the most part, yes - the compile should likely work given that the CDH release is based off Apache Hadoop 2.3. To build Tez, you will need to add the CDH maven repo to the pom.xml at the top-level. Modify hadoop.version to "2.3.0-cdh5.1.0” or something similar based on your requirements. And then run "mvn package -P\!hadoop24 -P\!hadoop26" so that you disable all the features/modules that need a minimum of hadoop-2.4 and hadoop-2.6 respectively. Let us know if you run into any problems. thanks — Hitesh On Aug 20, 2015, at 3:29 AM, hanked...@emar.com wrote: > > > Hi~ all > > Can use cdh-hadoop 2.3 to compile tez 7.0 ? > > Wirtten in the building.txt is support, lowest hadoop version 2.2 ? > > > Now, my cluster with CDH 5.1 version, can use tez ? > > Thanks ! > > > > hanked...@emar.com
tez compile
Hi~ all Can use cdh-hadoop 2.3 to compile tez 7.0 ? Wirtten in the building.txt is support, lowest hadoop version 2.2 ? Now, my cluster with CDH 5.1 version, can use tez ? Thanks ! hanked...@emar.com
