Hi Timo,
Am 05.02.19 um 21:24 schrieb Timo Burmeister:
> Hi Lav,
>
> the problem is, that the ambari hawq install process wants to install
> "/usr/bin/yum -d 0 -e 0 -y install hawq ".
> 2019-02-05 10:22:53,291 - Package['hawq'] {'retry_on_repo_unavailability':
> False, 'retry_count': 5}
> 2019-02-05 10:22:53,379 - Installing package hawq ('/usr/bin/yum -d 0 -e 0 -y
> install hawq')
> 2019-02-05 10:22:57,199 - Execution of '/usr/bin/yum -d 0 -e 0 -y install
> hawq' returned 1. Error: Nothing to do
>
> But the package now should be 'apache-hawq'!
>
> How can I change the the ambari hawq install, to process "/usr/bin/yum -d 0
> -e 0 -y install apache-hawq"
I just solved that using Lav's help.
You need to do two things:
Modify file
/var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/metainfo.xml
and change package name in the following section
<osSpecifics>
<osSpecific>
<osFamily>any</osFamily>
<packages>
<package>
<name>hawq</name>
</package>
</packages>
</osSpecific>
</osSpecifics>
from hawq to apache-hawq.
Modify
/var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
as follows:
[ec2-user@mds-ambari scripts]$ diff hawq_constants.py.bak hawq_constants.py
42,43c42,43
< hawq_home_dir = "/usr/local/hawq/"
< hawq_config_dir = "/usr/local/hawq/etc/"
---
> hawq_home_dir = "/usr/local/apache-hawq/"
> hawq_config_dir = "/usr/local/apache-hawq/etc/"
Thanks again to Lav for helping me to figure this out.
Cheers,
Harald
>
> Thanks
> Timo
>
> Am 01.02.19, 19:08 schrieb "Lav Jain" <[email protected]>:
>
> Glad to hear that you were able to install HAWQ using Ambari. The bug is
> in
> Ambari codebase as there has been no active development on the HAWQ common
> service in a while.
>
> Regards,
>
>
> *Lav Jain*
> *Pivotal Data*
>
> [email protected]
>
>
> On Thu, Jan 31, 2019 at 11:53 PM Harald Bögeholz
> <[email protected]> wrote:
>
> > Am 31.01.19 um 13:30 schrieb Lav Jain:
> > > Hi Harald,
> > >
> > > Most HAWQ constants are defined inside
> > >
> >
> /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/package/scripts/hawq_constants.py
> >
> > Thank you very much again. That helped a lot, I finally got HAWQ up and
> > running although I still had to manually fix some installation issues
> > like distributing ssh keys for gpadmin.
> >
> > > The last version that we used to deploy HAWQ with Ambari (ver 2.5.1)
> was
> > > over a year ago. You may also take a look at http://www.oushu.io for
> any
> > > updated instructions.
> >
> > I am wondering if these installation issues should be filed as a bug
> > report somewhere? But is it a bug in Ambari or a bug in the Ambari
> > plugin which is part of HAWQ? I don't know enough about these things to
> > determine that.
> >
> > Anyway, my biggest problem is solved for now.
> >
> > (PXF installation also failed, but I'll just ignore that until I
> > actually need it.)
> >
> >
> > Regards,
> > Harald
> >
> > >
> > >
> > > Regards,
> > >
> > >
> > > *Lav Jain*
> > > *Pivotal Data*
> > >
> > > [email protected]
> > >
> > >
> > > On Tue, Jan 29, 2019 at 7:54 PM Harald Bögeholz
> > > <[email protected]> wrote:
> > >
> > >> Hi Lav,
> > >>
> > >>
> > >> thanks for your quick help. I modified
> > >>
> /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/metainfo.xml
> > >> accordingly and restarted my Ambari server. Then I deleted the
> services
> > >> PXF and HAWQ and tried adding them again.
> > >>
> > >> This time it failed a little later in the process with
> > >>
> > >> resource_management.core.exceptions.ExecutionFailed: Execution of
> 'chown
> > >> -R gpadmin:gpadmin /usr/local/hawq/' returned 1. chown: cannot access
> > >> '/usr/local/hawq/': No such file or directory
> > >>
> > >> Indeed this directory doesn't exist; instead I have
> > /usr/local/apache-hawq
> > >>
> > >> Apparently something else has to be fixed but I am completely
> unfamiliar
> > >> with the inner workings of Ambari.
> > >>
> > >> Is it possible at all to use Ambari to deploy HAWQ 2.4.0? I was
> > >> following Ruilong Huo's recommendation earlier on this list to use
> HDP
> > >> 2.6.1 and Ambari 2.5.1.
> > >>
> > >>
> > >> Thanks
> > >> Harald
> > >>
> > >> Am 30.01.19 um 10:52 schrieb Lav Jain:
> > >>> I believe you need to
> > >>> modify
> > >>
> ambari-server/src/main/resources/common-services/HAWQ/2.0.0/metainfo.xml
> > >>>
> > >>> <osSpecifics>
> > >>> <osSpecific>
> > >>> <osFamily>any</osFamily>
> > >>> <packages>
> > >>> <package>
> > >>> <name>hawq</name>
> > >>> </package>
> > >>> </packages>
> > >>> </osSpecific>
> > >>> </osSpecifics>
> > >>>
> > >>> Regards,
> > >>>
> > >>>
> > >>> *Lav Jain*
> > >>> *Pivotal Data*
> > >>>
> > >>> [email protected]
> > >>>
> > >>>
> > >>> On Tue, Jan 29, 2019 at 3:10 PM Harald Bögeholz
> > >>> <[email protected]> wrote:
> > >>>
> > >>>> Am 29.01.19 um 17:51 schrieb shenxl:
> > >>>>> Hi Harald,
> > >>>>>
> > >>>>> the HAWQ Ambari plugin isn’t included in HAWQ binary distribition,
> > you
> > >>>> can get the plugin at .
> > >>>>>
> > https://github.com/apache/hawq/tree/master/contrib/hawq-ambari-plugin
> > >> <
> > >>>>
> https://github.com/apache/hawq/tree/master/contrib/hawq-ambari-plugin
> > >.
> > >>>>
> > >>>> Thank you for your help.
> > >>>>
> > >>>> I managed to install the Ambari plugin and setup a repository for
> the
> > >>>> HAWQ binaries.
> > >>>>
> > >>>> Now deployment fails with this error:
> > >>>>
> > >>>> resource_management.core.exceptions.ExecutionFailed: Execution of
> > >>>> '/usr/bin/yum -d 0 -e 0 -y install hawq' returned 1. Error:
> Nothing to
> > >> do
> > >>>>
> > >>>> I think it is because it should be yum install apache-hawq, but I
> > don't
> > >>>> know where to change this.
> > >>>>
> > >>>> Could I get another hint?
> > >>>>
> > >>>> Thanks
> > >>>> Harald
> > >>>>
> > >>>>>
> > >>>>>> ---------- Forwarded message ---------
> > >>>>>> From: Harald Bögeholz <[email protected]>
> > >>>>>> Date: 2019年1月29日周二 上午9:32
> > >>>>>> Subject: Re: Support Matrix for HAWQ
> > >>>>>> To: <[email protected] <mailto:[email protected]>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Am 23.01.19 um 15:01 schrieb Ruilong Huo:
> > >>>>>>> Hi Harald,
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> If you prefer to use binary, you can get 2.4 release binary at
> > >>>> http://hawq.apache.org/#download <http://hawq.apache.org/#download>
> > >>>>>>> For the support matrix, it works with RHEL 6.4+/7.2+ x86_64, HDP
> > >>>> 2.5.3/2.6.1, and Ambari 2.4.2, 2.5.1.
> > >>>>>>
> > >>>>>> I have a Hadoop cluster using HDP 2.6.1/Ambari 2.5.1 up and
> running
> > >> now.
> > >>>>>>
> > >>>>>> The HAWQ binary distribition doesn't seem to include the Ambari
> > >> plugin,
> > >>>>>> though.
> > >>>>>>
> > >>>>>> So I'm stuck and don't know how to deploy HAWQ.
> > >>>>>>
> > >>>>>> Could somebody point me to the right instructions?
> > >>>>>>
> > >>>>>>
> > >>>>>> Thanks
> > >>>>>> Harald
> > >>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Best regards,
> > >>>>>>> Ruilong Huo
> > >>>>>>> At 2019-01-23 09:35:21, "Harald Bögeholz" <
> > >> [email protected]
> > >>>> <mailto:[email protected]>.INVALID> wrote:
> > >>>>>>>> Hello,
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> I have seen this discussion and I'm a bit confused. I'm about
> to
> > >>>> setup a
> > >>>>>>>> new Hadoop cluster to run HAWQ on CentOS 7.
> > >>>>>>>>
> > >>>>>>>> What's the best way to get this up and running?
> > >>>>>>>>
> > >>>>>>>> I was thinking about starting with Hortonworks HDP. Which exact
> > >>>> version
> > >>>>>>>> should I use? I can't simply use the latest one?
> > >>>>>>>>
> > >>>>>>>> When I have Hadoop running, can I deploy HAWQ using Ambari? I
> > looked
> > >>>> in
> > >>>>>>>> the HAWQ wiki and there are instructions for compiling HAWQ,
> but I
> > >> was
> > >>>>>>>> hoping to be able to just drop the binaries somewhere for
> Ambari
> > to
> > >>>>>>>> deploy them?
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Thanks for any help
> > >>>>>>>> Harald
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Am 23.01.19 um 12:00 schrieb Yi JIN:
> > >>>>>>>>> Hi Timo,
> > >>>>>>>>>
> > >>>>>>>>> Yes, this is planned in version 2.5. The issue for tracking
> this
> > is
> > >>>> going
> > >>>>>>>>> to be added into the release schedule. I checked this with
> > release
> > >>>> manager
> > >>>>>>>>> for this requirement. Thanks.
> > >>>>>>>>>
> > >>>>>>>>> Best,
> > >>>>>>>>> Yi
> > >>>>>>>>>
> > >>>>>>>>> On Tue, Jan 15, 2019 at 8:20 PM Timo Burmeister <
> > >>>> [email protected] <mailto:[email protected]>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi Yi,
> > >>>>>>>>>>
> > >>>>>>>>>> do you think for HDP 3 we have to wait for Apache HAWQ 2.5?
> > >>>>>>>>>>
> > >>>>>>>>>> Best,
> > >>>>>>>>>>
> > >>>>>>>>>> Timo
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> i.V. Timo Burmeister
> > >>>>>>>>>>
> > >>>>>>>>>> Head of Innovationsmanagement
> > >>>>>>>>>> IT-Strategy & New Technologies
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> freenet AG
> > >>>>>>>>>> Deelbögenkamp 4c
> > >>>>>>>>>> D-22297 Hamburg
> > >>>>>>>>>>
> > >>>>>>>>>> Tel.: +49 (0) 4051306619
> > >>>>>>>>>> Fax: +49 (0) 4051306659
> > >>>>>>>>>>
> > >>>>>>>>>> [email protected]
> <mailto:[email protected]>
> > >>>>>>>>>>
> > >>>>>>>>>> www.mobilcom-debitel.de <http://www.mobilcom-debitel.de/> <
> > >>>> http://www.mobilcom-debitel.de <http://www.mobilcom-debitel.de/>>
> > >>>>>>>>>> www.freenet.de <http://www.freenet.de/>
> <http://www.freenet.de
> > <
> > >>>> http://www.freenet.de/>>
> > >>>>>>>>>> www.freenet-group.de <http://www.freenet-group.de/> <
> > >>>> http://www.freenet-group.de <http://www.freenet-group.de/>>
> > >>>>>>>>>>
> > >>>>>>>>>> Vorstand: Christoph Vilanek (Vorsitzender), Ingo Arnold,
> Stephan
> > >>>> Esch,
> > >>>>>>>>>> Antonius Fromme, Rickmann von Platen
> > >>>>>>>>>> Vorsitzender des Aufsichtsrats: Prof. Dr. Helmut Thoma
> > >>>>>>>>>> Sitz der Gesellschaft: Büdelsdorf
> > >>>>>>>>>> HRB 7306 KI, Amtsgericht Kiel
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> +++ freenet Group - EINE GRUPPE. ALLE MÖGLICHKEITEN - Hier <
> > >>>>>>>>>> https://stellenangebote.freenet-group.de/ <
> > >>>> https://stellenangebote.freenet-group.de/>> suchen wir noch
> > >> Verstärkung!
> > >>>>>>>>>> +++
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Am 15.01.19, 04:07 schrieb "Yi JIN" <[email protected] <mailto:
> > >>>> [email protected]>>:
> > >>>>>>>>>>
> > >>>>>>>>>> Hi Timo,
> > >>>>>>>>>>
> > >>>>>>>>>> I think HAWQ 2.4 supports HDP 2.6.1 and 2.5.3.
> > >>>>>>>>>>
> > >>>>>>>>>> Best,
> > >>>>>>>>>>
> > >>>>>>>>>> Yi
> > >>>>>>>>>>
> > >>>>>>>>>> On Sat, Dec 22, 2018 at 2:55 AM Timo Burmeister <
> > >>>>>>>>>> [email protected]
> <mailto:[email protected]>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> > Hi !
> > >>>>>>>>>> >
> > >>>>>>>>>> > Where can I find the supported Hadoop Version for the
> new
> > >>>> Apache
> > >>>>>>>>>> HAWQ 2.4
> > >>>>>>>>>> > Release?
> > >>>>>>>>>> >
> > >>>>>>>>>> > Does Apache HAWQ 2.4 support HDP 2.6 and HDP 3.0.1 ?
> > >>>>>>>>>> >
> > >>>>>>>>>> > Best regards
> > >>>>>>>>>> > Timo
> > >>>>>>>>>> >
> > >>>>>>>>>> >
> > >>>>>>>>>> >
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> > >
> >
> >
>
>
