Re:Re: Re: Impala make install

[sky]

Hi Tim,
I experimented with the manual copy of the file. But it be found that If 
you want to manually copy the file, then a lot of directory structure will be 
changed. And then  I found that the path set in some variable settings file is 
hard-coded. Looking for the file and then changing it is a lot of work. What 
are the files that you have manually copied ? How do you do it ? 







At 2017-06-22 08:14:36, "Tim Armstrong"  wrote:
> THIS IS NOT AN OFFICIALLY SUPPORTED DEPLOYMENT METHOD *
>I believe the artifacts that are actually used at runtime are impalad,
>catalogd, statestored, libfesupport.so, impala-frontend-0.1-SNAPSHOT.jar,
>impala-data-source-api-1.0-SNAPSHOT.jar, libstdc++.so* libkudu_client.so*
>
>You may be able to get something to work if you replace the versions
>currently on the cluster with the output from your Impala build. This is a
>purely experimental method of deployment and may or may not work as
>expected. If your cluster catches on fire, don't blame me.
>
>
>On Wed, Jun 21, 2017 at 4:03 PM, 孙清孟  wrote:
>
>> Hi Jeszy,
>>Sorry that I didn't make it clear, I've already installed impala servive
>> with CM.
>>I did some modification in Impala, and built it successfully.
>>now I want to replace the impala installed by CM.
>>
>> 2017-06-21 19:29 GMT+08:00 Jeszy :
>>
>> > With CM you can just add Impala as a new service to your cluster. Use
>> > the dropdown next to the cluster name. A binary version of impala is
>> > shipped as part of the CDH 5.11 parcels that you have installed.
>> >
>> > On 21 June 2017 at 11:58, 孙清孟  wrote:
>> > > Hi Tim,
>> > >   I've built Impala according to the describe here:
>> > > https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala
>> > >   But how can I install the Impala to an already running
>> > cdh-5.11.0-release
>> > > cluster that managered by Cloudera Manager.
>> > >   Build Debian packages and use `apt-get`?
>> > >
>> > > 2017-06-21 11:16 GMT+08:00 Henry Robinson :
>> > >
>> > >> I don't think there's any plan for this work. The CMake documentation
>> > would
>> > >> be where I'd start looking for ideas:
>> > >>
>> > >> https://cmake.org/cmake/help/v3.2/command/install.html
>> > >>
>> > >> Best,
>> > >> Henry
>> > >>
>> > >> On 20 June 2017 at 18:31, sky  wrote:
>> > >>
>> > >> > Hi Tim，
>> > >> >Is there a plan for this work? Could you provide a manual copy of
>> > the
>> > >> > example?Thanks.
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> >
>> > >> > At 2017-06-21 01:41:33, "Tim Armstrong" 
>> > wrote:
>> > >> > >Hi Sky,
>> > >> > >  We have not implemented an install target yet - for deployment we
>> > rely
>> > >> > on
>> > >> > >copying out the artifacts manually. I believe CMake has some
>> support
>> > for
>> > >> > >implementing install targets but nobody has picked up that work
>> yet.
>> > >> > >
>> > >> > >- Tim
>> > >> > >
>> > >> > >On Mon, Jun 19, 2017 at 8:45 PM, sky  wrote:
>> > >> > >
>> > >> > >> Hi all,
>> > >> > >> I am using cdh5.11.1-release,the compilation command is
>> > provided
>> > >> in
>> > >> > >> the documentation(./buildall.sh -notests -so),but there is no
>> > command
>> > >> > >> similar to 'make install'.In the current document compiled, the
>> > >> > directory
>> > >> > >> structure is too much and do not need too many files. Could you
>> > >> provide
>> > >> > an
>> > >> > >> "install" command to extract compiled files to other directories
>> > for
>> > >> > easy
>> > >> > >> management
>> > >> >
>> > >>
>> >
>>

Re: Re: Impala make install

[Tim Armstrong]

 THIS IS NOT AN OFFICIALLY SUPPORTED DEPLOYMENT METHOD *
I believe the artifacts that are actually used at runtime are impalad,
catalogd, statestored, libfesupport.so, impala-frontend-0.1-SNAPSHOT.jar,
impala-data-source-api-1.0-SNAPSHOT.jar, libstdc++.so* libkudu_client.so*

You may be able to get something to work if you replace the versions
currently on the cluster with the output from your Impala build. This is a
purely experimental method of deployment and may or may not work as
expected. If your cluster catches on fire, don't blame me.


On Wed, Jun 21, 2017 at 4:03 PM, 孙清孟  wrote:

> Hi Jeszy,
>Sorry that I didn't make it clear, I've already installed impala servive
> with CM.
>I did some modification in Impala, and built it successfully.
>now I want to replace the impala installed by CM.
>
> 2017-06-21 19:29 GMT+08:00 Jeszy :
>
> > With CM you can just add Impala as a new service to your cluster. Use
> > the dropdown next to the cluster name. A binary version of impala is
> > shipped as part of the CDH 5.11 parcels that you have installed.
> >
> > On 21 June 2017 at 11:58, 孙清孟  wrote:
> > > Hi Tim,
> > >   I've built Impala according to the describe here:
> > > https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala
> > >   But how can I install the Impala to an already running
> > cdh-5.11.0-release
> > > cluster that managered by Cloudera Manager.
> > >   Build Debian packages and use `apt-get`?
> > >
> > > 2017-06-21 11:16 GMT+08:00 Henry Robinson :
> > >
> > >> I don't think there's any plan for this work. The CMake documentation
> > would
> > >> be where I'd start looking for ideas:
> > >>
> > >> https://cmake.org/cmake/help/v3.2/command/install.html
> > >>
> > >> Best,
> > >> Henry
> > >>
> > >> On 20 June 2017 at 18:31, sky  wrote:
> > >>
> > >> > Hi Tim，
> > >> >Is there a plan for this work? Could you provide a manual copy of
> > the
> > >> > example?Thanks.
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > At 2017-06-21 01:41:33, "Tim Armstrong" 
> > wrote:
> > >> > >Hi Sky,
> > >> > >  We have not implemented an install target yet - for deployment we
> > rely
> > >> > on
> > >> > >copying out the artifacts manually. I believe CMake has some
> support
> > for
> > >> > >implementing install targets but nobody has picked up that work
> yet.
> > >> > >
> > >> > >- Tim
> > >> > >
> > >> > >On Mon, Jun 19, 2017 at 8:45 PM, sky  wrote:
> > >> > >
> > >> > >> Hi all,
> > >> > >> I am using cdh5.11.1-release,the compilation command is
> > provided
> > >> in
> > >> > >> the documentation(./buildall.sh -notests -so),but there is no
> > command
> > >> > >> similar to 'make install'.In the current document compiled, the
> > >> > directory
> > >> > >> structure is too much and do not need too many files. Could you
> > >> provide
> > >> > an
> > >> > >> "install" command to extract compiled files to other directories
> > for
> > >> > easy
> > >> > >> management
> > >> >
> > >>
> >
>

Re: Re: Impala make install

[孙清孟]

Hi Jeszy,
   Sorry that I didn't make it clear, I've already installed impala servive
with CM.
   I did some modification in Impala, and built it successfully.
   now I want to replace the impala installed by CM.

2017-06-21 19:29 GMT+08:00 Jeszy :

> With CM you can just add Impala as a new service to your cluster. Use
> the dropdown next to the cluster name. A binary version of impala is
> shipped as part of the CDH 5.11 parcels that you have installed.
>
> On 21 June 2017 at 11:58, 孙清孟  wrote:
> > Hi Tim,
> >   I've built Impala according to the describe here:
> > https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala
> >   But how can I install the Impala to an already running
> cdh-5.11.0-release
> > cluster that managered by Cloudera Manager.
> >   Build Debian packages and use `apt-get`?
> >
> > 2017-06-21 11:16 GMT+08:00 Henry Robinson :
> >
> >> I don't think there's any plan for this work. The CMake documentation
> would
> >> be where I'd start looking for ideas:
> >>
> >> https://cmake.org/cmake/help/v3.2/command/install.html
> >>
> >> Best,
> >> Henry
> >>
> >> On 20 June 2017 at 18:31, sky  wrote:
> >>
> >> > Hi Tim，
> >> >Is there a plan for this work? Could you provide a manual copy of
> the
> >> > example?Thanks.
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > At 2017-06-21 01:41:33, "Tim Armstrong" 
> wrote:
> >> > >Hi Sky,
> >> > >  We have not implemented an install target yet - for deployment we
> rely
> >> > on
> >> > >copying out the artifacts manually. I believe CMake has some support
> for
> >> > >implementing install targets but nobody has picked up that work yet.
> >> > >
> >> > >- Tim
> >> > >
> >> > >On Mon, Jun 19, 2017 at 8:45 PM, sky  wrote:
> >> > >
> >> > >> Hi all,
> >> > >> I am using cdh5.11.1-release,the compilation command is
> provided
> >> in
> >> > >> the documentation(./buildall.sh -notests -so),but there is no
> command
> >> > >> similar to 'make install'.In the current document compiled, the
> >> > directory
> >> > >> structure is too much and do not need too many files. Could you
> >> provide
> >> > an
> >> > >> "install" command to extract compiled files to other directories
> for
> >> > easy
> >> > >> management
> >> >
> >>
>

Re: Broken build from Sentry

[Thomas Tauber-Marshall]

The problem has now been fixed on the Sentry side. I have a patch out to
revert back to 5.13 for Sentry.

For anyone who's curious: the problem turned out to be that Sentry wasn't
shading all of their packages through maven, which resulted in dependency
conflicts since we use a different version of thrift than they do.

On Tue, Jun 20, 2017 at 11:44 AM Tim Armstrong 
wrote:

> Yeah my change should build against either version by design.
>
> On 20 Jun. 2017 9:21 am, "Henry Robinson"  wrote:
>
> > Yes, I did. AFAICT it worked fine.
> >
> > On 20 June 2017 at 09:19, Alexander Behm  wrote:
> >
> > > Henry, did you try the revert on top of Tim's already-checked-in
> change?
> > >
> > > On Tue, Jun 20, 2017 at 9:18 AM, Alexander Behm <
> alex.b...@cloudera.com>
> > > wrote:
> > >
> > > > Let's revert the version to buy us some time. That solution is a
> > ticking
> > > > time bomb though since that version will disappear soon.
> > > >
> > > > On Tue, Jun 20, 2017 at 8:56 AM, Henry Robinson 
> > > wrote:
> > > >
> > > >> I was able to run a build with EE and FE tests with Sentry reverted
> to
> > > >> 5.12
> > > >> - unless there are objections I'm going to post a patch to revert
> the
> > > >> version bump.
> > > >>
> > > >> On 20 June 2017 at 06:53, Thomas Tauber-Marshall <
> > > tmarsh...@cloudera.com>
> > > >> wrote:
> > > >>
> > > >> > So we've had a successful run of the nightlies now, and I've
> > uploaded
> > > >> the
> > > >> > new jars to the s3 bucket, but Sentry still fails for some reason.
> > > >> >
> > > >> > I filed: https://issues.apache.org/jira/browse/IMPALA-5540 to
> track
> > > >> this
> > > >> >
> > > >> > On Tue, Jun 20, 2017 at 1:25 AM Alexander Kolbasov <
> > > ak...@cloudera.com>
> > > >> > wrote:
> > > >> >
> > > >> > > Note that Apache upstream story is more complicated - there was
> a
> > > >> change
> > > >> > > done upstream that refactored a bunch of Sentry code that will
> > cause
> > > >> > > similar issue (I think it is SENTRY-1205). The change is present
> > in
> > > >> > Sentry
> > > >> > > master but not in upstream sentry HA branch.
> > > >> > >
> > > >> > > On Mon, Jun 19, 2017 at 11:02 PM, Dimitris Tsirogiannis <
> > > >> > > dtsirogian...@cloudera.com> wrote:
> > > >> > >
> > > >> > > > +Sasha, who I believe has more up-to-date information on this.
> > > >> > > >
> > > >> > > > On Mon, Jun 19, 2017 at 10:56 PM, Henry Robinson <
> > > he...@apache.org>
> > > >> > > wrote:
> > > >> > > >
> > > >> > > >> FWIW, I've been able to start Sentry by setting:
> > > >> > > >>
> > > >> > > >> export IMPALA_SENTRY_VERSION=1.5.1-cdh5.12.0-SNAPSHOT
> > > >> > > >>
> > > >> > > >> (i.e. rolling back to the previous version of Sentry). I
> > haven't
> > > >> yet
> > > >> > > tried
> > > >> > > >> to run tests - does anyone know an ETA for a fix coming out
> of
> > > >> > Cloudera
> > > >> > > >> for
> > > >> > > >> the 5.13-SNAPSHOT? If it might be a while, we should consider
> > > >> > regressing
> > > >> > > >> the Sentry version to unblock checkins.
> > > >> > > >>
> > > >> > > >> On 19 June 2017 at 15:31, Tim Armstrong <
> > tarmstr...@cloudera.com
> > > >
> > > >> > > wrote:
> > > >> > > >>
> > > >> > > >> > It's unfortunately not that simple. The API change has been
> > in
> > > >> > Apache
> > > >> > > >> > sentry
> > > >> > > >> >
> > > >> > > >> > So rolling back the API change temporarily solves the
> problem
> > > for
> > > >> > > >> Cloudera,
> > > >> > > >> > but we're going to have to deal with it at some point and
> get
> > > >> Impala
> > > >> > > >> > building against both versions of the API.
> > > >> > > >> >
> > > >> > > >> > On Mon, Jun 19, 2017 at 2:55 PM, Thomas Tauber-Marshall <
> > > >> > > >> > tmarsh...@cloudera.com> wrote:
> > > >> > > >> >
> > > >> > > >> > > Yes, the Sentry team has been contacted and they're going
> > to
> > > be
> > > >> > > >> rolling
> > > >> > > >> > it
> > > >> > > >> > > back.
> > > >> > > >> > >
> > > >> > > >> > > On Mon, Jun 19, 2017 at 4:53 PM Todd Lipcon <
> > > t...@cloudera.com
> > > >> >
> > > >> > > >> wrote:
> > > >> > > >> > >
> > > >> > > >> > > > Quick question from a bystander: it seems like Sentry
> > > >> committed
> > > >> > an
> > > >> > > >> > > > API-incompatible change. Instead of fixing on the
> Impala
> > > >> side,
> > > >> > > >> should
> > > >> > > >> > the
> > > >> > > >> > > > Sentry project be notified that they may want to roll
> > back
> > > >> such
> > > >> > a
> > > >> > > >> > change?
> > > >> > > >> > > > It seems like an error on their part to do such a thing
> > > >> within a
> > > >> > > >> minor
> > > >> > > >> > > > version.
> > > >> > > >> > > >
> > > >> > > >> > > > On Mon, Jun 19, 2017 at 1:56 PM, Thomas
> Tauber-Marshall <
> > > >> > > >> > > > tmarsh...@cloudera.com> wrote:
> > > >> > > >> > > >
> > > >> > > >> > > > > I'm working on getting the s3 jars updated, which
> > > >> presumably
> > > >> > > will

IMPALA-2782 Allow Impala Shell to connect directly to impalad when config with proxy load balancer and kerberos

[Vincent Tran]

FYI. I sent in a patch for this, but I'm not sure which test would be
appropriate for this case. Let me know if you have suggestions...

Re: Re: Impala make install

[Jeszy]

With CM you can just add Impala as a new service to your cluster. Use
the dropdown next to the cluster name. A binary version of impala is
shipped as part of the CDH 5.11 parcels that you have installed.

On 21 June 2017 at 11:58, 孙清孟  wrote:
> Hi Tim,
>   I've built Impala according to the describe here:
> https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala
>   But how can I install the Impala to an already running cdh-5.11.0-release
> cluster that managered by Cloudera Manager.
>   Build Debian packages and use `apt-get`?
>
> 2017-06-21 11:16 GMT+08:00 Henry Robinson :
>
>> I don't think there's any plan for this work. The CMake documentation would
>> be where I'd start looking for ideas:
>>
>> https://cmake.org/cmake/help/v3.2/command/install.html
>>
>> Best,
>> Henry
>>
>> On 20 June 2017 at 18:31, sky  wrote:
>>
>> > Hi Tim，
>> >Is there a plan for this work? Could you provide a manual copy of the
>> > example?Thanks.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > At 2017-06-21 01:41:33, "Tim Armstrong"  wrote:
>> > >Hi Sky,
>> > >  We have not implemented an install target yet - for deployment we rely
>> > on
>> > >copying out the artifacts manually. I believe CMake has some support for
>> > >implementing install targets but nobody has picked up that work yet.
>> > >
>> > >- Tim
>> > >
>> > >On Mon, Jun 19, 2017 at 8:45 PM, sky  wrote:
>> > >
>> > >> Hi all,
>> > >> I am using cdh5.11.1-release,the compilation command is provided
>> in
>> > >> the documentation(./buildall.sh -notests -so),but there is no command
>> > >> similar to 'make install'.In the current document compiled, the
>> > directory
>> > >> structure is too much and do not need too many files. Could you
>> provide
>> > an
>> > >> "install" command to extract compiled files to other directories for
>> > easy
>> > >> management
>> >
>>

Re: Re: Impala make install

[孙清孟]

Hi Tim,
  I've built Impala according to the describe here:
https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala
  But how can I install the Impala to an already running cdh-5.11.0-release
cluster that managered by Cloudera Manager.
  Build Debian packages and use `apt-get`?

2017-06-21 11:16 GMT+08:00 Henry Robinson :

> I don't think there's any plan for this work. The CMake documentation would
> be where I'd start looking for ideas:
>
> https://cmake.org/cmake/help/v3.2/command/install.html
>
> Best,
> Henry
>
> On 20 June 2017 at 18:31, sky  wrote:
>
> > Hi Tim，
> >Is there a plan for this work? Could you provide a manual copy of the
> > example?Thanks.
> >
> >
> >
> >
> >
> >
> >
> >
> > At 2017-06-21 01:41:33, "Tim Armstrong"  wrote:
> > >Hi Sky,
> > >  We have not implemented an install target yet - for deployment we rely
> > on
> > >copying out the artifacts manually. I believe CMake has some support for
> > >implementing install targets but nobody has picked up that work yet.
> > >
> > >- Tim
> > >
> > >On Mon, Jun 19, 2017 at 8:45 PM, sky  wrote:
> > >
> > >> Hi all,
> > >> I am using cdh5.11.1-release,the compilation command is provided
> in
> > >> the documentation(./buildall.sh -notests -so),but there is no command
> > >> similar to 'make install'.In the current document compiled, the
> > directory
> > >> structure is too much and do not need too many files. Could you
> provide
> > an
> > >> "install" command to extract compiled files to other directories for
> > easy
> > >> management
> >
>