date:20170830

Re: [VOTE] 2.10.0 release candidate 1 (RC1)

2017-08-30 Thread Bharath Vissapragada

Thanks Tim. I'll cut another RC and post a separate [VOTE] thread.

On Wed, Aug 30, 2017 at 10:33 PM, Tim Armstrong 
wrote:

> Tests passed, I've pushed to release-2.10.0.
>
> - Tim
>
> On Wed, Aug 30, 2017 at 2:59 PM, Tim Armstrong 
> wrote:
>
> > There were a few other fixes for critical issues and usability bugs that
> > seemed worth including. I've put together a branch here that I will run
> > tests on. I was going to include these commits:
> >
> > 23d79462da5d0108709e8b1399c97606f4ebdf92 IMPALA-5855: reserve enough
> > memory for preaggs
> > a58394be7c7998a5dfea53d8a3dbf8beb3370a48 IMPALA-5850: Cast sender
> > partition exprs under unions.
> > 2912a0f9d9b32caf586b9383c7e027af3fe4c5c4 IMPALA-5857: avoid invalid free
> > of hedged read metrics
> > ebe8ddd451b3f14d3f778339978a76bcd14b2589 IMPALA-5830:
> > SET_DENY_RESERVATION_PROBABILITY test
> > 1faf89f047e7d78c3a1f3b518269a3ae21a4ddea IMPALA-5840: Don't write
> > page-level statistics in Parquet files.
> > 73cb9b8b0f6020fb90acf4fa12a00753a3120058 IMPALA-5852: improve
> > MINIMUM_RESERVATION_UNAVAILABLE error
> > 99fe9b3fd602180d63cbfe73ac2c9171c31ae455 IMPALA-5838: Improve errors on
> > AC buffer mem rejection
> >
> > See: https://github.com/timarmstrong/incubator-impala/
> > commits/release-2.10.0
> >
> >
> > On Wed, Aug 30, 2017 at 2:24 PM, Jim Apple  wrote:
> >
> >> I ran some release tests following the instructions
> >> https://cwiki.apache.org/confluence/display/IMPALA/How+to+
> >> Release#HowtoRelease-HowtoVoteonaReleaseCandidate
> >> and https://cwiki.apache.org/confluence/display/IMPALA/How+to+
> >> load+and+run+Impala+tests.
> >> Everything passed.
> >>
> >> I would +1, but I notice downthread that there is going to be an rc2,
> >> so: +0 for now.
> >>
> >> On Sun, Aug 27, 2017 at 10:32 PM, Bharath Vissapragada
> >>  wrote:
> >> > This is a vote to release Impala 2.10.0.
> >> >
> >> > - The artefacts for testing can be downloaded from <
> >> > https://dist.apache.org/repos/dist/dev/incubator/impala/2.10.0/RC1/>.
> >> >
> >> > - The git tag for this release candidate is 2.10.0-rc1 and tree hash
> is
> >> > visible at
> >> > <
> >> > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.g
> >> it;a=tree;hb=2a7c8b9011905bfeb21b0610f0739f9df9daacef
> >> >>
> >> >
> >> > Please vote +1 or -1. -1 votes should be accompanied by an explanation
> >> of
> >> > the reason. Only PPMC members and mentors have binding votes, but
> other
> >> > community members are encouraged to cast non-binding votes. This vote
> >> will
> >> > pass if there are 3 binding +1 votes and more binding +1 votes than -1
> >> > votes.
> >> >
> >> > This wiki page describes how to check the release before you vote:
> >> > *https://cwiki.apache.org/confluence/display/IMPALA/How+to+
> >> Release#HowtoRelease-HowtoVoteonaReleaseCandidate
> >> >  >> Release#HowtoRelease-HowtoVoteonaReleaseCandidate>*
> >> >
> >> > The vote will be open until the end of Wednesday, August 30, Pacific
> >> time
> >> > zone (UTC-08:00).
> >> > Once the vote passes the Impala PPMC vote, it still must pass the
> >> incubator
> >> > PMC vote before a release is made.
> >>
> >
> >
>

Re: [VOTE] 2.10.0 release candidate 1 (RC1)

2017-08-30 Thread Tim Armstrong

Tests passed, I've pushed to release-2.10.0.

- Tim

On Wed, Aug 30, 2017 at 2:59 PM, Tim Armstrong 
wrote:

> There were a few other fixes for critical issues and usability bugs that
> seemed worth including. I've put together a branch here that I will run
> tests on. I was going to include these commits:
>
> 23d79462da5d0108709e8b1399c97606f4ebdf92 IMPALA-5855: reserve enough
> memory for preaggs
> a58394be7c7998a5dfea53d8a3dbf8beb3370a48 IMPALA-5850: Cast sender
> partition exprs under unions.
> 2912a0f9d9b32caf586b9383c7e027af3fe4c5c4 IMPALA-5857: avoid invalid free
> of hedged read metrics
> ebe8ddd451b3f14d3f778339978a76bcd14b2589 IMPALA-5830:
> SET_DENY_RESERVATION_PROBABILITY test
> 1faf89f047e7d78c3a1f3b518269a3ae21a4ddea IMPALA-5840: Don't write
> page-level statistics in Parquet files.
> 73cb9b8b0f6020fb90acf4fa12a00753a3120058 IMPALA-5852: improve
> MINIMUM_RESERVATION_UNAVAILABLE error
> 99fe9b3fd602180d63cbfe73ac2c9171c31ae455 IMPALA-5838: Improve errors on
> AC buffer mem rejection
>
> See: https://github.com/timarmstrong/incubator-impala/
> commits/release-2.10.0
>
>
> On Wed, Aug 30, 2017 at 2:24 PM, Jim Apple  wrote:
>
>> I ran some release tests following the instructions
>> https://cwiki.apache.org/confluence/display/IMPALA/How+to+
>> Release#HowtoRelease-HowtoVoteonaReleaseCandidate
>> and https://cwiki.apache.org/confluence/display/IMPALA/How+to+
>> load+and+run+Impala+tests.
>> Everything passed.
>>
>> I would +1, but I notice downthread that there is going to be an rc2,
>> so: +0 for now.
>>
>> On Sun, Aug 27, 2017 at 10:32 PM, Bharath Vissapragada
>>  wrote:
>> > This is a vote to release Impala 2.10.0.
>> >
>> > - The artefacts for testing can be downloaded from <
>> > https://dist.apache.org/repos/dist/dev/incubator/impala/2.10.0/RC1/>.
>> >
>> > - The git tag for this release candidate is 2.10.0-rc1 and tree hash is
>> > visible at
>> > <
>> > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.g
>> it;a=tree;hb=2a7c8b9011905bfeb21b0610f0739f9df9daacef
>> >>
>> >
>> > Please vote +1 or -1. -1 votes should be accompanied by an explanation
>> of
>> > the reason. Only PPMC members and mentors have binding votes, but other
>> > community members are encouraged to cast non-binding votes. This vote
>> will
>> > pass if there are 3 binding +1 votes and more binding +1 votes than -1
>> > votes.
>> >
>> > This wiki page describes how to check the release before you vote:
>> > *https://cwiki.apache.org/confluence/display/IMPALA/How+to+
>> Release#HowtoRelease-HowtoVoteonaReleaseCandidate
>> > > Release#HowtoRelease-HowtoVoteonaReleaseCandidate>*
>> >
>> > The vote will be open until the end of Wednesday, August 30, Pacific
>> time
>> > zone (UTC-08:00).
>> > Once the vote passes the Impala PPMC vote, it still must pass the
>> incubator
>> > PMC vote before a release is made.
>>
>
>

Re:Re: Impala Show Tables

2017-08-30 Thread sky

Hi Dimitris,
What you mean is to query the database that stores the impala metadata ? 
hive(mysql,pg,derby and so on) ?








At 2017-08-30 16:41:40, "Dimitris Tsirogiannis"  
wrote:
>Hi sky,
>
>You could use HiveServer2 API (
>https://github.com/apache/hive/blob/master/service-rpc/if/TCLIService.thrift)
>to list the tables (see GetTables). Depending on your preference on
>programming language, a number of clients exist that use this API (e.g.
>https://github.com/cloudera/impyla).
>
>Dimitris
>
>On Wed, Aug 30, 2017 at 12:03 AM, sky  wrote:
>
>> Hi all,
>> In addition to "show tables" command, is there any other ways to show
>> all the tables in impala ?
>> I need a way to handle collection of all tables through SQL, but "show
>> tables"  can not be combined with SQL.

Re: jenkins.impala.io pre-existing workspace

2017-08-30 Thread Philip Zeyliger

On Wed, Aug 30, 2017 at 4:45 PM, Jim Apple  wrote:

> The workspace cleanup isn't working - see the last bit of any recent
> ub1604 job: https://jenkins.impala.io/view/Utility/job/ubuntu-16.04-
> from-scratch/206/console
>
> 03:56:40.920 [WS-CLEANUP] Deleting project workspace...Cannot delete
> workspace :remote file operation failed: /home/ubuntu at
> hudson.remoting.Channel@4384d5b9:ubuntu-16.04 (i-032d527b9c801df4c):
> java.io.IOException: Unable to delete '/home/ubuntu'. Tried 3 times
> (of a maximum of 3) waiting 0.1 sec between attempts.
> 03:56:48.161 ERROR: Step ‘Delete workspace when build is done’ failed:
> Cannot delete workspace: remote file operation failed: /home/ubuntu at
> hudson.remoting.Channel@4384d5b9:ubuntu-16.04 (i-032d527b9c801df4c):
> java.io.IOException: Unable to delete '/home/ubuntu'. Tried 3 times
> (of a maximum of 3) waiting 0.1 sec between attempts.
>
> The workspace is $HOME, so you can't just delete it without being root.
>
> This could be changed to
>
> 1. A post-build script to "rm -rf ~/*". This doesn't reset everything,
> though - the job makes changes to other parts of the filesystem.
>
> 2. A post-build script to "sudo shutdown -h now" to make sure ec2
> instances are not re-used. I'm not sure how Jenkins would feel about
> this. :-)
>
> 3. A post-build script to move $HOME to some archived location on the
> disk, to preserve debuggability.
>
> 4. A bash trap in the script to do one of the above.
>
> 5. Run the whole thing in a docker in the build machine, then delete
> the container when the script is done. Or don't, if there's enough
> disk space to not worry about that.
>
> 6. Do all of the work in a workspace inside $HOME. This would require
> some changes to bootstrap_development.sh.
>
> #5 is the most hermetic, I'd guess.
>

I like #6 in the short term: I don't think anything is too bound to $HOME
except a few "~/" in that script which are easily approached. If you do #5,
you have to teach Jenkins about how to save the build output logs from
inside of Docker, which is, I think, more work.

-- Philip


>
> On Thu, Aug 24, 2017 at 8:29 AM, Michael Brown  wrote:
> > Looks like someone has done this.
> >
> > On Wed, Aug 23, 2017 at 8:16 PM, Alexander Behm 
> > wrote:
> >
> >> Yes, let's please add the post-build action for sanity and consistency
> with
> >> our other jobs.
> >>
> >> On Wed, Aug 23, 2017 at 7:42 PM, Tim Armstrong  >
> >> wrote:
> >>
> >> > Maybe the workspace just got left in a weird state - I think in most
> >> cases
> >> > "git init" followed by checking out a branch and doing a clean would
> >> work.
> >> >
> >> > Should we add the delete workspace post-build action?
> >> >
> >> > On Wed, Aug 23, 2017 at 5:32 PM, Michael Brown 
> >> wrote:
> >> >
> >> > > Not a known issue. I noticed ubuntu-16.04-from-scratch is not set to
> >> > clean
> >> > > up its workspace, and its config has not been touched since Aug 11.
> It
> >> > > seems strange we only saw this now
> >> > >
> >> > > On Wed, Aug 23, 2017 at 5:25 PM, Tim Armstrong <
> >> tarmstr...@cloudera.com>
> >> > > wrote:
> >> > >
> >> > > > Is this a known problem? My job failed because the Impala repo
> >> already
> >> > > > existed on the machine:
> >> > > >
> >> > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/164/
> >> > > >
> >> > > > *23:00:24* + /usr/bin/git init /home/ubuntu/Impala*23:00:24*
> >> > > > Reinitialized existing Git repository in /home/ubuntu/Impala/.git/
> >> > > > 
> >> > > > *23:02:18* + for ITER in '$(seq 1 10)'*23:02:18* + echo 'ATTEMPT:
> >> > > > 1'*23:02:18* ATTEMPT: 1*23:02:18* + /usr/bin/git checkout
> >> > > > FETCH_HEAD*23:02:18* + cat
> >> > > > /home/ubuntu/Impala/tmp.3tYBn0GUga*23:02:18* 23:02:18.712300
> >> git.c:344
> >> > > >   trace: built-in: git 'checkout'
> 'FETCH_HEAD'*23:02:18*
> >> > > > error: The following untracked working tree files would be
> >> overwritten
> >> > > > by checkout:*23:02:18*  .clang-format*23:02:18*
> >> > > >  .clang-tidy*23:02:18*
> >> > > > .gitignore*23:02:18*CMakeLists.txt*23:02:18*
> >> > > > DISCLAIMER*23:02:18*
> >> > > > EXPORT_CONTROL.md*23:02:18* LICENSE.txt*23:02:18*
> >> > > >  LOGS.md*23:02:18*
> >> > > > NOTICE.txt*23:02:18*README.md*23:02:18*
> >> > > >  be/.gitignore*23:02:18*
> >> > > > be/.impala.doxy*23:02:18*
>  be/CMakeLists.txt*23:02:18*
> >> > > > be/src/benchmarks/CMakeLists.txt*23:02:18*
> >> > > > be/src/benchmarks/atod-benchmark.cc*23:02:18*
> >> > > > be/src/benchmarks/atof-benchmark.cc*23:02:18*
> >> > > > be/src/benchmarks/atoi-benchmark.cc*23:02:18*
> >> > > > be/src/benchmarks/bit-packing-benchmark.cc*23:02:18*
> >> > > > be/src/benchmarks/bitmap-benchmark.cc
> >> > > > ...
> >> > > >
> >> > >
> >> >
> >>
>

Re: jenkins.impala.io pre-existing workspace

2017-08-30 Thread Jim Apple

The workspace cleanup isn't working - see the last bit of any recent
ub1604 job: 
https://jenkins.impala.io/view/Utility/job/ubuntu-16.04-from-scratch/206/console

03:56:40.920 [WS-CLEANUP] Deleting project workspace...Cannot delete
workspace :remote file operation failed: /home/ubuntu at
hudson.remoting.Channel@4384d5b9:ubuntu-16.04 (i-032d527b9c801df4c):
java.io.IOException: Unable to delete '/home/ubuntu'. Tried 3 times
(of a maximum of 3) waiting 0.1 sec between attempts.
03:56:48.161 ERROR: Step ‘Delete workspace when build is done’ failed:
Cannot delete workspace: remote file operation failed: /home/ubuntu at
hudson.remoting.Channel@4384d5b9:ubuntu-16.04 (i-032d527b9c801df4c):
java.io.IOException: Unable to delete '/home/ubuntu'. Tried 3 times
(of a maximum of 3) waiting 0.1 sec between attempts.

The workspace is $HOME, so you can't just delete it without being root.

This could be changed to

1. A post-build script to "rm -rf ~/*". This doesn't reset everything,
though - the job makes changes to other parts of the filesystem.

2. A post-build script to "sudo shutdown -h now" to make sure ec2
instances are not re-used. I'm not sure how Jenkins would feel about
this. :-)

3. A post-build script to move $HOME to some archived location on the
disk, to preserve debuggability.

4. A bash trap in the script to do one of the above.

5. Run the whole thing in a docker in the build machine, then delete
the container when the script is done. Or don't, if there's enough
disk space to not worry about that.

6. Do all of the work in a workspace inside $HOME. This would require
some changes to bootstrap_development.sh.

#5 is the most hermetic, I'd guess.

On Thu, Aug 24, 2017 at 8:29 AM, Michael Brown  wrote:
> Looks like someone has done this.
>
> On Wed, Aug 23, 2017 at 8:16 PM, Alexander Behm 
> wrote:
>
>> Yes, let's please add the post-build action for sanity and consistency with
>> our other jobs.
>>
>> On Wed, Aug 23, 2017 at 7:42 PM, Tim Armstrong 
>> wrote:
>>
>> > Maybe the workspace just got left in a weird state - I think in most
>> cases
>> > "git init" followed by checking out a branch and doing a clean would
>> work.
>> >
>> > Should we add the delete workspace post-build action?
>> >
>> > On Wed, Aug 23, 2017 at 5:32 PM, Michael Brown 
>> wrote:
>> >
>> > > Not a known issue. I noticed ubuntu-16.04-from-scratch is not set to
>> > clean
>> > > up its workspace, and its config has not been touched since Aug 11. It
>> > > seems strange we only saw this now
>> > >
>> > > On Wed, Aug 23, 2017 at 5:25 PM, Tim Armstrong <
>> tarmstr...@cloudera.com>
>> > > wrote:
>> > >
>> > > > Is this a known problem? My job failed because the Impala repo
>> already
>> > > > existed on the machine:
>> > > >
>> > > > https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/164/
>> > > >
>> > > > *23:00:24* + /usr/bin/git init /home/ubuntu/Impala*23:00:24*
>> > > > Reinitialized existing Git repository in /home/ubuntu/Impala/.git/
>> > > > 
>> > > > *23:02:18* + for ITER in '$(seq 1 10)'*23:02:18* + echo 'ATTEMPT:
>> > > > 1'*23:02:18* ATTEMPT: 1*23:02:18* + /usr/bin/git checkout
>> > > > FETCH_HEAD*23:02:18* + cat
>> > > > /home/ubuntu/Impala/tmp.3tYBn0GUga*23:02:18* 23:02:18.712300
>> git.c:344
>> > > >   trace: built-in: git 'checkout' 'FETCH_HEAD'*23:02:18*
>> > > > error: The following untracked working tree files would be
>> overwritten
>> > > > by checkout:*23:02:18*  .clang-format*23:02:18*
>> > > >  .clang-tidy*23:02:18*
>> > > > .gitignore*23:02:18*CMakeLists.txt*23:02:18*
>> > > > DISCLAIMER*23:02:18*
>> > > > EXPORT_CONTROL.md*23:02:18* LICENSE.txt*23:02:18*
>> > > >  LOGS.md*23:02:18*
>> > > > NOTICE.txt*23:02:18*README.md*23:02:18*
>> > > >  be/.gitignore*23:02:18*
>> > > > be/.impala.doxy*23:02:18*   be/CMakeLists.txt*23:02:18*
>> > > > be/src/benchmarks/CMakeLists.txt*23:02:18*
>> > > > be/src/benchmarks/atod-benchmark.cc*23:02:18*
>> > > > be/src/benchmarks/atof-benchmark.cc*23:02:18*
>> > > > be/src/benchmarks/atoi-benchmark.cc*23:02:18*
>> > > > be/src/benchmarks/bit-packing-benchmark.cc*23:02:18*
>> > > > be/src/benchmarks/bitmap-benchmark.cc
>> > > > ...
>> > > >
>> > >
>> >
>>

Re: Question about the multi-thread scan node model

2017-08-30 Thread Tim Armstrong

Hi,
  The new scanner model is part of the multithreading work to support
running multiple instances of each fragment on each Impala daemon. The idea
there is that parallelisation is done at the fragment level so that all
execution including aggregations, sorts, joins is parallelised - not just
scans. This is enabled by setting mt_dop > 0. Currently it doesn't work for
plans including joins and HDFS inserts.

We find that a lot of queries are compute bound, particularly by
aggregations and joins. In those cases we get big speedups from the newer
multithreading model. E.g. "compute stats" is a lot faster.

On Wed, Aug 30, 2017 at 3:50 PM, 黄权隆  wrote:

> Hi all,
>
>
> I’m working on applying our orc-support patch into the latest code bases (
> IMPALA-5717 ). Since
> our
> patch is based on cdh-5.7.3-release which was released one year ago,
> there’re lots of work to merge it.
>
>
> One of the biggest changes from cdh-5.7.3-release I notice is the new scan
> node & scanner model introduced in IMPALA-3902
> . I think it’s inspired
> by the investigating task in IMPALA-2849
> , but I cannot find any
> performance report in this issue. Could you share some report about this
> multi-thread refactor?
>
>
> I’m wondering how much this can improve the performance, since the old
> single thread scan node & multi-thread scanners model has supplied
> concurrent IO for reading, and most of the queries in OLAP are IO bound.
>
>
> Thanks,
>
> Quanlong
>

Question about the multi-thread scan node model

2017-08-30 Thread 黄权隆

Hi all,


I’m working on applying our orc-support patch into the latest code bases (
IMPALA-5717 ). Since our
patch is based on cdh-5.7.3-release which was released one year ago,
there’re lots of work to merge it.


One of the biggest changes from cdh-5.7.3-release I notice is the new scan
node & scanner model introduced in IMPALA-3902
. I think it’s inspired
by the investigating task in IMPALA-2849
, but I cannot find any
performance report in this issue. Could you share some report about this
multi-thread refactor?


I’m wondering how much this can improve the performance, since the old
single thread scan node & multi-thread scanners model has supplied
concurrent IO for reading, and most of the queries in OLAP are IO bound.


Thanks,

Quanlong

Re: [VOTE] 2.10.0 release candidate 1 (RC1)

2017-08-30 Thread Tim Armstrong

There were a few other fixes for critical issues and usability bugs that
seemed worth including. I've put together a branch here that I will run
tests on. I was going to include these commits:

23d79462da5d0108709e8b1399c97606f4ebdf92 IMPALA-5855: reserve enough memory
for preaggs
a58394be7c7998a5dfea53d8a3dbf8beb3370a48 IMPALA-5850: Cast sender partition
exprs under unions.
2912a0f9d9b32caf586b9383c7e027af3fe4c5c4 IMPALA-5857: avoid invalid free of
hedged read metrics
ebe8ddd451b3f14d3f778339978a76bcd14b2589 IMPALA-5830:
SET_DENY_RESERVATION_PROBABILITY test
1faf89f047e7d78c3a1f3b518269a3ae21a4ddea IMPALA-5840: Don't write
page-level statistics in Parquet files.
73cb9b8b0f6020fb90acf4fa12a00753a3120058 IMPALA-5852: improve
MINIMUM_RESERVATION_UNAVAILABLE error
99fe9b3fd602180d63cbfe73ac2c9171c31ae455 IMPALA-5838: Improve errors on AC
buffer mem rejection

See: https://github.com/timarmstrong/incubator-impala/commits/release-2.10.0

On Wed, Aug 30, 2017 at 2:24 PM, Jim Apple  wrote:

> I ran some release tests following the instructions
> https://cwiki.apache.org/confluence/display/IMPALA/How+
> to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
> and https://cwiki.apache.org/confluence/display/IMPALA/How+
> to+load+and+run+Impala+tests.
> Everything passed.
>
> I would +1, but I notice downthread that there is going to be an rc2,
> so: +0 for now.
>
> On Sun, Aug 27, 2017 at 10:32 PM, Bharath Vissapragada
>  wrote:
> > This is a vote to release Impala 2.10.0.
> >
> > - The artefacts for testing can be downloaded from <
> > https://dist.apache.org/repos/dist/dev/incubator/impala/2.10.0/RC1/>.
> >
> > - The git tag for this release candidate is 2.10.0-rc1 and tree hash is
> > visible at
> > <
> > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.
> git;a=tree;hb=2a7c8b9011905bfeb21b0610f0739f9df9daacef
> >>
> >
> > Please vote +1 or -1. -1 votes should be accompanied by an explanation of
> > the reason. Only PPMC members and mentors have binding votes, but other
> > community members are encouraged to cast non-binding votes. This vote
> will
> > pass if there are 3 binding +1 votes and more binding +1 votes than -1
> > votes.
> >
> > This wiki page describes how to check the release before you vote:
> > *https://cwiki.apache.org/confluence/display/IMPALA/How+
> to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
> >  to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate>*
> >
> > The vote will be open until the end of Wednesday, August 30, Pacific time
> > zone (UTC-08:00).
> > Once the vote passes the Impala PPMC vote, it still must pass the
> incubator
> > PMC vote before a release is made.
>

Re: [VOTE] 2.10.0 release candidate 1 (RC1)

2017-08-30 Thread Jim Apple

I ran some release tests following the instructions
https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
and 
https://cwiki.apache.org/confluence/display/IMPALA/How+to+load+and+run+Impala+tests.
Everything passed.

I would +1, but I notice downthread that there is going to be an rc2,
so: +0 for now.

On Sun, Aug 27, 2017 at 10:32 PM, Bharath Vissapragada
 wrote:
> This is a vote to release Impala 2.10.0.
>
> - The artefacts for testing can be downloaded from <
> https://dist.apache.org/repos/dist/dev/incubator/impala/2.10.0/RC1/>.
>
> - The git tag for this release candidate is 2.10.0-rc1 and tree hash is
> visible at
> <
> https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tree;hb=2a7c8b9011905bfeb21b0610f0739f9df9daacef
>>
>
> Please vote +1 or -1. -1 votes should be accompanied by an explanation of
> the reason. Only PPMC members and mentors have binding votes, but other
> community members are encouraged to cast non-binding votes. This vote will
> pass if there are 3 binding +1 votes and more binding +1 votes than -1
> votes.
>
> This wiki page describes how to check the release before you vote:
> *https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release#HowtoRelease-HowtoVoteonaReleaseCandidate
> *
>
> The vote will be open until the end of Wednesday, August 30, Pacific time
> zone (UTC-08:00).
> Once the vote passes the Impala PPMC vote, it still must pass the incubator
> PMC vote before a release is made.

RE: IGFS (Ignite FS) support

2017-08-30 Thread Andrey Kuznetsov

Hi,
Looks like,
Has been done:
https://issues.apache.org/jira/browse/IMPALA-5862

Best regards,
ANDREY KUZNETSOV
Software Engineering Team Leader, Assessment Global Discipline Head (Java)
 
Office: +7 482 263 00 70 x 42766   Cell: +7 920 154 05 72   Email: 
andrey_kuznet...@epam.com
Tver, Russia   epam.com
 
CONFIDENTIALITY CAUTION AND DISCLAIMER
This message is intended only for the use of the individual(s) or entity(ies) 
to which it is addressed and contains information that is legally privileged 
and confidential. If you are not the intended recipient, or the person 
responsible for delivering the message to the intended recipient, you are 
hereby notified that any dissemination, distribution or copying of this 
communication is strictly prohibited. All unintended recipients are obliged to 
delete this message and destroy any printed copies.

-Original Message-
From: Jim Apple [mailto:jbap...@cloudera.com] 
Sent: Wednesday, August 30, 2017 3:42 PM
To: dev@impala 
Cc: Special SBER-BPOC Team 
Subject: Re: IGFS (Ignite FS) support

Doesn't look like there is a JIRA yet:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20IMPALA%20AND%20text%20~%20%22ignite%22

https://issues.apache.org/jira/issues/?jql=project%20%3D%20IMPALA%20AND%20text%20~%20%22igfs%22

You might want to file one as a single point of coordination for people 
interested in this work.

On Wed, Aug 30, 2017 at 2:59 AM, Andrey Kuznetsov  
wrote:
> Hi team,
> My team has faced with follow issue:
> Impala doesn't work with IGFS because IgniteHadoopFileSystem is not validated 
>  as WritableFilesystem, because  IgniteHadoopFileSystem  does not extend 
> org.apache.hadoop.hdfs.DistributedFileSystem  and not listed in others 
> acceptable file systems list:
>
>* Returns true iff the given location is on a filesystem that Impala can 
> write to.
>
>*/
>
>   public static boolean isImpalaWritableFilesystem(String location)
>
>   throws IOException {
>
> Path path = new Path(location);
>
> return (FileSystemUtil.isDistributedFileSystem(path) ||
>
> FileSystemUtil.isLocalFileSystem(path) ||
>
> FileSystemUtil.isS3AFileSystem(path) ||
>
> FileSystemUtil.isADLFileSystem(path));
>
>   }
>
>
> Does anybody know if we plan to support IGFS?
>
> Best regards,
> ANDREY KUZNETSOV

Re: IGFS (Ignite FS) support

2017-08-30 Thread Jim Apple

Doesn't look like there is a JIRA yet:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20IMPALA%20AND%20text%20~%20%22ignite%22

https://issues.apache.org/jira/issues/?jql=project%20%3D%20IMPALA%20AND%20text%20~%20%22igfs%22

You might want to file one as a single point of coordination for
people interested in this work.

On Wed, Aug 30, 2017 at 2:59 AM, Andrey Kuznetsov
 wrote:
> Hi team,
> My team has faced with follow issue:
> Impala doesn't work with IGFS because IgniteHadoopFileSystem is not validated 
>  as WritableFilesystem, because  IgniteHadoopFileSystem  does not extend 
> org.apache.hadoop.hdfs.DistributedFileSystem  and not listed in others 
> acceptable file systems list:
>
>* Returns true iff the given location is on a filesystem that Impala can 
> write to.
>
>*/
>
>   public static boolean isImpalaWritableFilesystem(String location)
>
>   throws IOException {
>
> Path path = new Path(location);
>
> return (FileSystemUtil.isDistributedFileSystem(path) ||
>
> FileSystemUtil.isLocalFileSystem(path) ||
>
> FileSystemUtil.isS3AFileSystem(path) ||
>
> FileSystemUtil.isADLFileSystem(path));
>
>   }
>
>
> Does anybody know if we plan to support IGFS?
>
> Best regards,
> ANDREY KUZNETSOV

IGFS (Ignite FS) support

2017-08-30 Thread Andrey Kuznetsov

Hi team,
My team has faced with follow issue:
Impala doesn't work with IGFS because IgniteHadoopFileSystem is not validated  
as WritableFilesystem, because  IgniteHadoopFileSystem  does not extend 
org.apache.hadoop.hdfs.DistributedFileSystem  and not listed in others 
acceptable file systems list:

   * Returns true iff the given location is on a filesystem that Impala can 
write to.

   */

  public static boolean isImpalaWritableFilesystem(String location)

  throws IOException {

Path path = new Path(location);

return (FileSystemUtil.isDistributedFileSystem(path) ||

FileSystemUtil.isLocalFileSystem(path) ||

FileSystemUtil.isS3AFileSystem(path) ||

FileSystemUtil.isADLFileSystem(path));

  }


Does anybody know if we plan to support IGFS?

Best regards,
ANDREY KUZNETSOV

Re: Impala Show Tables

2017-08-30 Thread Dimitris Tsirogiannis

Hi sky,

You could use HiveServer2 API (
https://github.com/apache/hive/blob/master/service-rpc/if/TCLIService.thrift)
to list the tables (see GetTables). Depending on your preference on
programming language, a number of clients exist that use this API (e.g.
https://github.com/cloudera/impyla).

Dimitris

On Wed, Aug 30, 2017 at 12:03 AM, sky  wrote:

> Hi all,
> In addition to "show tables" command, is there any other ways to show
> all the tables in impala ?
> I need a way to handle collection of all tables through SQL, but "show
> tables"  can not be combined with SQL.

Impala Show Tables

2017-08-30 Thread sky

Hi all,
In addition to "show tables" command, is there any other ways to show all 
the tables in impala ? 
I need a way to handle collection of all tables through SQL, but "show 
tables"  can not be combined with SQL.

Re: [VOTE] 2.10.0 release candidate 1 (RC1)

Re: [VOTE] 2.10.0 release candidate 1 (RC1)

Re:Re: Impala Show Tables

Re: jenkins.impala.io pre-existing workspace

Re: jenkins.impala.io pre-existing workspace

Re: Question about the multi-thread scan node model

Question about the multi-thread scan node model

Re: [VOTE] 2.10.0 release candidate 1 (RC1)

Re: [VOTE] 2.10.0 release candidate 1 (RC1)

RE: IGFS (Ignite FS) support

Re: IGFS (Ignite FS) support

IGFS (Ignite FS) support

Re: Impala Show Tables

Impala Show Tables

14 matches

Site Navigation

Mail list logo

Footer information