Re: Contributions to Cloudera Impala

[Jim Apple]

I have recently learned that contributors do not need to sign a CLA unless
they are invited to become "Committers", a special Apache designation that
you don't need to worry too much about right now:

http://www.apache.org/legal/resolved.html#are-contributors-required-to-sign-a-ccla

On Wed, Oct 26, 2016 at 3:37 AM, Nishidha Panpaliya <nishi...@us.ibm.com>
wrote:

> Hi Jim,
>
> I'm really sorry for the delay in starting the new thread for Impala
> branching.
>
> Actually this time, before we port the current code of Impala, we have
> also thought of porting Impala's native toolchain as we'd really difficult
> time building it manually in past. We could build most of the tools from
> it. But porting "kudu" has become our major hurdle as it is equally big and
> challenging as that of Impala. Though we've been able to fix most of the
> stuff with our prior Impala experience, but everyday we see a new set of
> problems with kudu as we are progressing.
>
> I think once we reach near completion of porting Impala's toolchain, I
> will start the new thread for Impala branching as it is evolving every day.
> I hope we get to that state sooner.
>
> Even on contributions legal approval front, we've got the required
> information and we are also in the process of signing ICLA. Our filled
> forms have reached to IBM legal team. Hope to hear back from them soon.
>
> I thank you for your concern and once again apologize for making you
> follow up on this. I will definitely get back to you soon.
>
>
> Thanks & Regards,
> Nishidha
>
> [image: Inactive hide details for Jim Apple ---10/26/2016 01:19:40
> AM---Nishidha, I still haven't heard about the branching. Is there a]Jim
> Apple ---10/26/2016 01:19:40 AM---Nishidha, I still haven't heard about the
> branching. Is there anything in your way I can help move a
>
> From: Jim Apple <jbap...@cloudera.com>
> To: Nishidha Panpaliya/Austin/Contr/IBM@IBMUS
> Cc: "dev@impala" <dev@impala.incubator.apache.org>, Manish
> Patil/Austin/Contr/IBM@IBMUS, Silvius Rus <s...@cloudera.com>, Sudarshan
> Jagadale/Austin/Contr/IBM@IBMUS, Valencia Serrao/Austin/Contr/IBM@IBMUS
> Date: 10/26/2016 01:19 AM
> Subject: Re: Contributions to Cloudera Impala
> ----------
>
>
>
> Nishidha, I still haven't heard about the branching. Is there anything
> in your way I can help move aside?
>
> On Mon, Oct 3, 2016 at 8:26 AM, Jim Apple <jbap...@cloudera.com> wrote:
> >> Yes, I'll start a new discussion about branching. Could you please
> share the
> >> complete address of dev@? Is it different than
> >> "dev@impala.incubator.apache.org"?
> >
> > That is the address.
>
>
>
>
>

Unable to start Hive with testdata.bin/run-all.sh

[Jim Apple]

I am seeing the error below when trying to get my development
environment up and running. Has anyone else seen and worked around
this before?

2016-10-26 10:27:16,719 ERROR Datastore.Schema
(Log4JLogger.java:error(125)) - Failed initialising database.
Unable to open a test connection to the given database. JDBC url =
jdbc:derby:;databaseName=metastore_db;create=true, username = APP.
Terminating connection pool (set lazyInit to true if you expect to
start your database after your app). Original Exception: --
java.sql.SQLException: Failed to start database 'metastore_db' with
class loader sun.misc.Launcher$AppClassLoader@3ad6a0e0, see the next
exception for details.
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
Source)
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown
Source)
at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown
Source)
at org.apache.derby.impl.jdbc.EmbedConnection.(Unknown Source)
[MANY MORE LINES]
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: ERROR XJ040: Failed to start database 'metastore_db' with
class loader sun.misc.Launcher$AppClassLoader@3ad6a0e0, see the next
exception for details.
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
at 
org.apache.derby.impl.jdbc.SQLExceptionFactory.wrapArgsForTransportAcrossDRDA(Unknown
Source)
... 84 more
Caused by: ERROR XSDB6: Another instance of Derby may have already
booted the database /home/jbapple/Impala/metastore_db.
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
at org.apache.derby.iapi.error.StandardException.newException(Unknown
Source)
[MANY MORE LINES]

Re: distcc now available

[Jim Apple]

Thanks to Casey Ching, who did almost all of the work a few months ago!

On Wed, Oct 26, 2016 at 9:29 AM, Tim Armstrong <tarmstr...@cloudera.com> wrote:
> Thanks Jim!
>
> I've been using distcc with Impala for a while now and it makes a huge
> difference. I'm no longer scared to do a full build from scratch.
>
> On Tue, Oct 25, 2016 at 9:24 AM, Jim Apple <jbap...@cloudera.com> wrote:
>
>> Impala now supports distcc to speed up your builds. The instructions
>> are in bin/distcc/README.md.
>>

Re: Contributions to Cloudera Impala

[Jim Apple]

Nishidha, I still haven't heard about the branching. Is there anything
in your way I can help move aside?

On Mon, Oct 3, 2016 at 8:26 AM, Jim Apple <jbap...@cloudera.com> wrote:
>> Yes, I'll start a new discussion about branching. Could you please share the
>> complete address of dev@? Is it different than
>> "dev@impala.incubator.apache.org"?
>
> That is the address.

distcc now available

[Jim Apple]

Impala now supports distcc to speed up your builds. The instructions
are in bin/distcc/README.md.

Re: Cannot push new patch set to Gerrit

[Jim Apple]

OK, pushing (even without rebase) is now working for me.

On Mon, Oct 24, 2016 at 2:14 PM, Tim Armstrong <tarmstr...@cloudera.com> wrote:
> Yeah, I think I probably clobbered whatever commit was there before with my
> force push.
>
> On Mon, Oct 24, 2016 at 2:13 PM, Henry Robinson <he...@cloudera.com> wrote:
>
>> Ah, that might be something magit-push did in Emacs - I was trying to push
>> from Emacs and it clearly didn't do the right thing. I think we can delete
>> that new branch.
>>
>> On 24 October 2016 at 14:06, Jim Apple <jbap...@cloudera.com> wrote:
>>
>>> Doing a git fetch of the gerrit server, I see
>>>
>>>  * [new branch]  refs/for/master -> gerrit-asf/refs/for/master
>>>
>>> I'm not sure what's going on with that - the "master" branch is pushed
>>> to by "refs/for/master" or "refs/drafts/master", so this could be a
>>> typo of some sort.
>>>
>>> The commit 119bff3ac17f847f1a4137ea56707c72cd1119be is on
>>> refs/for/master without a "Reviewed-on" line. Tim, could you take a
>>> look? It's
>>>
>>> IMPALA-1430,IMPALA-4108: codegen all builtin aggregate functions
>>>
>>> On Mon, Oct 24, 2016 at 1:57 PM, Jim Apple <jbap...@cloudera.com> wrote:
>>> > What do you think we should force-push?
>>> >
>>> > What did we do to cause it, and how can we avoid it in the future?
>>> >
>>> > On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson <he...@cloudera.com>
>>> wrote:
>>> >> I also saw the same thing. The way I got around it was to try pushing
>>> to
>>> >> /drafts/ and then publishing it, which worked.
>>> >>
>>> >> We might need to force-push to gerrit/master to clean this up.
>>> >>
>>> >> On 24 October 2016 at 12:37, Lars Volker <l...@cloudera.com> wrote:
>>> >>
>>> >>> I tried to push another patch set to Gerrit and got the error below.
>>> This
>>> >>> worked about an hour ago and now stopped working. The local parent
>>> commit
>>> >>> is the same one as remote, I didn't rebase either. Has anyone seen
>>> this
>>> >>> before? I suspect I can rebase but then the review gets much more
>>> tedious
>>> >>> to follow.
>>> >>>
>>> >>> Thanks, Lars
>>> >>>
>>> >>> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
>>> >>> To ssh://gerrit.cloudera.org:29418/Impala-ASF
>>> >>>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
>>> >>> error: failed to push some refs to 'ssh://
>>> >>> l...@gerrit.cloudera.org:29418/Impala-ASF'
>>> >>> hint: Updates were rejected because a pushed branch tip is behind its
>>> >>> remote
>>> >>> hint: counterpart. Check out this branch and integrate the remote
>>> changes
>>> >>> hint: (e.g. 'git pull ...') before pushing again.
>>> >>> hint: See the 'Note about fast-forwards' in 'git push --help' for
>>> details.
>>> >>> ✗ ~/i2(i2521) ✗$
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Henry Robinson
>>> >> Software Engineer
>>> >> Cloudera
>>> >> 415-994-6679
>>>
>>
>>
>>
>> --
>> Henry Robinson
>> Software Engineer
>> Cloudera
>> 415-994-6679
>>

Re: Cannot push new patch set to Gerrit

[Jim Apple]

That's this patch, which isn't +2ed yet:

https://gerrit.cloudera.org/#/c/4655/

I checked, and that's the only patch in refs/for/master that's not in
master, though master has a few changes not in refs/for/master:

+commit 61fcb489745f3f0b3f1abbf9fbf666a29a6363de
+commit 0fbb5b7e71e55346c8b97ec143854dba0088f124
+commit ff6b450ad380ce840e18875a89d9cf98058277a3
+commit 51268c053ffe41dc1aa9f1b250878113d4225258
+commit 48085274fa8ae57453477db21dae0e53eae6b766
+commit e39f1676e1c9269fb6549e8c319adf1a7ea9d445
+commit d1d88aaccd03c461c4ce6224f765afe94b7c073b
+commit 8d7b01faea6362af675a2a335b462fad3e0caa03
+commit 8a49ceaae532163f17836b1050b639329424ee5c
+commit 041fa6d946e1cbe309593c4a5515bef88e06cdb4
+commit f8d48b8582b9e460c2e0e3dbb4881636f179ae73
+commit 9ef9512e5b58a75e075538b3b94ac551363609e5
+commit 2fa1633e409e9018c012df81f291a0d7566e

On Mon, Oct 24, 2016 at 2:06 PM, Jim Apple <jbap...@cloudera.com> wrote:
> Doing a git fetch of the gerrit server, I see
>
>  * [new branch]  refs/for/master -> gerrit-asf/refs/for/master
>
> I'm not sure what's going on with that - the "master" branch is pushed
> to by "refs/for/master" or "refs/drafts/master", so this could be a
> typo of some sort.
>
> The commit 119bff3ac17f847f1a4137ea56707c72cd1119be is on
> refs/for/master without a "Reviewed-on" line. Tim, could you take a
> look? It's
>
> IMPALA-1430,IMPALA-4108: codegen all builtin aggregate functions
>
> On Mon, Oct 24, 2016 at 1:57 PM, Jim Apple <jbap...@cloudera.com> wrote:
>> What do you think we should force-push?
>>
>> What did we do to cause it, and how can we avoid it in the future?
>>
>> On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson <he...@cloudera.com> wrote:
>>> I also saw the same thing. The way I got around it was to try pushing to
>>> /drafts/ and then publishing it, which worked.
>>>
>>> We might need to force-push to gerrit/master to clean this up.
>>>
>>> On 24 October 2016 at 12:37, Lars Volker <l...@cloudera.com> wrote:
>>>
>>>> I tried to push another patch set to Gerrit and got the error below. This
>>>> worked about an hour ago and now stopped working. The local parent commit
>>>> is the same one as remote, I didn't rebase either. Has anyone seen this
>>>> before? I suspect I can rebase but then the review gets much more tedious
>>>> to follow.
>>>>
>>>> Thanks, Lars
>>>>
>>>> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
>>>> To ssh://gerrit.cloudera.org:29418/Impala-ASF
>>>>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
>>>> error: failed to push some refs to 'ssh://
>>>> l...@gerrit.cloudera.org:29418/Impala-ASF'
>>>> hint: Updates were rejected because a pushed branch tip is behind its
>>>> remote
>>>> hint: counterpart. Check out this branch and integrate the remote changes
>>>> hint: (e.g. 'git pull ...') before pushing again.
>>>> hint: See the 'Note about fast-forwards' in 'git push --help' for details.
>>>> ✗ ~/i2(i2521) ✗$
>>>>
>>>
>>>
>>>
>>> --
>>> Henry Robinson
>>> Software Engineer
>>> Cloudera
>>> 415-994-6679

Re: Cannot push new patch set to Gerrit

[Jim Apple]

What do you think we should force-push?

What did we do to cause it, and how can we avoid it in the future?

On Mon, Oct 24, 2016 at 12:40 PM, Henry Robinson  wrote:
> I also saw the same thing. The way I got around it was to try pushing to
> /drafts/ and then publishing it, which worked.
>
> We might need to force-push to gerrit/master to clean this up.
>
> On 24 October 2016 at 12:37, Lars Volker  wrote:
>
>> I tried to push another patch set to Gerrit and got the error below. This
>> worked about an hour ago and now stopped working. The local parent commit
>> is the same one as remote, I didn't rebase either. Has anyone seen this
>> before? I suspect I can rebase but then the review gets much more tedious
>> to follow.
>>
>> Thanks, Lars
>>
>> ✔ ~/i2(i2521) ✗$ git push asf HEAD:refs/for/master
>> To ssh://gerrit.cloudera.org:29418/Impala-ASF
>>  ! [rejected]HEAD -> refs/for/master (non-fast-forward)
>> error: failed to push some refs to 'ssh://
>> l...@gerrit.cloudera.org:29418/Impala-ASF'
>> hint: Updates were rejected because a pushed branch tip is behind its
>> remote
>> hint: counterpart. Check out this branch and integrate the remote changes
>> hint: (e.g. 'git pull ...') before pushing again.
>> hint: See the 'Note about fast-forwards' in 'git push --help' for details.
>> ✗ ~/i2(i2521) ✗$
>>
>
>
>
> --
> Henry Robinson
> Software Engineer
> Cloudera
> 415-994-6679

Re: Bulk JIRA email, or how to find good newbie tasks

[Jim Apple]

Why do we need to fix the "ramp*" bugs BEFORE we label "newbie" bugs?

On Sat, Oct 22, 2016 at 1:12 PM, Marcel Kornacker <mar...@cloudera.com> wrote:
> Before you do that we need to sit down and reclassify what's currently
> labeled 'ramp-up', some of those tasks aren't really suitable as that
> (because they don't have an educational component).
>
> On Fri, Oct 21, 2016 at 10:53 AM, Jim Apple <jbap...@cloudera.com> wrote:
>> SGTM. I will pick some of the PMC members and ask them to individually
>> classify all of the ramp* bugs in a single component by marking the
>> ones good for newbies as "newbie" and removing the "ramp*" tags from
>> "newbie" issues.
>>
>> I hope this will produce a nice set of issues for newbies to choose
>> from to help us entice new developers, testers, doc writers, and so
>> on.
>>
>> On Fri, Oct 21, 2016 at 9:32 AM, Marcel Kornacker <mar...@cloudera.com> 
>> wrote:
>>> I agree, we have enough rules regarding jira labeling already.
>>>
>>> A "newbie" label to indicate tasks that someone without any/much
>>> exposure to the codebase should be able to handle seems like a good
>>> idea.
>>>
>>> "Ramp-up" really means something else (good learning experience, but
>>> could be moderately complex and require some understanding of the
>>> codebase).
>>>
>>> On Thu, Oct 20, 2016 at 10:32 PM, Henry Robinson <he...@cloudera.com> wrote:
>>>> I still think that adding more complexity to the JIRA state space is only
>>>> going to increase the cognitive burden of managing JIRAs, something I spend
>>>> quite a lot of my life doing already. It would need to be a pretty big win
>>>> for it to be worth it to me.
>>>>
>>>> Happy to go with the consensus, but would prefer to avoid more labels where
>>>> possible.
>>>>
>>>> On 20 October 2016 at 16:44, Jim Apple <jbap...@cloudera.com> wrote:
>>>>
>>>>> Ah, I was only suggesting this "non-newbie" as a tool for committers
>>>>> to use to help us find good newbie bugs, not as a tool for newbies.
>>>>>
>>>>> It will probably take a while to triage everything, and some people
>>>>> will not get their triaging work done in a timely fashion, and new
>>>>> bugs will show up and need triaging, and old bugs will (hopefully) be
>>>>> fixed by newbies, requiring us to go and find new bugs to label
>>>>> "newbie"
>>>>>
>>>>> On Thu, Oct 20, 2016 at 4:40 PM, Henry Robinson <he...@cloudera.com>
>>>>> wrote:
>>>>> > Right. Which 'newbie' is going to have that degree of JIRA nuance?
>>>>> >
>>>>> > I also feel it's better to show new contributors what they *could* do,
>>>>> than
>>>>> > to carefully mark things that they *should not* do.
>>>>> >
>>>>> > On 20 October 2016 at 16:28, Jim Apple <jbap...@cloudera.com> wrote:
>>>>> >
>>>>> >> So you're suggesting that there be no label for "this is triaged and
>>>>> >> not suitable for newbies"?
>>>>> >>
>>>>> >> On Thu, Oct 20, 2016 at 4:14 PM, Henry Robinson <he...@cloudera.com>
>>>>> >> wrote:
>>>>> >> > I would strongly prefer not adding "non-newbie". It seems to have
>>>>> limited
>>>>> >> > use, and is another way to increase the state space of JIRA labels,
>>>>> >> > components, etc, etc.
>>>>> >> >
>>>>> >> > Let's just find some good first-patch candidates and tag them.
>>>>> >> >
>>>>> >> > On 20 October 2016 at 16:03, Jim Apple <jbap...@cloudera.com> wrote:
>>>>> >> >
>>>>> >> >> Since I have heard no objection to these tag names, I'm picking
>>>>> >> >> "newbie" and "non-newbie". I am going to start emailing some PMC
>>>>> >> >> members to ask them to categorize issues.
>>>>> >> >>
>>>>> >> >> On Tue, Oct 18, 2016 at 9:57 AM, Jim Apple <jbap...@cloudera.com>
>>>>> >> wrote:
>>>>> >> >> > How shall we distinguish between the following t

Re: crash in PHJ on master

[Jim Apple]

I have a Jenkins build that passed all of the e2e tests (core
exploration strategy, debug build) with no crashes.

On Mon, Oct 24, 2016 at 8:35 AM, Marcel Kornacker  wrote:
> I'm seeing a crash in PartitionedHashJoinNode, is anyone experiencing the
> same or know what's going on?
>
> Here is the stack:
> C  [libExec.so+0x53706c]  std::unique_ptr std::default_delete >::get() const+0x18
> C  [libExec.so+0x54d6bc]
>  impala::PartitionedHashJoinNode::NodeDebugString() const+0x296
> C  [libExec.so+0x54d0bd]
>  
> impala::PartitionedHashJoinNode::UpdateState(impala::PartitionedHashJoinNode::HashJoinState)+0x37f
> C  [libExec.so+0x53eeb5]
>  impala::PartitionedHashJoinNode::Open(impala::RuntimeState*)+0x793
> C  [libExec.so+0x512fde]
>  impala::PartitionedAggregationNode::Open(impala::RuntimeState*)+0x4b4
> C  [libRuntime.so+0x53ff0e]
>  impala::PlanFragmentExecutor::OpenInternal()+0x3ae
> C  [libRuntime.so+0x53fa33]  impala::PlanFragmentExecutor::Open()+0x229
> C  [libService.so+0x30e7a8]
>  impala::FragmentMgr::FragmentExecState::Exec()+0x66
> C  [libService.so+0x31c0c8]
>  impala::FragmentMgr::FragmentThread(impala::TUniqueId)+0x78
> C  [libService.so+0x33863e]  boost::_mfi::mf1 impala::TUniqueId>::operator()(impala::FragmentMgr*, impala::TUniqueId)
> const+0x84
> C  [libService.so+0x336c6b]  void
> boost::_bi::list2,
> boost::_bi::value >::operator() impala::FragmentMgr, impala::TUniqueId>,
> boost::_bi::list0>(boost::_bi::type, boost::_mfi::mf1 impala::FragmentMgr, impala::TUniqueId>&, boost::_bi::list0&, int)+0x7d
> C  [libService.so+0x335557]  boost::_bi::bind_t boost::_mfi::mf1,
> boost::_bi::list2,
> boost::_bi::value > >::operator()()+0x3b
> C  [libService.so+0x33288c]
>  boost::detail::function::void_function_obj_invoker0 boost::_mfi::mf1,
> boost::_bi::list2,
> boost::_bi::value > >,
> void>::invoke(boost::detail::function::function_buffer&)+0x23
> C  [libUtil.so+0x2e75f0]  boost::function0::operator()() const+0x52
> C  [libUtil.so+0x2e6b97]  impala::Thread::SuperviseThread(std::string
> const&, std::string const&, boost::function,
> impala::Promise*)+0x2c5
> C  [libUtil.so+0x2ee48e]  void
> boost::_bi::list4 boost::_bi::value, boost::_bi::value >, boost::_bi::value >::operator() (*)(std::string const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list0>(boost::_bi::type, void
> (*&)(std::string const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list0&, int)+0xb2
> C  [libUtil.so+0x2ee3d1]  boost::_bi::bind_t const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list4 boost::_bi::value, boost::_bi::value >, boost::_bi::value > >::operator()()+0x3b
> C  [libUtil.so+0x2ee32c]
>  boost::detail::thread_data const&, std::string const&, boost::function,
> impala::Promise*), boost::_bi::list4 boost::_bi::value, boost::_bi::value >, boost::_bi::value > > >::run()+0x1e
>
> I'm at git hash ff6b450  (IMPALA-4285/IMPALA-4286: Fixes for Parquet
> scanner with MT_DOP > 0), ie, the latest on master.

Re: Bulk JIRA email, or how to find good newbie tasks

[Jim Apple]

SGTM. I will pick some of the PMC members and ask them to individually
classify all of the ramp* bugs in a single component by marking the
ones good for newbies as "newbie" and removing the "ramp*" tags from
"newbie" issues.

I hope this will produce a nice set of issues for newbies to choose
from to help us entice new developers, testers, doc writers, and so
on.

On Fri, Oct 21, 2016 at 9:32 AM, Marcel Kornacker <mar...@cloudera.com> wrote:
> I agree, we have enough rules regarding jira labeling already.
>
> A "newbie" label to indicate tasks that someone without any/much
> exposure to the codebase should be able to handle seems like a good
> idea.
>
> "Ramp-up" really means something else (good learning experience, but
> could be moderately complex and require some understanding of the
> codebase).
>
> On Thu, Oct 20, 2016 at 10:32 PM, Henry Robinson <he...@cloudera.com> wrote:
>> I still think that adding more complexity to the JIRA state space is only
>> going to increase the cognitive burden of managing JIRAs, something I spend
>> quite a lot of my life doing already. It would need to be a pretty big win
>> for it to be worth it to me.
>>
>> Happy to go with the consensus, but would prefer to avoid more labels where
>> possible.
>>
>> On 20 October 2016 at 16:44, Jim Apple <jbap...@cloudera.com> wrote:
>>
>>> Ah, I was only suggesting this "non-newbie" as a tool for committers
>>> to use to help us find good newbie bugs, not as a tool for newbies.
>>>
>>> It will probably take a while to triage everything, and some people
>>> will not get their triaging work done in a timely fashion, and new
>>> bugs will show up and need triaging, and old bugs will (hopefully) be
>>> fixed by newbies, requiring us to go and find new bugs to label
>>> "newbie"
>>>
>>> On Thu, Oct 20, 2016 at 4:40 PM, Henry Robinson <he...@cloudera.com>
>>> wrote:
>>> > Right. Which 'newbie' is going to have that degree of JIRA nuance?
>>> >
>>> > I also feel it's better to show new contributors what they *could* do,
>>> than
>>> > to carefully mark things that they *should not* do.
>>> >
>>> > On 20 October 2016 at 16:28, Jim Apple <jbap...@cloudera.com> wrote:
>>> >
>>> >> So you're suggesting that there be no label for "this is triaged and
>>> >> not suitable for newbies"?
>>> >>
>>> >> On Thu, Oct 20, 2016 at 4:14 PM, Henry Robinson <he...@cloudera.com>
>>> >> wrote:
>>> >> > I would strongly prefer not adding "non-newbie". It seems to have
>>> limited
>>> >> > use, and is another way to increase the state space of JIRA labels,
>>> >> > components, etc, etc.
>>> >> >
>>> >> > Let's just find some good first-patch candidates and tag them.
>>> >> >
>>> >> > On 20 October 2016 at 16:03, Jim Apple <jbap...@cloudera.com> wrote:
>>> >> >
>>> >> >> Since I have heard no objection to these tag names, I'm picking
>>> >> >> "newbie" and "non-newbie". I am going to start emailing some PMC
>>> >> >> members to ask them to categorize issues.
>>> >> >>
>>> >> >> On Tue, Oct 18, 2016 at 9:57 AM, Jim Apple <jbap...@cloudera.com>
>>> >> wrote:
>>> >> >> > How shall we distinguish between the following three classes of
>>> >> issues:
>>> >> >> >
>>> >> >> > 1. Un-triaged ramp-up issues
>>> >> >> > 2. ramp-up issues that are not for newbies
>>> >> >> > 3. newbie issues
>>> >> >> >
>>> >> >> > We could add two tags, "newbie" and "non-newbie". We could call the
>>> >> >> > second tag something other than "non-newbie", like "second-patch"
>>> or
>>> >> >> > "sophomore".
>>> >> >> >
>>> >> >> > Thoughts?
>>> >> >> >
>>> >> >> > On Mon, Oct 17, 2016 at 10:27 PM, Marcel Kornacker <
>>> >> mar...@cloudera.com>
>>> >> >> wrote:
>>> >> >> >> Please don't add that comment. :)
>>> >> >> >>
>>> >

Re: Bulk JIRA email, or how to find good newbie tasks

[Jim Apple]

Ah, I was only suggesting this "non-newbie" as a tool for committers
to use to help us find good newbie bugs, not as a tool for newbies.

It will probably take a while to triage everything, and some people
will not get their triaging work done in a timely fashion, and new
bugs will show up and need triaging, and old bugs will (hopefully) be
fixed by newbies, requiring us to go and find new bugs to label
"newbie"

On Thu, Oct 20, 2016 at 4:40 PM, Henry Robinson <he...@cloudera.com> wrote:
> Right. Which 'newbie' is going to have that degree of JIRA nuance?
>
> I also feel it's better to show new contributors what they *could* do, than
> to carefully mark things that they *should not* do.
>
> On 20 October 2016 at 16:28, Jim Apple <jbap...@cloudera.com> wrote:
>
>> So you're suggesting that there be no label for "this is triaged and
>> not suitable for newbies"?
>>
>> On Thu, Oct 20, 2016 at 4:14 PM, Henry Robinson <he...@cloudera.com>
>> wrote:
>> > I would strongly prefer not adding "non-newbie". It seems to have limited
>> > use, and is another way to increase the state space of JIRA labels,
>> > components, etc, etc.
>> >
>> > Let's just find some good first-patch candidates and tag them.
>> >
>> > On 20 October 2016 at 16:03, Jim Apple <jbap...@cloudera.com> wrote:
>> >
>> >> Since I have heard no objection to these tag names, I'm picking
>> >> "newbie" and "non-newbie". I am going to start emailing some PMC
>> >> members to ask them to categorize issues.
>> >>
>> >> On Tue, Oct 18, 2016 at 9:57 AM, Jim Apple <jbap...@cloudera.com>
>> wrote:
>> >> > How shall we distinguish between the following three classes of
>> issues:
>> >> >
>> >> > 1. Un-triaged ramp-up issues
>> >> > 2. ramp-up issues that are not for newbies
>> >> > 3. newbie issues
>> >> >
>> >> > We could add two tags, "newbie" and "non-newbie". We could call the
>> >> > second tag something other than "non-newbie", like "second-patch" or
>> >> > "sophomore".
>> >> >
>> >> > Thoughts?
>> >> >
>> >> > On Mon, Oct 17, 2016 at 10:27 PM, Marcel Kornacker <
>> mar...@cloudera.com>
>> >> wrote:
>> >> >> Please don't add that comment. :)
>> >> >>
>> >> >> What's currently labelled ramp-up is often not a good newbie task
>> (and
>> >> >> maybe not even a good ramp-up task). The best way to identify newbie
>> >> >> tasks is for a few senior engineers to sift through the ramp-up tasks
>> >> >> and pick out maybe a few dozen that truly qualify as newbie tasks.
>> >> >>
>> >> >> I'm happy to help out with that when I get back.
>> >> >>
>> >> >> On Mon, Oct 17, 2016 at 6:50 PM, Jim Apple <jbap...@cloudera.com>
>> >> wrote:
>> >> >>> The Impala JIRA has 129 tasks that have no assignee, are still open,
>> >> >>> and are labelled ramp* (i.e. ramp-up, ramp-up-introductory, etc.).
>> >> >>>
>> >> >>> I'd like to find which of those tasks are good tasks for someone who
>> >> >>> is making their first Impala patch. I intend to promote those on one
>> >> >>> or more of : the blog, the twitter account, this list, the user
>> list,
>> >> >>> helpwanted.apache.org, and so on.
>> >> >>>
>> >> >>> The tasks should be the kind of thing that someone won't need too
>> much
>> >> >>> hand-holding on, once their have their dev environment up and
>> working.
>> >> >>>
>> >> >>> To do this, I was thinking of adding a comment to all 129 tasks to
>> ask
>> >> >>> the watchers of each issue if it should be labelled "newbie". This
>> >> >>> will send hundreds of emails, which is a bummer, but it seems to me
>> >> >>> like the best way to track the discussions and decisions.
>> >> >>>
>> >> >>> What does everyone think?
>> >>
>> >
>> >
>> >
>> > --
>> > Henry Robinson
>> > Software Engineer
>> > Cloudera
>> > 415-994-6679
>>
>
>
>
> --
> Henry Robinson
> Software Engineer
> Cloudera
> 415-994-6679

Re: Bulk JIRA email, or how to find good newbie tasks

[Jim Apple]

So you're suggesting that there be no label for "this is triaged and
not suitable for newbies"?

On Thu, Oct 20, 2016 at 4:14 PM, Henry Robinson <he...@cloudera.com> wrote:
> I would strongly prefer not adding "non-newbie". It seems to have limited
> use, and is another way to increase the state space of JIRA labels,
> components, etc, etc.
>
> Let's just find some good first-patch candidates and tag them.
>
> On 20 October 2016 at 16:03, Jim Apple <jbap...@cloudera.com> wrote:
>
>> Since I have heard no objection to these tag names, I'm picking
>> "newbie" and "non-newbie". I am going to start emailing some PMC
>> members to ask them to categorize issues.
>>
>> On Tue, Oct 18, 2016 at 9:57 AM, Jim Apple <jbap...@cloudera.com> wrote:
>> > How shall we distinguish between the following three classes of issues:
>> >
>> > 1. Un-triaged ramp-up issues
>> > 2. ramp-up issues that are not for newbies
>> > 3. newbie issues
>> >
>> > We could add two tags, "newbie" and "non-newbie". We could call the
>> > second tag something other than "non-newbie", like "second-patch" or
>> > "sophomore".
>> >
>> > Thoughts?
>> >
>> > On Mon, Oct 17, 2016 at 10:27 PM, Marcel Kornacker <mar...@cloudera.com>
>> wrote:
>> >> Please don't add that comment. :)
>> >>
>> >> What's currently labelled ramp-up is often not a good newbie task (and
>> >> maybe not even a good ramp-up task). The best way to identify newbie
>> >> tasks is for a few senior engineers to sift through the ramp-up tasks
>> >> and pick out maybe a few dozen that truly qualify as newbie tasks.
>> >>
>> >> I'm happy to help out with that when I get back.
>> >>
>> >> On Mon, Oct 17, 2016 at 6:50 PM, Jim Apple <jbap...@cloudera.com>
>> wrote:
>> >>> The Impala JIRA has 129 tasks that have no assignee, are still open,
>> >>> and are labelled ramp* (i.e. ramp-up, ramp-up-introductory, etc.).
>> >>>
>> >>> I'd like to find which of those tasks are good tasks for someone who
>> >>> is making their first Impala patch. I intend to promote those on one
>> >>> or more of : the blog, the twitter account, this list, the user list,
>> >>> helpwanted.apache.org, and so on.
>> >>>
>> >>> The tasks should be the kind of thing that someone won't need too much
>> >>> hand-holding on, once their have their dev environment up and working.
>> >>>
>> >>> To do this, I was thinking of adding a comment to all 129 tasks to ask
>> >>> the watchers of each issue if it should be labelled "newbie". This
>> >>> will send hundreds of emails, which is a bummer, but it seems to me
>> >>> like the best way to track the discussions and decisions.
>> >>>
>> >>> What does everyone think?
>>
>
>
>
> --
> Henry Robinson
> Software Engineer
> Cloudera
> 415-994-6679

Re: Bulk JIRA email, or how to find good newbie tasks

[Jim Apple]

Since I have heard no objection to these tag names, I'm picking
"newbie" and "non-newbie". I am going to start emailing some PMC
members to ask them to categorize issues.

On Tue, Oct 18, 2016 at 9:57 AM, Jim Apple <jbap...@cloudera.com> wrote:
> How shall we distinguish between the following three classes of issues:
>
> 1. Un-triaged ramp-up issues
> 2. ramp-up issues that are not for newbies
> 3. newbie issues
>
> We could add two tags, "newbie" and "non-newbie". We could call the
> second tag something other than "non-newbie", like "second-patch" or
> "sophomore".
>
> Thoughts?
>
> On Mon, Oct 17, 2016 at 10:27 PM, Marcel Kornacker <mar...@cloudera.com> 
> wrote:
>> Please don't add that comment. :)
>>
>> What's currently labelled ramp-up is often not a good newbie task (and
>> maybe not even a good ramp-up task). The best way to identify newbie
>> tasks is for a few senior engineers to sift through the ramp-up tasks
>> and pick out maybe a few dozen that truly qualify as newbie tasks.
>>
>> I'm happy to help out with that when I get back.
>>
>> On Mon, Oct 17, 2016 at 6:50 PM, Jim Apple <jbap...@cloudera.com> wrote:
>>> The Impala JIRA has 129 tasks that have no assignee, are still open,
>>> and are labelled ramp* (i.e. ramp-up, ramp-up-introductory, etc.).
>>>
>>> I'd like to find which of those tasks are good tasks for someone who
>>> is making their first Impala patch. I intend to promote those on one
>>> or more of : the blog, the twitter account, this list, the user list,
>>> helpwanted.apache.org, and so on.
>>>
>>> The tasks should be the kind of thing that someone won't need too much
>>> hand-holding on, once their have their dev environment up and working.
>>>
>>> To do this, I was thinking of adding a comment to all 129 tasks to ask
>>> the watchers of each issue if it should be labelled "newbie". This
>>> will send hundreds of emails, which is a bummer, but it seems to me
>>> like the best way to track the discussions and decisions.
>>>
>>> What does everyone think?

Re: Bulk JIRA email, or how to find good newbie tasks

[Jim Apple]

How shall we distinguish between the following three classes of issues:

1. Un-triaged ramp-up issues
2. ramp-up issues that are not for newbies
3. newbie issues

We could add two tags, "newbie" and "non-newbie". We could call the
second tag something other than "non-newbie", like "second-patch" or
"sophomore".

Thoughts?

On Mon, Oct 17, 2016 at 10:27 PM, Marcel Kornacker <mar...@cloudera.com> wrote:
> Please don't add that comment. :)
>
> What's currently labelled ramp-up is often not a good newbie task (and
> maybe not even a good ramp-up task). The best way to identify newbie
> tasks is for a few senior engineers to sift through the ramp-up tasks
> and pick out maybe a few dozen that truly qualify as newbie tasks.
>
> I'm happy to help out with that when I get back.
>
> On Mon, Oct 17, 2016 at 6:50 PM, Jim Apple <jbap...@cloudera.com> wrote:
>> The Impala JIRA has 129 tasks that have no assignee, are still open,
>> and are labelled ramp* (i.e. ramp-up, ramp-up-introductory, etc.).
>>
>> I'd like to find which of those tasks are good tasks for someone who
>> is making their first Impala patch. I intend to promote those on one
>> or more of : the blog, the twitter account, this list, the user list,
>> helpwanted.apache.org, and so on.
>>
>> The tasks should be the kind of thing that someone won't need too much
>> hand-holding on, once their have their dev environment up and working.
>>
>> To do this, I was thinking of adding a comment to all 129 tasks to ask
>> the watchers of each issue if it should be labelled "newbie". This
>> will send hundreds of emails, which is a bummer, but it seems to me
>> like the best way to track the discussions and decisions.
>>
>> What does everyone think?

Re: 答复: problems for the debuging the frontend in Eclipse IDE

[Jim Apple]

Unfortunately, this mailing list is configured not to forward pictures.

On Mon, Oct 17, 2016 at 11:04 PM, Zhangjun (Jerry) <
jerry.zhang...@huawei.com> wrote:

> I run the below command outside Eclipse IDE, it can run successfully
>
> Command is :
>
>   mvn test -Dtest=com.cloudera.impala.planner.PlannerTest
>
>
>
> In Eclipse IDE, when I run the test case, the console pint the below
> message and shutdown suddenly
>
>
>
> -邮件原件-
> 发件人: Wangtieying
> 发送时间: 2016年10月18日 8:59
> 收件人: Zhangjun (Jerry)
> 主题: 转发: problems for the debuging the frontend in Eclipse IDE
>
>
>
>
>
>
>
> -邮件原件-
>
> 发件人: Jim Apple [mailto:jbap...@cloudera.com <jbap...@cloudera.com>]
>
> 发送时间: 2016年10月18日 1:41
>
> 收件人: dev@impala
>
> 抄送: Wangtieying
>
> 主题: Re: problems for the debuging the frontend in Eclipse IDE
>
>
>
> Do the planner tests pass when you run them outside of the Eclipse IDE?
>
>
>
> On Sun, Oct 16, 2016 at 6:47 PM, Zhangjun (Jerry) <
> jerry.zhang...@huawei.com> wrote:
>
> > Hi，
>
> > I download the source code from master branch. And build it and run all
> service with testdata successfully. But when I debug the planner test
> case(PlannerTest.java) for frontend in Eclipse IDE, the application
> shutdown and no any exception print( I have modified the log level to “all
> ” level in Log4j.properties file.
>
> > I trace the case and found it occur after the loaded the libfessuport.so
> in the course of initialization of Frontend object.
>
> >
>
> > Thanks!
>

Re: Impala build error with libgflags-dev installed

[Jim Apple]

Thanks! Can you file a bug for this?

On Mon, Oct 17, 2016 at 12:01 PM, Yonghyun Hwang  wrote:
> Impala build sees an error if "libgflags-dev" package  is already installed
> in a machine. Here is the error.
>
> $ source bin/impala-config.sh && buildall.sh -skiptests -notests
> ...
> ...
> *Linking CXX shared library* ../../build/debug/gutil/libgutil.so
> /home/impala/Impala/toolchain/binutils-2.26-p1/bin/ld:
> /usr/lib/x86_64-linux-gnu/libgflags.a(libgflags_la-gflags.o): relocation
> R_X86_64_32 against `.rodata' can not be used when making a shared object;
> recompile with -fPIC
> */usr/lib/x86_64-linux-gnu/libgflags.a*: *error adding symbols: Bad value*
> collect2: error: ld returned 1 exit status
> make[3]: *** [be/build/debug/gutil/libgutil.so] Error 1
>
> $ apt search libgflags-dev | grep installed
>
> libgflags-dev/trusty,now 2.0-1.1ubuntu1 amd64 [installed]
>
> $ apt-file list libgflags-dev | grep libgflags.a
> libgflags-dev:* /usr/lib/x86_64-linux-gnu/libgflags.a*
>
>
> I think cmake picks up libgflags.a from "libgflags-dev" instead of that
> from ${IMPALA_HOME}/toolchain/gflags-2.0, which causes the build failure.
>
>
> -Yonghyun

Re: Bootstrapping an Impala Development Environment From Scratch

[Jim Apple]

I think this takes at least 80GB of disk space and 16GB of RAM.

On Mon, Oct 17, 2016 at 11:50 AM, Laszlo Gaal <laszlo.g...@cloudera.com> wrote:
> Running in a VM is good idea. Do  you have a recommendation on how much
> memory the VM should be configured with?
>
> On Mon, Oct 17, 2016 at 7:58 PM, Jim Apple <jbap...@cloudera.com> wrote:
>
>> If you are running Ubuntu 14.04, you can bootstrap a development
>> environment using the script bin/bootstrap_development.sh[0]. It will
>> alter your environment, including ~/.ssh/config and /etc/hosts, so
>> consider running it in a VM or container.
>>
>> It takes 6-7 hours in total to load all of the testdata and run all of
>> the tests. See the comments in the file for more information.
>>
>> [0]: https://git-wip-us.apache.org/repos/asf?p=incubator-impala.
>> git;a=blob;f=bin/bootstrap_development.sh;h=8c4f742ae058f8017858d2a749e882
>> 4be58bd410;hb=HEAD
>>

Bootstrapping an Impala Development Environment From Scratch

[Jim Apple]

If you are running Ubuntu 14.04, you can bootstrap a development
environment using the script bin/bootstrap_development.sh[0]. It will
alter your environment, including ~/.ssh/config and /etc/hosts, so
consider running it in a VM or container.

It takes 6-7 hours in total to load all of the testdata and run all of
the tests. See the comments in the file for more information.

[0]: 
https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=blob;f=bin/bootstrap_development.sh;h=8c4f742ae058f8017858d2a749e8824be58bd410;hb=HEAD

Re: problems for the debuging the frontend in Eclipse IDE

[Jim Apple]

Do the planner tests pass when you run them outside of the Eclipse IDE?

On Sun, Oct 16, 2016 at 6:47 PM, Zhangjun (Jerry)
 wrote:
> Hi，
> I download the source code from master branch. And build it and run all 
> service with testdata successfully. But when I debug the planner test 
> case(PlannerTest.java) for frontend in Eclipse IDE, the application shutdown 
> and no any exception print( I have modified the log level to “all” level in 
> Log4j.properties file.
> I trace the case and found it occur after the loaded the libfessuport.so in 
> the course of initialization of Frontend object.
>
> Thanks!

Bulk JIRA email, or how to find good newbie tasks

[Jim Apple]

The Impala JIRA has 129 tasks that have no assignee, are still open,
and are labelled ramp* (i.e. ramp-up, ramp-up-introductory, etc.).

I'd like to find which of those tasks are good tasks for someone who
is making their first Impala patch. I intend to promote those on one
or more of : the blog, the twitter account, this list, the user list,
helpwanted.apache.org, and so on.

The tasks should be the kind of thing that someone won't need too much
hand-holding on, once their have their dev environment up and working.

To do this, I was thinking of adding a comment to all 129 tasks to ask
the watchers of each issue if it should be labelled "newbie". This
will send hundreds of emails, which is a bummer, but it seems to me
like the best way to track the discussions and decisions.

What does everyone think?

Short git hashes are sometimes ambiguous

[Jim Apple]

There are 5079 commits in trunk, which means there are 12.9 million
unordered pairs of git hashes. git uses 7 characters for short git
hashes by default, though some git tools will use longer hashes for
ambiguous commits. In fact, we already have that situation: the seven
character string "fa5eca0" is a prefix of two different commit hashes
in the "master" branch.

Consider using longer hashes in the future in your tooling and your
references to commits that appear in bug reports or comments or the
like. A bug report that referenced fa5eca0 would now be ambiguous.

[Impala-CR](cdh5-2.6.0 5.8.0) PREVIEW: IMPALA-4223: Handle truncated file read from HDFS cache

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: PREVIEW: IMPALA-4223: Handle truncated file read from HDFS cache
..


Patch Set 1:

> This should probably go in ASF's master branch, then get
 > cherry-picked.

Any thoughts on this, Lars?

-- 
To view, visit http://gerrit.cloudera.org:8080/4645
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id1e1fdb0211819c5938956abb13b512350a46f1a
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-HasComments: No

Re: A modest proposal: use the gold linker by default

[Jim Apple]

I think clang doesn't have a CLI option to use gold. Does buildall.sh -asan
interact well with gold?

On Thu, Oct 13, 2016 at 8:25 AM, Tim Armstrong 
wrote:

> The gold linker is much faster for building static binaries so will be much
> friendlier for new developers.
>
> I don't think I've seen a problem attributable to gold since it was added ~
> 6 months ago. I've been using it since then and we've been using it for
> some Impala test jobs as well for at least a couple of months. I feel
> confident that any major problems have been shaken out.
>
> Does anyone have any objections?
>
> - Tim
>

Re: Impala building on Ubuntu 16.04

[Jim Apple]

Relevant bugs:

https://issues.cloudera.org/browse/IMPALA-3924

https://issues.cloudera.org/browse/IMPALA-3926

https://issues.cloudera.org/browse/IMPALA-3932

On Wed, Oct 12, 2016 at 4:17 PM, Tim Armstrong  wrote:
> I was able to get it working and passing at least some tests.
>
> Unfortunately I had to resolve to adding this to my .bashrc file to
> override the toolchain libraries. Hopefully it won't be necessary at some
> point in the future.
>
> export LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu

Re: clang-tidy: sweeping linter changes?

[Jim Apple]

Thanks - this is working for me!

I'm going to try and curate some config settings. I'll update dev@ if
and when I have something useful.

On Sat, Oct 8, 2016 at 1:03 PM, Todd Lipcon <t...@cloudera.com> wrote:
> Check out clang-tidy-diff which can limit warnings based on a patch. You
> can control the amount of context given to it so that people get warnings
> about surrounding code or really just strictly the changed lines.
>
> We are using this in kudu and it's moderately helpful.
>
> Todd
>
> On Oct 8, 2016 12:13 PM, "Jim Apple" <jbap...@cloudera.com> wrote:
>
>> Should we make large (but mostly cosmetic) changes to our source code to
>> help new contributors write clean C++?
>>
>> We now have a .clang-format file to help new contributors write code with
>> correct whitespace formatting. Many other parts of our C++ style can be
>> enforced by clang-tidy, a stronger tool included in our toolchain that
>> understands much more than whitespace.
>>
>> There is a cost, however -- clang-tidy is not nearly as good at targeting
>> its warnings at newly-written code. To use it well(*), we will need to fix
>> a lot of warnings about our existing code.
>>
>> Should we fix the existing code to make clang-tidy accept it? This would be
>> a big change from what we did with clang-format, in which we ask that new
>> code be clang-formatted, but we allow old code to remain unformatted unless
>> it is touched again.
>>
>> Some randomly-selected examples of the kind of things clang-tidy can
>> enforce (**):
>>
>> * TypeNamesAreCamelCase
>> * "if (b == true)" should be written "if (b)"
>> * Function prototypes should have the same parameter names as function
>> definitions
>> * Single-argument constructors should be "explicit"
>>
>> We can configure clang-tidy, just like clang-format, to be more or less
>> strict for any given check.
>>
>> (*) without building up a lot of ad-hoc tooling to suppress warnings about
>> existing code
>>
>> (**) partial list here:
>> http://clang.llvm.org/extra/clang-tidy/checks/list.html. This list does
>> not
>> include some diagnostics that are part of clang proper and that clang-tidy
>> can warn about, though we can also acquire a list of those and tune them
>> for our needs.
>>

clang-tidy: sweeping linter changes?

[Jim Apple]

Should we make large (but mostly cosmetic) changes to our source code to
help new contributors write clean C++?

We now have a .clang-format file to help new contributors write code with
correct whitespace formatting. Many other parts of our C++ style can be
enforced by clang-tidy, a stronger tool included in our toolchain that
understands much more than whitespace.

There is a cost, however -- clang-tidy is not nearly as good at targeting
its warnings at newly-written code. To use it well(*), we will need to fix
a lot of warnings about our existing code.

Should we fix the existing code to make clang-tidy accept it? This would be
a big change from what we did with clang-format, in which we ask that new
code be clang-formatted, but we allow old code to remain unformatted unless
it is touched again.

Some randomly-selected examples of the kind of things clang-tidy can
enforce (**):

* TypeNamesAreCamelCase
* "if (b == true)" should be written "if (b)"
* Function prototypes should have the same parameter names as function
definitions
* Single-argument constructors should be "explicit"

We can configure clang-tidy, just like clang-format, to be more or less
strict for any given check.

(*) without building up a lot of ad-hoc tooling to suppress warnings about
existing code

(**) partial list here:
http://clang.llvm.org/extra/clang-tidy/checks/list.html. This list does not
include some diagnostics that are part of clang proper and that clang-tidy
can warn about, though we can also acquire a list of those and tune them
for our needs.

[Impala-CR](cdh5-2.4.0 5.6.x) CDH-41243: Parquet scanner regression on wide tables

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: CDH-41243: Parquet scanner regression on wide tables
..


Abandoned

Please use "Impala-ASF"

-- 
To view, visit http://gerrit.cloudera.org:8080/3491
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ic94d818275709bfeb6880e26f45a0dd16ed07af4
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.4.0_5.6.x
Gerrit-Owner: Huaisi Xu <h...@cloudera.com>

[Impala-CR](cdh5-2.3.0 5.5.x) IMPALA-2385: The repeated runs configuration for running tests should not restart Impala.

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-2385: The repeated runs configuration for running tests 
should not restart Impala.
..


Abandoned

Please use "Impala-ASF"

-- 
To view, visit http://gerrit.cloudera.org:8080/1119
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ifb9cbeb390f26dac53186e85bc1f36b74b5560c7
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.3.0_5.5.x
Gerrit-Owner: Ishaan Joshi <ish...@cloudera.com>
Gerrit-Reviewer: Casey Ching <ca...@cloudera.com>
Gerrit-Reviewer: Harrison Sheinblatt <h...@hotmail.com>
Gerrit-Reviewer: Juan Yu <j...@cloudera.com>

How did I InconsistentFSStateException myself?

[Jim Apple]

How do I un-InconsistentFSStateException my development environment?

I did an asan build, then deleted all of the invalid cmake files (*), then
when I did another build and tried to testdata/bin/run-all.sh, datanodes
won't start and I get the following error message:

org.apache.hadoop.hdfs.server.common.InconsistentFSStateException:
Directory /home/jbapple/Impala/testdata/cluster/cdh5/node-2/data/dfs/dn is
in an inconsistent state: Can't format the storage directory because the
current/ directory is not empty.
at
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.checkEmptyCurrent(Storage.java:480)
at
org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:585)
at
org.apache.hadoop.hdfs.server.datanode.DataStorage.loadStorageDirectory(DataStorage.java:279)
at
org.apache.hadoop.hdfs.server.datanode.DataStorage.loadDataStorage(DataStorage.java:418)
at
org.apache.hadoop.hdfs.server.datanode.DataStorage.addStorageLocations(DataStorage.java:397)
at
org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:575)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1486)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1446)
at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:313)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:219)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:673)
at java.lang.Thread.run(Thread.java:745)
2016-10-06 15:03:43,300 FATAL
org.apache.hadoop.hdfs.server.datanode.DataNode: Initialization failed for
Block pool  (Datanode Uuid unassigned) service to localhost/
127.0.0.1:20500. Exiting.
java.io.IOException: All specified directories are failed to load.
at
org.apache.hadoop.hdfs.server.datanode.DataStorage.recoverTransitionRead(DataStorage.java:576)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.initStorage(DataNode.java:1486)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.initBlockPool(DataNode.java:1446)
at
org.apache.hadoop.hdfs.server.datanode.BPOfferService.verifyAndSetNamespaceInfo(BPOfferService.java:313)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.connectToNNAndHandshake(BPServiceActor.java:219)
at
org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:673)
at java.lang.Thread.run(Thread.java:745)


That message is
from ./testdata/cluster/cdh5/node-2/var/log/hadoop-hdfs/hdfs-datanode.log.

Should I clobber all of the stuff
in ./testdata/cluster/cdh5/node-2/data/dfs/dn/current?

(*) find -iname '*cmake*' -not -name CMakeLists.txt | grep -v -e
cmake_module | grep -v -e thirdparty | xargs rm -Rf

Squeasel and Mustache - "important" or "just an innocent embedded dependency"

[Jim Apple]

The Impala community, with our mentors, should decide if we should
request from CLoudera na additional software grant for our thirdparty
components.

https://lists.apache.org/thread.html/d0c3482cccb3d8bc0a121c14bd269c98fb078213e79432ad164d4e62@%3Cgeneral.incubator.apache.org%3E

Mustache and Squeasel are small components (4202 lines of code, total)
and we can include them in Impala because they are permissively
licensed. However, it may make sense for them to have a software grant
to the ASF if they "contain a patented algorithm or [are] core to how
Impala works".

My suspicion is that they do not and are not.

Thoughts?

[Impala-CR](cdh5-2.6.0 5.8.0) PREVIEW: IMPALA-4223: Handle truncated file read from HDFS cache

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: PREVIEW: IMPALA-4223: Handle truncated file read from HDFS cache
..


Patch Set 1:

This should probably go in ASF's master branch, then get cherry-picked.

-- 
To view, visit http://gerrit.cloudera.org:8080/4645
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: Id1e1fdb0211819c5938956abb13b512350a46f1a
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.6.0_5.8.0
Gerrit-Owner: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-HasComments: No

Re: How to install the custom built impala

[Jim Apple]

What have you tried? Did you try building with the old Cloudera Impala
repo at https://github.com/cloudera/impala/tree/cdh5-2.3.0_5.5.4?

On Wed, Oct 5, 2016 at 10:48 AM, Pradeep Nayak  wrote:
> Hi - I have verified my changes in the dev cluster which I had. However the
> test/staging cluster is at 5.5.4. Is there a quick way to build it for that
> version. ?
>
> Cheers!
> Pradeep
>
> http://pradeepnayak.in
> http://twitter.com/_prdp
>
> On Tue, Sep 13, 2016 at 4:54 PM, Tim Armstrong 
> wrote:
>
>> It looks like you built the debug Impala with dynamic linking
>> (-build_shared_libs or -so) whereas the CDH build uses static linking -
>> that might explain why the release build worked ok.
>>
>> It's expected that your binary is much larger, in the CDH packages the
>> debug symbols were stripped out and put in a separate impalad.debug file,
>> whereas just building a dev environment doesn't do that step. Should work
>> fine though.
>>
>> On Tue, Sep 13, 2016 at 4:44 PM, Pradeep Nayak 
>> wrote:
>>
>>> Well when I built the debug one and replaced it my cluster it seemed to
>>> have more dependencies which it was not able to find;
>>>
>>> ubuntu@ip-172-30-2-102:/opt/cloudera/parcels/CDH-5.8.0-1.cdh
>>> 5.8.0.p0.42/lib/impala/sbin-debug$ ldd impalad
>>> ./impalad: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version
>>> `GLIBCXX_3.4.20' not found (required by ./impalad)
>>> linux-vdso.so.1 =>  (0x7ffd87dc6000)
>>> libjsig.so => not found
>>> libTestUtil.so => not found
>>> libUtil.so => not found
>>> libRuntime.so => not found
>>> libExec.so => not found
>>> libCodeGen.so => not found
>>> libExprs.so => not found
>>> libRpc.so => not found
>>> libService.so => not found
>>> libStatestore.so => not found
>>> libCatalog.so => not found
>>> libResourceBroker.so => not found
>>> libImpalaThrift.so => not found
>>> libGlobalFlags.so => not found
>>> libCommon.so => not found
>>> libUdf.so => not found
>>> libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2
>>> (0x7fd4661be000)
>>> libThriftSaslTransport.so => not found
>>> libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0
>>> (0x7fd465f5f000)
>>> libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
>>> (0x7fd465b83000)
>>> libgutil.so => not found
>>> libhdfs.so.0.0.0 => not found
>>> libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x7fd46596a000)
>>> libjvm.so => not found
>>> libkudu_client.so.0 => not found
>>> librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
>>> (0x7fd465762000)
>>> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
>>> (0x7fd46555e000)
>>> libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
>>> (0x7fd46525a000)
>>> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x7fd464f54000)
>>> libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
>>> (0x7fd464d3e000)
>>> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
>>> (0x7fd464b2)
>>> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7fd46475b000)
>>> /lib64/ld-linux-x86-64.so.2 (0x7fd4663d9000)
>>>
>>>
>>>
>>> I went ahead and build the retail one and that was OKAY.
>>>
>>> ubuntu@ip-172-30-2-102:/opt/cloudera/parcels/CDH-5.8.0-1.cdh
>>> 5.8.0.p0.42/lib/impala/sbin-retail$ ldd impalad
>>> ./impalad: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version
>>> `CXXABI_1.3.8' not found (required by ./impalad)
>>> ./impalad: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version
>>> `GLIBCXX_3.4.20' not found (required by ./impalad)
>>> linux-vdso.so.1 =>  (0x7fffc2679000)
>>> libjsig.so => not found
>>> libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2
>>> (0x7f153629d000)
>>> libssl.so.1.0.0 => /lib/x86_64-linux-gnu/libssl.so.1.0.0
>>> (0x7f153603e000)
>>> libcrypto.so.1.0.0 => /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
>>> (0x7f1535c62000)
>>> libjvm.so => not found
>>> libkudu_client.so.0 => not found
>>> librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
>>> (0x7f1535a5a000)
>>> libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
>>> (0x7f1535856000)
>>> libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
>>> (0x7f1535552000)
>>> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x7f153524c000)
>>> libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
>>> (0x7f1535036000)
>>> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
>>> (0x7f1534e18000)
>>> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f1534a53000)
>>> /lib64/ld-linux-x86-64.so.2 (0x7f15364b8000)
>>>
>>>
>>> However I see the sizes being different. The one which came 

[ANNOUNCE] Apache Impala (incubating) 2.7.0 release

[Jim Apple]

The Apache Impala (incubating) team is pleased to announce the release
of Impala 2.7.0.

Impala is a high-performance C++ and Java SQL query engine for data
stored in Apache Hadoop-based clusters.

The release is available at: https://impala.incubator.apache.org/downloads.html

Thanks,

The Apache Impala (incubating) team

=

*Disclaimer*

Apache Impala is an effort undergoing incubation at The Apache
Software Foundation (ASF), sponsored by the name of Apache Incubator
PMC. Incubation is required of all newly accepted projects until a
further review indicates that the infrastructure, communications, and
decision making process have stabilized in a manner consistent with
other successful ASF projects. While incubation status is not
necessarily a reflection of the completeness or stability of the code,
it does indicate that the project has yet to be fully endorsed by the
ASF.

Looking for release managers for 2.8.0

[Jim Apple]

2.7.0 will be released tomorrow, I anticipate. Our next release will
be 2.8.0, I expect.

Some members of the IPMC, which is the group that will vote on our
graduation one day, have expressed a preference for incubator projects
to have more than one release manager before graduation. I am looking
for a volunteer for 2.8.0. I have written up instructions here:

https://cwiki.apache.org/confluence/display/IMPALA/DRAFT%3A+How+to+Release

You can contact me on- or off-list if you are interested. You must be
a committer to be a release manager.

Re: [RESULT] [VOTE] Impala 2.7.0 release candidate 3

[Jim Apple]

Thanks! A reminder to everyone: we are not yet permitted to publicize
this. We must wait 24 hours for the artefacts to propagate to the
mirrors. I will send an announcement email when we're set.

On Tue, Oct 4, 2016 at 2:09 PM, Henry Robinson <he...@apache.org> wrote:
> Amazing work Jim and everyone!
>
>
> -- Forwarded message ------
> From: Jim Apple <jbap...@cloudera.com>
> Date: 4 October 2016 at 14:05
> Subject: [RESULT] [VOTE] Impala 2.7.0 release candidate 3
> To: gene...@incubator.apache.org
>
>
> The vote passes with:
>
> +1's (all binding)
>
> Carl Steinbach
> Justin Mclean
> Tom White (one of Impala's mentors, see the PPMC vote thread:
> http://mail-archives.apache.org/mod_mbox/incubator-impala-
> dev/201609.mbox/%3CCAF-WD4Sf4wrNjn6MJjFJ7T07feuwE-KAmdrmjMava1jG5-wS-w@mail.
> gmail.com%3E
>
> or
>
> https://lists.apache.org/thread.html/56d822c7bdda9229ff8bb8e986862f
> 7e9f01729ebccdc278f5ae812f@%3Cdev.impala.apache.org%3E)
>
> No other votes.
>
> Thank you, everyone!
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org

Re: Contributions to Cloudera Impala

[Jim Apple]

> Yes, I'll start a new discussion about branching. Could you please share the
> complete address of dev@? Is it different than
> "dev@impala.incubator.apache.org"?

That is the address.

Re: Sorry for the spam

[Jim Apple]

I Abandoned all the cdh5-trunk patches that haven't been updates since
their warning one month ago. You can feel free to take step 2: shut
down all pushes to the gerrit project named "Impala"

On Thu, Sep 1, 2016 at 1:54 PM, Jim Apple <jbap...@cloudera.com> wrote:
> What if
>
> 1. (a) You re-open pushes (b) I post the following message to all open 
> patches:
>
> "Please update to using the new gerrit project, "Impala-ASF".
> Instructions are here:
>
> https://cwiki.apache.org/confluence/display/IMPALA/How+to+switch+to+Apache-hosted+git
>
> Pushes to this project will be disabled on October 1."
>
> 2. On October 1, you shut down all pushes to the gerrit project named "Impala"
>
> Thoughts?
>
> On Thu, Sep 1, 2016 at 1:23 PM, Henry Robinson <he...@cloudera.com> wrote:
>> Given the amount of time that has elapsed, perhaps it's time to say that
>> all changes should be moved over now?
>>
>> On 1 September 2016 at 13:21, Jim Apple <jbap...@cloudera.com> wrote:
>>
>>> For instance, there are still some open on
>>> https://gerrit.cloudera.org/#/q/project:Impala+branch:cdh5-trunk ,
>>> even though Impala-ASF opened at the end of July.
>>>
>>> On Thu, Sep 1, 2016 at 1:19 PM, Jim Apple <jbap...@cloudera.com> wrote:
>>> > When I sent out the instructions on how to move, I mentioned that some
>>> > people might want to keep iterating on their changes on that project
>>> > due to our inability to transfer the patch history and its gerrit
>>> > comments to the new "Impala-ASF" project. Some people may be iterating
>>> > on patches there for that reason.
>>> >
>>> > On Thu, Sep 1, 2016 at 1:04 PM, Henry Robinson <he...@cloudera.com>
>>> wrote:
>>> >> I just removed access for 'push' from everyone for the Impala project.
>>> Let
>>> >> me know if I should reinstate it, but AFAIK that project is no longer in
>>> >> any use.
>>> >>
>>> >> On 1 September 2016 at 13:00, Michael Brown <mi...@cloudera.com> wrote:
>>> >>
>>> >>> My hunch is you pushed to the Cloudera Impala Gerrit. There's an old
>>> >>> master branch lying there.
>>> >>>
>>> >>> Maybe use "git remote rename" or "git remote remove" ?
>>> >>>
>>> >>> (I muscle-memory-typo'd this same thing the other day but got lucky
>>> >>> and the push was rejected.)
>>> >>>
>>> >>> On Thu, Sep 1, 2016 at 12:49 PM, Thomas Tauber-Marshall
>>> >>> <tmarsh...@cloudera.com> wrote:
>>> >>> > For anyone wondering, I just pushed the wrong branch to gerrit,
>>> resulting
>>> >>> > in a few dozen reviews being created. I'm currently working on
>>> abandoning
>>> >>> > them all.
>>> >>> >
>>> >>> > Sorry for the spam.
>>> >>> >
>>> >>> > Thanks,
>>> >>> > Thomas
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Henry Robinson
>>> >> Software Engineer
>>> >> Cloudera
>>> >> 415-994-6679
>>>
>>
>>
>>
>> --
>> Henry Robinson
>> Software Engineer
>> Cloudera
>> 415-994-6679

[Impala-CR](cdh5-trunk) IMPALA-3804: Push per-split filtering into scanners

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-3804: Push per-split filtering into scanners
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3561
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I9f92178f642695e0e9ef901373a5e9f2878a78ce
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-2626: In-flight queries fail when statestore comes back online.

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-2626: In-flight queries fail when statestore comes back 
online.
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/1380
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I102391ab63270a9686cf45457b8384ffcd2abe8a
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Sailesh Mukil <sail...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-2328 Parquet scan should use min/max stats

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-2328 Parquet scan should use min/max stats
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3623
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I91de1f4d0fb2a982d06cd344e41901e3bf3c2cea
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Jian Wu <jan.chou...@gmail.com>
Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com>
Gerrit-Reviewer: Jian Wu <jan.chou...@gmail.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Michael Ho <k...@cloudera.com>
Gerrit-Reviewer: Mostafa Mokhtar <mmokh...@cloudera.com>

[Impala-CR](cdh5-trunk) Kimpala (Preview)

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: Kimpala (Preview)
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3571
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Idfa139e8393ed9f390ea8f3d1e352912f519953a
Gerrit-PatchSet: 4
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Bikramjeet Vig <bikramjeet@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-3766: Applying LZ4 compression on buffers before spilling

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-3766:  Applying LZ4 compression on buffers before 
spilling
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3478
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I4d49bd8d6d7643c84cefd1274c18b52907ca1488
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: anujphadke <apha...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Mostafa Mokhtar <mmokh...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Reviewer: anujphadke <apha...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-2890: Add support for ALTER for Kudu tables

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-2890: Add support for ALTER for Kudu tables
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3485
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I547623c090b4ffbe86e024a128576e7236d1ab41
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Bikramjeet Vig <bikramjeet@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>

[Impala-CR](cdh5-trunk) PREVIEW IMPALA-2550: RPC batching

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: PREVIEW IMPALA-2550: RPC batching
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3390
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I8e3a9d4a78b59394aaf343df08f6bfda22c3148e
Gerrit-PatchSet: 5
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <mar...@cloudera.com>

[Impala-CR](cdh5-trunk) PREVIEW: IMPALA-2550 Introduce query-wide execution context.

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: PREVIEW: IMPALA-2550 Introduce query-wide execution context.
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3686
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I9c513d08f718699ba0c4cdb90c117aaecf95d7fc
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <mar...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-2418: Display truncated Details column in profile summary

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-2418: Display truncated Details column in profile summary
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/2930
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I228057dc134cb46673d8370ff859c00cd9739cd4
Gerrit-PatchSet: 5
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Peter Ebert
Gerrit-Reviewer: Casey Ching <ca...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Peter Ebert

[Impala-CR](cdh5-trunk) Query gen: Add INSERT statements

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: Query gen: Add INSERT statements
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/2541
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I6110003789a4b6ae496c02a526c47c3249ef5307
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Casey Ching <ca...@cloudera.com>
Gerrit-Reviewer: Casey Ching <ca...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Silvius Rus <s...@cloudera.com>
Gerrit-Reviewer: Taras Bobrovytsky <tbobrovyt...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-3335: enable single-node optimizaion with joins

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-3335: enable single-node optimizaion with joins
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3161
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I7de9b759f444bbf3c6505e8dd0b0c7c58dbd1596
Gerrit-PatchSet: 5
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Taras Bobrovytsky <tbobrovyt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Taras Bobrovytsky <tbobrovyt...@cloudera.com>

[Impala-CR](cdh5-trunk) Fix Parquet timestamp behavior for Hive data

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: Fix Parquet timestamp behavior for Hive data
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/1681
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I81e8e14d3ec9d399c26756914a54c552757dfbd2
Gerrit-PatchSet: 10
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Taras Bobrovytsky <tbobrovyt...@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <mar...@cloudera.com>
Gerrit-Reviewer: Taras Bobrovytsky <tbobrovyt...@cloudera.com>

[Impala-CR](cdh5-trunk) Kudu: Remove TODO about checking for unsupported types in table loading

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: Kudu: Remove TODO about checking for unsupported types in table 
loading
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/2834
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I681d80776f28904a6171cac5aebde3a02327b295
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Casey Ching <ca...@cloudera.com>
Gerrit-Reviewer: Casey Ching <ca...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <mar...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-2518: DROP DATABASE CASCADE does not remove cache directives of tables

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-2518: DROP DATABASE CASCADE does not remove cache 
directives of tables
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/1966
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I58270e1be49e71a08e12021e7dddab01969d1810
Gerrit-PatchSet: 7
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Marcell Szabo <sz...@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Juan Yu <j...@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <mar...@cloudera.com>
Gerrit-Reviewer: Marcell Szabo <sz...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-2864: Ensure that client connections are closed after a failed RPC and a failed Open()

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-2864: Ensure that client connections are closed after a 
failed RPC and a failed Open()
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/1599
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ia7e883d8224304ad13a051f595d0e8abf4f9671e
Gerrit-PatchSet: 7
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Sailesh Mukil <sail...@cloudera.com>
Gerrit-Reviewer: Casey Ching <ca...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Matthew Jacobs <m...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-1694: Improve logging in the catalog

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-1694: Improve logging in the catalog
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/2620
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ib0cec4518a9a916c343e40d3d74e3498574816aa
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Dimitris Tsirogiannis <dtsirogian...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-1661: Netezza compatibility functions: strings

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-1661: Netezza compatibility functions: strings
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/20
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I3e90274e832e1564a4b981106dd80db1b5951995
Gerrit-PatchSet: 4
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Zuo Wang <zuo.a.w...@intel.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <mar...@cloudera.com>
Gerrit-Reviewer: Martin Grund <grundprin...@gmail.com>
Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-2761: OS X: Misc Fixes

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-2761: OS X: Misc Fixes
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/1754
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ifde44f9bb463061ecded32884af6942ddd3e208d
Gerrit-PatchSet: 4
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Martin Grund <grundprin...@gmail.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Martin Grund <grundprin...@gmail.com>
Gerrit-Reviewer: Skye Wanderman-Milne <s...@cloudera.com>

[Impala-CR](cdh5-trunk) Add testing utility to convert nested data to flat data

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: Add testing utility to convert nested data to flat data
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/390
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I407514245b327dbea0b0eea62677026ec9ca83d7
Gerrit-PatchSet: 5
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Casey Ching <ca...@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com>
Gerrit-Reviewer: Casey Ching <ca...@cloudera.com>
Gerrit-Reviewer: Dimitris Tsirogiannis <dtsirogian...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Taras Bobrovytsky <tbobrovyt...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-1862: Invalid value not reported as a scanner error

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-1862: Invalid value not reported as a scanner error
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/767
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ibc05c2e8bccbb99c9e41f78b2411fc6c929a54b7
Gerrit-PatchSet: 7
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Sailesh Mukil <sail...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Ippokratis Pandis <ipan...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Marcel Kornacker <mar...@cloudera.com>
Gerrit-Reviewer: Sailesh Mukil <sail...@cloudera.com>
Gerrit-Reviewer: Skye Wanderman-Milne <s...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-2717: impala-shell breaks on non-ascii chars in resultset

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-2717: impala-shell breaks on non-ascii chars in resultset
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/2041
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ic1625c70bd5d0690cd0cfacd27990bd1328aba5d
Gerrit-PatchSet: 3
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Marcell Szabo <sz...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>

[Impala-CR](cdh5-trunk) Add a synthetic query to stress the number of open hdfs connections.

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: Add a synthetic query to stress the number of open hdfs 
connections.
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/469
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: Ic8c4df67f22d14c6a834483dcab00724ea5a07a6
Gerrit-PatchSet: 1
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Ishaan Joshi <ish...@cloudera.com>
Gerrit-Reviewer: Alex Behm <alex.b...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Ishaan Joshi <ish...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>

[Impala-CR](cdh5-trunk) IMPALA-1766: Misc. statistical functions. Implemented aggregate corr().

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-1766: Misc. statistical functions. Implemented aggregate 
corr().
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/2534
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I7347c173ec4f80037d45dd463c17eb81ceef14a1
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Abdur Rafay <ara...@ucmerced.edu>
Gerrit-Reviewer: Abdur Rafay
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Skye Wanderman-Milne <s...@cloudera.com>

Re: [VOTE] Impala 2.7.0 release candidate 3

[Jim Apple]

Thanks again, Justin. One question:

> [8] may also have an issue with 2 owners was this part of the original 
> software grant or not?

I believe it probably wasn't but I'll check. What is the issue if
there are two owners?

> 8. ./bin/file2array.sh

Re: Contributions to Cloudera Impala

[Jim Apple]

Any updates on this, Nishidha?

On Fri, Sep 16, 2016 at 10:43 AM, Jim Apple <jbap...@cloudera.com> wrote:
> Nishidha, have you thought about starting that discussion on dev@ about
> branching?
>
> On Wed, Aug 17, 2016 at 9:12 AM, Jim Apple <jbap...@cloudera.com> wrote:
>>
>> > I'm glad to tell you that we are able to build and test Impala on Ubuntu
>> > linux ppc64le with the great support from the Cloudera Community.
>>
>> Excellent!
>>
>> > Our next action is to upstream all our changes to Cloudera Impala.
>>
>> Great!
>>
>> Cloudera has donated Impala to the Apache Software Foundation (aka
>> "ASF"). Cloudera now contributes to the project, and the project is
>> managed by the Impala community.
>>
>> > With
>> > this, our plan is to start building latest Impala on Power8 as we'd been
>> > porting quite an old version (code from cdh5-trunk branch till 23rd
>> > March,
>> > 2016). Since then, I know there have been many many changes happened
>> > which
>> > are yet to be ported, specially kudu stuff.
>>
>> Yes, there have been many changes. One is that Impala is now hosted on
>> ASF-owned git. Please see
>>
>> https://cwiki.apache.org/confluence/display/IMPALA/How+to+switch+to+Apache-hosted+git
>>
>> >We know we need CLA to be signed to start contributions. We have
>> > already
>> >initiated the process and hoping to get it done soon.
>>
>> I think the right thing to do here is use the Apache CLAs. See
>>
>> https://www.apache.org/licenses/cla-corporate.txt
>>
>> and
>>
>> https://www.apache.org/licenses/icla.txt
>>
>> > By the time we get CLA signed, we would start porting the changes
>> > done
>> >in last 5 months. So, I wanted to know which tag/branch should we
>> > take
>> >up for this.
>>
>> This is a question we could all discuss together, and it might end up
>> being a decision made by the Project Management Committee (PMC).
>>
>> This is a big question about how Apache Impala will evolve. Our bylaws
>> state:
>>
>> "Significant, pervasive features may be developed in a speculative
>> branch of the repository. The PMC may grant commit rights on the
>> branch to its consistent contributors for the duration of the
>> initiative. Branch committers are responsible for shepherding their
>> feature into an active release and do not cast binding votes or vetoes
>> in the project."
>>
>> So perhaps this should happen on a separate branch?
>>
>> One question the community should also consider, IMHO, is whether the
>> community will have sufficient resources to maintain a working ppc64le
>> codebase indefinitely into the future.
>>
>>
>> > Working on cdh5-trunk will put us into an unending loop of
>> >porting as it is being modified everyday. We are thinking to create a
>> >branch from cdh5.8.0-release tag and start working on it. Please
>> > suggest
>> >us the best way to do this.
>>
>> Since Impala is now developed on Apache infrastructure, we have
>> switched branching schemas. Our main branch is now "master". We do not
>> have any release branches yet.
>>
>> >Verifying all the changes on x86 platforms ourself here will also be
>> >time consuming and add potential delays in upstreaming. So, we were
>> >thinking if we can get a job on Cloudera's Continuous integration
>> > server
>> >which would simply fetch our branch and verify it on all the
>> > supported
>> >platforms and do all the required checks. I'm not sure if this is
>> >feasible but just a thought. Any other suggestions  to foster this
>> >activity would be appreciated.
>>
>> We are working on making a publicly-available CI setup, but we aren't done
>> yet.
>>
>> Do you have a CI setup and x86-64 machines that your CI workers can run
>> on?
>>
>> >For every Pull Request, what are the basic sanity tests required to
>> > be
>> >ensured? Do you test all BE, FE, End-to-End tests, Custom cluster
>> > tests?
>>
>> Patches are sent to gerrit for review. Before they are merged, all
>> tests must pass in "core" (but not "exhaustive") mode.
>>
>> ==
>>
>> If I were in your shoes, I might take the following steps:
>>
>> 1. Start a discussion on dev@ about whether a new branch is the right
>> way to develop.
>>
>> 2. Work out long-term maintenance plans and commitments and CI plans
>>
>> 3. Do the arduous work of rebasing on a recent HEAD.
>
>

Re: Cannot build with custom toolchain

[Jim Apple]

Martin noticed this, too:

https://issues.cloudera.org/browse/IMPALA-4067

On Thu, Sep 29, 2016 at 2:56 AM, Lars Volker  wrote:
> Hi,
>
> I tried to follow the instructions here:
> https://cwiki.apache.org/confluence/display/IMPALA/Building+native-toolchain+from+scratch+and+using+with+Impala
>
> However I keep running into the following error:
>
> SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping download of Python dependencies.
> SKIP_TOOLCHAIN_BOOTSTRAP is true, skipping toolchain bootstrap.
> cp: target
> ‘/home/lv/i4/toolchain/cdh_components/hadoop-2.6.0-cdh5.10.0-SNAPSHOT//lib/native’
> is not a directory
> Error in ./buildall.sh at line 281: cp
> "$HADOOP_LZO"/build/native/Linux-*-*/lib/libgplcompression.*
> "$HADOOP_HOME/lib/native"
>
> The problem seems to be that CDH component download is disabled by
> SKIP_TOOLCHAIN_DOWNLOAD, too.
>
> Is there a way around this?
>
> Thanks, Lars

Re: [RESULT] [VOTE] 2.7.0 release candidate 3

[Jim Apple]

You can follow the release vote in the IPMC by following the thread at
gene...@incubator.apache.org. You can either subscribe to that list by
sending mail to general-subscribe@ or you can follow it on the web at:

https://lists.apache.org/thread.html/ad34d0be66e23df70a8187c2db2c8b1d5ec56cce9c3e60321256663a@%3Cgeneral.incubator.apache.org%3E

Keep in mind that the release isn't done if that vote passes: I'll
still need to upload it to the right places, wait for the mirrors to
catch up, and do an announcement.

On Tue, Sep 27, 2016 at 8:12 AM, Jim Apple <jbap...@cloudera.com> wrote:
> The vote passed:
>
> +1 (binding):
>
> Tom White
> Tim Armstrong
> Matthew Jacobs
> Henry Robinson
> Juan Yu
>
> +1 (non-binding)
>
> Jim Apple
>
> I will now ask the Incubator PMC to allow us to release, so this
> release is not official yet.

[RESULT] [VOTE] 2.7.0 release candidate 3

[Jim Apple]

The vote passed:

+1 (binding):

Tom White
Tim Armstrong
Matthew Jacobs
Henry Robinson
Juan Yu

+1 (non-binding)

Jim Apple

I will now ask the Incubator PMC to allow us to release, so this
release is not official yet.

Re: does impala allow for secondary indices

[Jim Apple]

This was the first hit on Google for [impala indexes] for me:

http://www.cloudera.com/documentation/enterprise/latest/topics/impala_faq.html

It says "What features from relational databases or Hive are not
available in Impala? ... Indexing (not currently)."

On Mon, Sep 26, 2016 at 9:54 PM, amoners  wrote:
> Hi all,
>
> I want to know does impala allow for secondary indices?
>
>
> Thanks for your answer.
>
>
> Best Regards,
> Amoners
>
>

Re: impala start error. please help me to solve the troubles. thanks!

[Jim Apple]

Images are not forwarded by the apache mailing lists, as far as I can tell.
Please use plain text.

On Mon, Sep 26, 2016 at 7:50 PM, Linxiaoyong  wrote:

> Dear Guys:
>
>
>
> Recently we compile impala using our development environment and when we
> run the complied impala, we met the following problem.
>
>
>
> Problem: Impala runs successfully if we do not reboot our machine.
> However, when we reboot the machine, we cannot restart the impala process.
> We try a lot of machines, the problem occurs on every machine.
>
>
>
> We struggle for a long time , but it still does not work. We are wondering
> whether you guys can help us to solve the problem.
>
>
>
> The environment and error message is as follows.
>
>
>
> environment:
>
> OS: Distributor ID: CentOS
>
> Description:CentOS Linux release 7.2.1511 (Core)
>
> Release:7.2.1511
>
> Codename:   Core
>
> Kernel：Linux version 3.10.0-327.28.2.el7.x86_64
>
> Impala version: cdh5-trunk
>
>
>
> 1.   We start Impala using: ${IMPALA_HOME}/testdata/bin/run-all.sh,
> and get the following message.
>
>
>
> 2.   Vim cluster_logs/hbase/hbase-root-master-localhost.localdomain.
> out
>
> Errors follow as:
>
>
>
>
>
>
>
>
>
>
>
> I used “jps” to watch the processes like as:
>
>
>
>
>

Re: [VOTE] 2.7.0 release candidate 3

[Jim Apple]

OK, let's extend the voting to today. I have sent a clarification
patch to our bylaws about weekend voting:
https://gerrit.cloudera.org/#/c/4536/

Please vote today!

P.S. It looks like our other mentors, Todd, Brock, and Todd, are on
the Incubator PMC:
http://people.apache.org/committers-by-project.html#incubator-pmc


On Sat, Sep 24, 2016 at 1:43 AM, Tom White <t...@cloudera.com> wrote:
> 72 hours is a minimum, and it's customary not to count weekends, so you
> might let the vote run longer to get more binding votes.
>
> I'm on the IPMC so my vote counts as a binding one
>
> Tom
>
> On 24 Sep 2016 00:21, "Jim Apple" <jbap...@cloudera.com> wrote:
>
>> It's been 36 hours, so we're halfway through the vote. No binding
>> votes have been cast yet (assuming mentors are not implicitly part of
>> the PMC)
>>
>> On Fri, Sep 23, 2016 at 7:09 AM, Jim Apple <jbap...@cloudera.com> wrote:
>> > +1 (non-binding)
>> >
>> > I checked the sigs and checksums, ran RAT, checked for diffs from the
>> > source tag, and ran the exhaustive tests.
>> >
>> > On Fri, Sep 23, 2016 at 1:22 AM, Tom White <t...@cloudera.com> wrote:
>> >> +1
>> >>
>> >> I checked signature and checksums, ran RAT to check for license
>> >> headers, checked there are no binary files, verified the source tag
>> >> matches the tarball, inspected the LICENSE/NOTICE/DISCLAIMER files,
>> >> and built from source.
>> >>
>> >> BTW I hit https://issues.cloudera.org/browse/IMPALA-3979, since it has
>> >> not been applied to the 2.7.0 branch. There's an easy workaround
>> >> though, so not a blocker.
>> >>
>> >> Cheers,
>> >> Tom
>> >>
>> >> On Thu, Sep 22, 2016 at 12:17 PM, Jim Apple <jbap...@cloudera.com>
>> wrote:
>> >>> This vote is about whether to release Apache Impala (incubating) 2.7.0
>> >>> release candidate 3.
>> >>>
>> >>> The artifacts for testing are here:
>> >>> https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/RC3/
>> >>>
>> >>> They correspond to the git tag 2.7.0-rc3, tree
>> >>> b93b91f67321d75754185d58c9624f5806690099
>> >>>
>> >>> https://git-wip-us.apache.org/repos/asf?p=incubator-impala.
>> git;a=tree;h=b93b91f67321d75754185d58c9624f5806690099
>> >>>
>> >>> This vote will remain open for 72 hours.
>> >>>
>> >>> Anyone can vote, but only the votes of PMC members are binding.
>> >>> Available votes are +1 and -1. If there are three binding +1 votes and
>> >>> more binding +1 votes then binding -1 votes, the vote passes. If the
>> >>> vote passes, I will initiate a vote on this release candidate with the
>> >>> Incubator PMC.
>> >>>
>> >>> A +1 vote means "I have downloaded the signed source code package,
>> >>> compiled it as provided, and tested the resulting executable on my own
>> >>> platform, along with also verifying that the package meets the
>> >>> requirements of the ASF policy on releases."
>> >>>
>> >>> The ASF policy on releases is here: http://www.apache.org/dev/
>> release.html
>> >>>
>> >>> Special extra rules for Incubator releases are here:
>> >>> http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
>> >>>
>> >>> -1 votes should be accompanied by a reason.
>> >>>
>> >>> After a vote, the voter should include "(binding)" or "(non-binding)"
>> >>> indicating their status as PMC members or PMC non-members. Example:
>> >>> "+1 (non-binding)"
>> >>>
>> >>> Instructions on downloading and checking the release candidate are
>> >>> available at https://cwiki.apache.org/confluence/display/IMPALA/
>> DRAFT%3A+How+to+Release
>>

Re: [VOTE] 2.7.0 release candidate 3

[Jim Apple]

It's been 36 hours, so we're halfway through the vote. No binding
votes have been cast yet (assuming mentors are not implicitly part of
the PMC)

On Fri, Sep 23, 2016 at 7:09 AM, Jim Apple <jbap...@cloudera.com> wrote:
> +1 (non-binding)
>
> I checked the sigs and checksums, ran RAT, checked for diffs from the
> source tag, and ran the exhaustive tests.
>
> On Fri, Sep 23, 2016 at 1:22 AM, Tom White <t...@cloudera.com> wrote:
>> +1
>>
>> I checked signature and checksums, ran RAT to check for license
>> headers, checked there are no binary files, verified the source tag
>> matches the tarball, inspected the LICENSE/NOTICE/DISCLAIMER files,
>> and built from source.
>>
>> BTW I hit https://issues.cloudera.org/browse/IMPALA-3979, since it has
>> not been applied to the 2.7.0 branch. There's an easy workaround
>> though, so not a blocker.
>>
>> Cheers,
>> Tom
>>
>> On Thu, Sep 22, 2016 at 12:17 PM, Jim Apple <jbap...@cloudera.com> wrote:
>>> This vote is about whether to release Apache Impala (incubating) 2.7.0
>>> release candidate 3.
>>>
>>> The artifacts for testing are here:
>>> https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/RC3/
>>>
>>> They correspond to the git tag 2.7.0-rc3, tree
>>> b93b91f67321d75754185d58c9624f5806690099
>>>
>>> https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tree;h=b93b91f67321d75754185d58c9624f5806690099
>>>
>>> This vote will remain open for 72 hours.
>>>
>>> Anyone can vote, but only the votes of PMC members are binding.
>>> Available votes are +1 and -1. If there are three binding +1 votes and
>>> more binding +1 votes then binding -1 votes, the vote passes. If the
>>> vote passes, I will initiate a vote on this release candidate with the
>>> Incubator PMC.
>>>
>>> A +1 vote means "I have downloaded the signed source code package,
>>> compiled it as provided, and tested the resulting executable on my own
>>> platform, along with also verifying that the package meets the
>>> requirements of the ASF policy on releases."
>>>
>>> The ASF policy on releases is here: http://www.apache.org/dev/release.html
>>>
>>> Special extra rules for Incubator releases are here:
>>> http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
>>>
>>> -1 votes should be accompanied by a reason.
>>>
>>> After a vote, the voter should include "(binding)" or "(non-binding)"
>>> indicating their status as PMC members or PMC non-members. Example:
>>> "+1 (non-binding)"
>>>
>>> Instructions on downloading and checking the release candidate are
>>> available at 
>>> https://cwiki.apache.org/confluence/display/IMPALA/DRAFT%3A+How+to+Release

Re: [VOTE] 2.7.0 release candidate 3

[Jim Apple]

+1 (non-binding)

I checked the sigs and checksums, ran RAT, checked for diffs from the
source tag, and ran the exhaustive tests.

On Fri, Sep 23, 2016 at 1:22 AM, Tom White <t...@cloudera.com> wrote:
> +1
>
> I checked signature and checksums, ran RAT to check for license
> headers, checked there are no binary files, verified the source tag
> matches the tarball, inspected the LICENSE/NOTICE/DISCLAIMER files,
> and built from source.
>
> BTW I hit https://issues.cloudera.org/browse/IMPALA-3979, since it has
> not been applied to the 2.7.0 branch. There's an easy workaround
> though, so not a blocker.
>
> Cheers,
> Tom
>
> On Thu, Sep 22, 2016 at 12:17 PM, Jim Apple <jbap...@cloudera.com> wrote:
>> This vote is about whether to release Apache Impala (incubating) 2.7.0
>> release candidate 3.
>>
>> The artifacts for testing are here:
>> https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/RC3/
>>
>> They correspond to the git tag 2.7.0-rc3, tree
>> b93b91f67321d75754185d58c9624f5806690099
>>
>> https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tree;h=b93b91f67321d75754185d58c9624f5806690099
>>
>> This vote will remain open for 72 hours.
>>
>> Anyone can vote, but only the votes of PMC members are binding.
>> Available votes are +1 and -1. If there are three binding +1 votes and
>> more binding +1 votes then binding -1 votes, the vote passes. If the
>> vote passes, I will initiate a vote on this release candidate with the
>> Incubator PMC.
>>
>> A +1 vote means "I have downloaded the signed source code package,
>> compiled it as provided, and tested the resulting executable on my own
>> platform, along with also verifying that the package meets the
>> requirements of the ASF policy on releases."
>>
>> The ASF policy on releases is here: http://www.apache.org/dev/release.html
>>
>> Special extra rules for Incubator releases are here:
>> http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
>>
>> -1 votes should be accompanied by a reason.
>>
>> After a vote, the voter should include "(binding)" or "(non-binding)"
>> indicating their status as PMC members or PMC non-members. Example:
>> "+1 (non-binding)"
>>
>> Instructions on downloading and checking the release candidate are
>> available at 
>> https://cwiki.apache.org/confluence/display/IMPALA/DRAFT%3A+How+to+Release

[VOTE] 2.7.0 release candidate 3

[Jim Apple]

This vote is about whether to release Apache Impala (incubating) 2.7.0
release candidate 3.

The artifacts for testing are here:
https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/RC3/

They correspond to the git tag 2.7.0-rc3, tree
b93b91f67321d75754185d58c9624f5806690099

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tree;h=b93b91f67321d75754185d58c9624f5806690099

This vote will remain open for 72 hours.

Anyone can vote, but only the votes of PMC members are binding.
Available votes are +1 and -1. If there are three binding +1 votes and
more binding +1 votes then binding -1 votes, the vote passes. If the
vote passes, I will initiate a vote on this release candidate with the
Incubator PMC.

A +1 vote means "I have downloaded the signed source code package,
compiled it as provided, and tested the resulting executable on my own
platform, along with also verifying that the package meets the
requirements of the ASF policy on releases."

The ASF policy on releases is here: http://www.apache.org/dev/release.html

Special extra rules for Incubator releases are here:
http://incubator.apache.org/incubation/Incubation_Policy.html#Releases

-1 votes should be accompanied by a reason.

After a vote, the voter should include "(binding)" or "(non-binding)"
indicating their status as PMC members or PMC non-members. Example:
"+1 (non-binding)"

Instructions on downloading and checking the release candidate are
available at 
https://cwiki.apache.org/confluence/display/IMPALA/DRAFT%3A+How+to+Release

Re: Issues with Apache Impala (incubating) website

[Jim Apple]

> Almost. It really would be better to spell it explicitly (and avoid a phrase
> Cloudera's Hadoop distribution -- since this is something ASF is not
> looking too kindly on as per http://www.apache.org/foundation/marks/)
>
> I'd suggest something along the lines:
>"Documentation for previous versions of Impala in CDH"

Done.

Re: Issues with Apache Impala (incubating) website

[Jim Apple]

Roman, thank you for taking the time to write to us.

Oddly, just as you were sending this, I was resolving

https://issues.cloudera.org/browse/IMPALA-4068

with this commit

https://github.com/apache/incubator-impala/commit/90ffefd1140e3fcfbbdc26496a7e177eb752ae1c

It removes the links to download, install, try as VM, try as Docker, and
try in cloud.

We have started migrating our docs to ASF, though John and Laurel (cced)
have not finished yet. Their organizing JIRA is here:

https://issues.cloudera.org/browse/IMPALA-3398

In the meantime, the commit above changed the docs link to be a page about
available documentation. This page includes a link to the Cloudera docs,
but it calls them out as such. Does that meet the guidelines, in your
opinion? The guidelines do allow "links to external documentation".

As far as the docs saying "Apache Impala (incubating)", I have cc:ed David
Middler, a lawyer for Cloudera. I am not a lawyer and I do not quite
understand what words one should use to refer to this Impala thing between
the TM transfer and the first release - "Cloudera Impala"? just "Impala"?

The patch I linked to above also changes the github link to point to the
Apache github repo for Impala.

I think that touches on each of the issues you raised.

Thanks again,
Jim


On Tue, Sep 20, 2016 at 3:43 PM, Roman Shaposhnik  wrote:

> Hi!
>
> while reviewing Impala website
> http://impala.incubator.apache.org/
>
> I've come across a few issues that need
> to be corrected soon in order to get the site
> in line with ASF branding guidelines for podling
> websites:
> http://incubator.apache.org/guides/sites.html
>
> First of all, since your community hasn't had a single
> release yet, you need to get rid of
>Dowload (at the top)
>Install
>Try as VM
>Try as Docker Image
>Try in the Cloud buttons
> since they all point to something that clearly has no
> relationship to Apache Impala (incubating) and is
> very confusing to the users who may assume that
> these are links to official releases.
>
> I also suggest that you start migrating your docs to
> ASF and get rid of Docs button linking to cloudera.com
> published Impala Docs. If you click on that Docs link
> it says (at the top of the page)
>This is the documentation for Cloudera Enterprise 5.8.x.
> and then it proceeds to talk about Apache Impala (incubating).
> This is misleading for the same reasons -- there's no such
> thing as publicly available  Apache Impala (incubating)
> until you have your first release.
>
> Contribute and Git Hub buttons at the top still point to
> Cloudera's GitHub. This has to change ASAP.
>
> Please CC me directly if you have any further questions. I'm not
> subscribed to this mailing list.
>
> Thanks,
> Roman.
>

Re: Please evaluate 2.7.0rc2

[Jim Apple]

WFM. I'll fix this JAR thing (https://issues.cloudera.org/browse/IMPALA-4171)
then roll another RC. That one I'll take a vote on, and I'll change the
wiki page with release instructions to say that the RM can jump into a vote
after his or her own testing and can always roll another RC and take
another vote even if one RC has passed or is about to pass.

Everyone else: please continue testing and verifying so we can fix as much
stuff in RC3 as possible; thank you!

Thanks, Tom and Brock!

On Tue, Sep 20, 2016 at 12:44 PM, Brock Noland <br...@apache.org> wrote:

> In this case, as RM, I just role another RC. Done that many times myself.
>
> AFAIK there isn't anything that "forces" you to release and that seems
> contrary to everything I've been involved with at the ASF.
>
> On Tue, Sep 20, 2016 at 12:30 PM, Jim Apple <jbap...@cloudera.com> wrote:
>
> > I wanted to leave in our release procedures the opportunity for the
> release
> > manager to NOT release if there are three +1s but the RM finds something
> he
> > or she wants to fix first. For instance, what if we had had three +1s
> > already before you found that JAR? Would the RM have to release?
> >
> > I don't know, so I added in the extra step.
> >
> > On Tue, Sep 20, 2016 at 10:08 AM, Tom White <t...@cloudera.com> wrote:
> >
> > > Any reason this isn't a vote? Usually an RC will be voted on, and if
> > > problems are found then another RC will be created, until the PPMC is
> > > happy with it, at which point it can go to the IPMC.
> > >
> > > I ran through the legal checks and it looks good to me. I noticed that
> > > there's a binary file (testdata/udfs/impala-hive-udfs.jar), which
> > > should really be removed.
> > >
> > > Cheers,
> > > Tom
> > >
> > >
> > > On Sun, Sep 18, 2016 at 1:15 AM, Jim Apple <jbap...@cloudera.com>
> wrote:
> > > > This is the second release candidate for Apache Impala (incubating)
> > > 2.7.0:
> > > >
> > > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/RC2/
> > > >
> > > > The purpose of this thread is to gather feedback on the release
> > > > candidate to see if a third release candidate is needed or if we can
> > > > go ahead and vote on rc2.
> > > >
> > > > The git tag of the tree I made the tarball from is "2.7.0-rc2":
> > > >
> > > > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.
> > git;a=tag;h=
> > > cabaa58fa5c15b0457c520a658dc74b4d174b1c7
> > > >
> > > > That is commit:
> > > >
> > > > 53439d3d8cdca2dd1ca2cdf2c36c6a7ef101f7e3
> > > >
> > > > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.
> > > git;a=commit;h=53439d3d8cdca2dd1ca2cdf2c36c6a7ef101f7e3
> > > >
> > > > and this tree:
> > > >
> > > > 0b8e8a3f48b8086d53ee96064668dfccd56d57cc
> > > >
> > > > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.
> > git;a=tree;h=
> > > 0b8e8a3f48b8086d53ee96064668dfccd56d57cc;hb=
> > 53439d3d8cdca2dd1ca2cdf2c36c6a
> > > 7ef101f7e3
> > > >
> > > > --
> > > >
> > > > You can find instructions for how to evaluate a release candidate
> here:
> > > >
> > > > https://cwiki.apache.org/confluence/display/IMPALA/
> > > DRAFT%3A+How+to+Release#DRAFT:HowtoRelease-HowtoVoteonaRelea
> seCandidate
> > > >
> > > > Here is a copy of that section of the wiki:
> > > >
> > > > 1. Download the tarball. Check the signature and the checksums.
> > > >
> > > > # change to a new directory
> > > > cd $(mktemp -d)
> > > >
> > > > # Download the keys of the release managers
> > > > wget https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
> > > > gpg --import KEYS
> > > >
> > > > # Set the keys of the release managers as trusted
> > > > gpg --edit-key jbapple trust
> > > > # At the prompt, enter '5' for "I trust ultimately", then 'y' for
> > > > "yes", then 'q' for "quit"
> > > >
> > > > # Download the release artifacts:
> > > > wget https://dist.apache.org/repos/dist/dev/incubator/impala/x.y.
> > > z/RCq/apache-impala-incubating-x.y.z.tar.{gz,gz.asc,gz.md5,gz.sha}
> > > >
> > > > 

Re: Please evaluate 2.7.0rc2

[Jim Apple]

I wanted to leave in our release procedures the opportunity for the release
manager to NOT release if there are three +1s but the RM finds something he
or she wants to fix first. For instance, what if we had had three +1s
already before you found that JAR? Would the RM have to release?

I don't know, so I added in the extra step.

On Tue, Sep 20, 2016 at 10:08 AM, Tom White <t...@cloudera.com> wrote:

> Any reason this isn't a vote? Usually an RC will be voted on, and if
> problems are found then another RC will be created, until the PPMC is
> happy with it, at which point it can go to the IPMC.
>
> I ran through the legal checks and it looks good to me. I noticed that
> there's a binary file (testdata/udfs/impala-hive-udfs.jar), which
> should really be removed.
>
> Cheers,
> Tom
>
>
> On Sun, Sep 18, 2016 at 1:15 AM, Jim Apple <jbap...@cloudera.com> wrote:
> > This is the second release candidate for Apache Impala (incubating)
> 2.7.0:
> >
> > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/RC2/
> >
> > The purpose of this thread is to gather feedback on the release
> > candidate to see if a third release candidate is needed or if we can
> > go ahead and vote on rc2.
> >
> > The git tag of the tree I made the tarball from is "2.7.0-rc2":
> >
> > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tag;h=
> cabaa58fa5c15b0457c520a658dc74b4d174b1c7
> >
> > That is commit:
> >
> > 53439d3d8cdca2dd1ca2cdf2c36c6a7ef101f7e3
> >
> > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.
> git;a=commit;h=53439d3d8cdca2dd1ca2cdf2c36c6a7ef101f7e3
> >
> > and this tree:
> >
> > 0b8e8a3f48b8086d53ee96064668dfccd56d57cc
> >
> > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tree;h=
> 0b8e8a3f48b8086d53ee96064668dfccd56d57cc;hb=53439d3d8cdca2dd1ca2cdf2c36c6a
> 7ef101f7e3
> >
> > --
> >
> > You can find instructions for how to evaluate a release candidate here:
> >
> > https://cwiki.apache.org/confluence/display/IMPALA/
> DRAFT%3A+How+to+Release#DRAFT:HowtoRelease-HowtoVoteonaReleaseCandidate
> >
> > Here is a copy of that section of the wiki:
> >
> > 1. Download the tarball. Check the signature and the checksums.
> >
> > # change to a new directory
> > cd $(mktemp -d)
> >
> > # Download the keys of the release managers
> > wget https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
> > gpg --import KEYS
> >
> > # Set the keys of the release managers as trusted
> > gpg --edit-key jbapple trust
> > # At the prompt, enter '5' for "I trust ultimately", then 'y' for
> > "yes", then 'q' for "quit"
> >
> > # Download the release artifacts:
> > wget https://dist.apache.org/repos/dist/dev/incubator/impala/x.y.
> z/RCq/apache-impala-incubating-x.y.z.tar.{gz,gz.asc,gz.md5,gz.sha}
> >
> > # Check the checksums:
> > md5sum --check apache-impala-incubating-x.y.z.tar.gz.md5
> > sha1sum --check apache-impala-incubating-x.y.z.tar.gz.sha
> >
> > # Check the signature:
> > gpg --verify apache-impala-incubating-x.y.z.tar.gz.asc
> >
> > 2. Check that it matches the upstream tag
> > # move to your git directory and checkout the tag:
> > cd incubator-impala
> > git fetch apache --tags
> > git checkout x.y.z-rcw
> >
> > # compare the tarball and the repo:
> > cd ..
> > tar xzf apache-impala-incubating-x.y.z.tar.gz
> > diff -r apache-impala-incubating-x.y.z incubator-impala
> > # You should see something like "Only in incubator-impala: .git", but
> > no other output
> >
> > 3. Test the release quality, possibly using bin/run-all-tests.py.
> >
> > 4. Check compliance with ASF release policy. Use Apache RAT and follow
> > the instructions in bin/check-rat-report.py to check licence
> > compliance.
> >
> > 
> --
> >
> > Changes since the last release candidate:
> >
> > 1. Minor changes in the way the release candidate was made (directory
> > name in tarball, location on dist.apache.org, filename). The current
> > procedure is documented on
> > https://cwiki.apache.org/confluence/pages/viewpage.
> action?pageId=65869538
> >
> > 2. Un-break building from non-git directories:
> >
> > https://git-wip-us.apache.org/repos/asf?p=incubator-impala.
> git;a=commit;h=9f08d1ab3c876dc1fc92c9decb8104400eaeec7c
>

[Impala-CR](cdh5-trunk) IMPALA-889: Add support for ISO-SQL trim()

[Jim Apple (Code Review)]

Jim Apple has abandoned this change.

Change subject: IMPALA-889: Add support for ISO-SQL trim()
..


Abandoned

-- 
To view, visit http://gerrit.cloudera.org:8080/3213
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: abandon
Gerrit-Change-Id: I4753c608b0b00569bf8c5e95b132df6df358e602
Gerrit-PatchSet: 10
Gerrit-Project: Impala
Gerrit-Branch: cdh5-trunk
Gerrit-Owner: Youwei Wang <youwei.a.w...@intel.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Youwei Wang <youwei.a.w...@intel.com>

Please evaluate 2.7.0rc2

[Jim Apple]

This is the second release candidate for Apache Impala (incubating) 2.7.0:

https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/RC2/

The purpose of this thread is to gather feedback on the release
candidate to see if a third release candidate is needed or if we can
go ahead and vote on rc2.

The git tag of the tree I made the tarball from is "2.7.0-rc2":

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tag;h=cabaa58fa5c15b0457c520a658dc74b4d174b1c7

That is commit:

53439d3d8cdca2dd1ca2cdf2c36c6a7ef101f7e3

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=53439d3d8cdca2dd1ca2cdf2c36c6a7ef101f7e3

and this tree:

0b8e8a3f48b8086d53ee96064668dfccd56d57cc

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tree;h=0b8e8a3f48b8086d53ee96064668dfccd56d57cc;hb=53439d3d8cdca2dd1ca2cdf2c36c6a7ef101f7e3

--

You can find instructions for how to evaluate a release candidate here:

https://cwiki.apache.org/confluence/display/IMPALA/DRAFT%3A+How+to+Release#DRAFT:HowtoRelease-HowtoVoteonaReleaseCandidate

Here is a copy of that section of the wiki:

1. Download the tarball. Check the signature and the checksums.

# change to a new directory
cd $(mktemp -d)

# Download the keys of the release managers
wget https://dist.apache.org/repos/dist/dev/incubator/impala/KEYS
gpg --import KEYS

# Set the keys of the release managers as trusted
gpg --edit-key jbapple trust
# At the prompt, enter '5' for "I trust ultimately", then 'y' for
"yes", then 'q' for "quit"

# Download the release artifacts:
wget 
https://dist.apache.org/repos/dist/dev/incubator/impala/x.y.z/RCq/apache-impala-incubating-x.y.z.tar.{gz,gz.asc,gz.md5,gz.sha}

# Check the checksums:
md5sum --check apache-impala-incubating-x.y.z.tar.gz.md5
sha1sum --check apache-impala-incubating-x.y.z.tar.gz.sha

# Check the signature:
gpg --verify apache-impala-incubating-x.y.z.tar.gz.asc

2. Check that it matches the upstream tag
# move to your git directory and checkout the tag:
cd incubator-impala
git fetch apache --tags
git checkout x.y.z-rcw

# compare the tarball and the repo:
cd ..
tar xzf apache-impala-incubating-x.y.z.tar.gz
diff -r apache-impala-incubating-x.y.z incubator-impala
# You should see something like "Only in incubator-impala: .git", but
no other output

3. Test the release quality, possibly using bin/run-all-tests.py.

4. Check compliance with ASF release policy. Use Apache RAT and follow
the instructions in bin/check-rat-report.py to check licence
compliance.

--

Changes since the last release candidate:

1. Minor changes in the way the release candidate was made (directory
name in tarball, location on dist.apache.org, filename). The current
procedure is documented on
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65869538

2. Un-break building from non-git directories:

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=9f08d1ab3c876dc1fc92c9decb8104400eaeec7c

3. RAT script to check licensing:

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=85a0e36423c6c938c1a540629c16d27b3a9e1522

4. License fixes:

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=586ae2d7f357d1b2fd0d7b59dad0229e7383747d

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=d385ac72d1400a6c1eb5da6de6f8c28aba33a3d4

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=fb646d1b06a21f44518616bd49a91b88998ff602

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=53439d3d8cdca2dd1ca2cdf2c36c6a7ef101f7e3

5. Remove 'cdh' from version string:

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=046267b72e5b63b134a4ef004ccde2a268a9ae42

Re: Contributions to Cloudera Impala

[Jim Apple]

Nishidha, have you thought about starting that discussion on dev@ about
branching?

On Wed, Aug 17, 2016 at 9:12 AM, Jim Apple <jbap...@cloudera.com> wrote:

> > I'm glad to tell you that we are able to build and test Impala on Ubuntu
> > linux ppc64le with the great support from the Cloudera Community.
>
> Excellent!
>
> > Our next action is to upstream all our changes to Cloudera Impala.
>
> Great!
>
> Cloudera has donated Impala to the Apache Software Foundation (aka
> "ASF"). Cloudera now contributes to the project, and the project is
> managed by the Impala community.
>
> > With
> > this, our plan is to start building latest Impala on Power8 as we'd been
> > porting quite an old version (code from cdh5-trunk branch till 23rd
> March,
> > 2016). Since then, I know there have been many many changes happened
> which
> > are yet to be ported, specially kudu stuff.
>
> Yes, there have been many changes. One is that Impala is now hosted on
> ASF-owned git. Please see
> https://cwiki.apache.org/confluence/display/IMPALA/How+
> to+switch+to+Apache-hosted+git
>
> >We know we need CLA to be signed to start contributions. We have
> already
> >initiated the process and hoping to get it done soon.
>
> I think the right thing to do here is use the Apache CLAs. See
>
> https://www.apache.org/licenses/cla-corporate.txt
>
> and
>
> https://www.apache.org/licenses/icla.txt
>
> > By the time we get CLA signed, we would start porting the changes
> done
> >in last 5 months. So, I wanted to know which tag/branch should we take
> >up for this.
>
> This is a question we could all discuss together, and it might end up
> being a decision made by the Project Management Committee (PMC).
>
> This is a big question about how Apache Impala will evolve. Our bylaws
> state:
>
> "Significant, pervasive features may be developed in a speculative
> branch of the repository. The PMC may grant commit rights on the
> branch to its consistent contributors for the duration of the
> initiative. Branch committers are responsible for shepherding their
> feature into an active release and do not cast binding votes or vetoes
> in the project."
>
> So perhaps this should happen on a separate branch?
>
> One question the community should also consider, IMHO, is whether the
> community will have sufficient resources to maintain a working ppc64le
> codebase indefinitely into the future.
>
>
> > Working on cdh5-trunk will put us into an unending loop of
> >porting as it is being modified everyday. We are thinking to create a
> >branch from cdh5.8.0-release tag and start working on it. Please
> suggest
> >us the best way to do this.
>
> Since Impala is now developed on Apache infrastructure, we have
> switched branching schemas. Our main branch is now "master". We do not
> have any release branches yet.
>
> >Verifying all the changes on x86 platforms ourself here will also be
> >time consuming and add potential delays in upstreaming. So, we were
> >thinking if we can get a job on Cloudera's Continuous integration
> server
> >which would simply fetch our branch and verify it on all the supported
> >platforms and do all the required checks. I'm not sure if this is
> >feasible but just a thought. Any other suggestions  to foster this
> >activity would be appreciated.
>
> We are working on making a publicly-available CI setup, but we aren't done
> yet.
>
> Do you have a CI setup and x86-64 machines that your CI workers can run on?
>
> >For every Pull Request, what are the basic sanity tests required to be
> >ensured? Do you test all BE, FE, End-to-End tests, Custom cluster
> tests?
>
> Patches are sent to gerrit for review. Before they are merged, all
> tests must pass in "core" (but not "exhaustive") mode.
>
> ==
>
> If I were in your shoes, I might take the following steps:
>
> 1. Start a discussion on dev@ about whether a new branch is the right
> way to develop.
>
> 2. Work out long-term maintenance plans and commitments and CI plans
>
> 3. Do the arduous work of rebasing on a recent HEAD.
>

ASF-hosted gerrit

[Jim Apple]

The ASF doesn't host any gerrit servers right now. I have asked for one and
re-opened

https://issues.apache.org/jira/browse/INFRA-2205

I think it's not likely to happen soon.

Re: 2.7.0 release blocker: testdata/data/mstr origin

[Jim Apple]

OK, I will remove it in my RAT cleanup patch once I make sure the removal
passes when I do the rest of the RAT clean, too:

https://gerrit.cloudera.org/#/c/4361/

That will take a few hours for the test to complete.

On Mon, Sep 12, 2016 at 4:06 PM, Alex Behm <alex.b...@cloudera.com> wrote:

> Let's remove it then!
>
> On Mon, Sep 12, 2016 at 3:24 PM, Jim Apple <jbap...@cloudera.com> wrote:
>
> > The test is complete; removing that directory did not cause any tests to
> > fail.
> >
> > On Mon, Sep 12, 2016 at 11:46 AM, Jim Apple <jbap...@cloudera.com>
> wrote:
> >
> > > I'm running a test now to see if we can just delete that whole
> directory.
> > > Full tests take a few hours, so I'll post when I have more info.
> > >
> > > 'mstr' doesn't seem to appear as a meaningful string in our repo, based
> > on
> > > a cursory grep.
> > >
> > > On Mon, Sep 12, 2016 at 10:53 AM, Todd Lipcon <t...@cloudera.com>
> wrote:
> > >
> > >> +Carl Steinbach since it seems he was the original author (and not
> sure
> > if
> > >> he pays attention to dev@)
> > >>
> > >> On Mon, Sep 12, 2016 at 10:49 AM, Jim Apple <jbap...@cloudera.com>
> > wrote:
> > >>
> > >> > In order to do a 2.7.0 release, we must ensure that everything in
> the
> > >> > release is OK with http://www.apache.org/legal/resolved.html
> > >> >
> > >> > I am using Apache RAT to do that. It is flagging the files in
> > >> > testdata/data/mstr. Those files do not have a very informative git
> log
> > >> > history:
> > >> >
> > >> > https://github.com/apache/incubator-impala/commits/
> > >> > master/testdata/data/mstr
> > >> >
> > >> > Can anyone vouch for where they came from? This is a 2.7.0 release
> > >> blocker.
> > >> >
> > >> > Thanks,
> > >> > Jim
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Todd Lipcon
> > >> Software Engineer, Cloudera
> > >>
> > >
> > >
> >
>

Re: 2.7.0 release blocker: testdata/data/mstr origin

[Jim Apple]

The test is complete; removing that directory did not cause any tests to
fail.

On Mon, Sep 12, 2016 at 11:46 AM, Jim Apple <jbap...@cloudera.com> wrote:

> I'm running a test now to see if we can just delete that whole directory.
> Full tests take a few hours, so I'll post when I have more info.
>
> 'mstr' doesn't seem to appear as a meaningful string in our repo, based on
> a cursory grep.
>
> On Mon, Sep 12, 2016 at 10:53 AM, Todd Lipcon <t...@cloudera.com> wrote:
>
>> +Carl Steinbach since it seems he was the original author (and not sure if
>> he pays attention to dev@)
>>
>> On Mon, Sep 12, 2016 at 10:49 AM, Jim Apple <jbap...@cloudera.com> wrote:
>>
>> > In order to do a 2.7.0 release, we must ensure that everything in the
>> > release is OK with http://www.apache.org/legal/resolved.html
>> >
>> > I am using Apache RAT to do that. It is flagging the files in
>> > testdata/data/mstr. Those files do not have a very informative git log
>> > history:
>> >
>> > https://github.com/apache/incubator-impala/commits/
>> > master/testdata/data/mstr
>> >
>> > Can anyone vouch for where they came from? This is a 2.7.0 release
>> blocker.
>> >
>> > Thanks,
>> > Jim
>> >
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>
>

Re: Prepare to vote on release candidate 1

[Jim Apple]

Just a quick update with where I am with your RC1 feedback. First, let me
thank you for all of the feedback - it has been great for Impala, in my
opinion.

Here is a list of things that I am still planning to do for RC2:

1. Change the tarball name to remove the '-rc1'; instead, I will put the
next tarball, checksums, and signature into an 'rc2' directory.

2. Change the tarball so it unpacks into apache-impala-incubating-2.7.0.

3. Fix some 'Cloudera' licensing: pending at
https://gerrit.cloudera.org/#/c/4386/

4. Make buildall.sh work when not in a git repo: CI verification happening
at https://gerrit.cloudera.org/#/c/4336/

5. Version numbering: Apparently it is a problem for Cloudera downstream if
the version doesn't have the string 'cdh5'. If they (with my Apache hat on)
/ we (with my Cloudera hat on) can't fix this ASAP, I will change the
version string in the 2.7.0 branch (though not on master) and let Cloudera
deal with whatever Cloudera problems there are.

6. Git tag (and provide a git SHA for) the next RC

7. Get a clean RAT run: in progress at
https://gerrit.cloudera.org/#/c/4361/2 . This may take a while as I sort
out the testdata/data/mstr and /tests/comparison/leopard directories.

On Fri, Sep 9, 2016 at 11:52 AM, Jim Apple <jbap...@cloudera.com> wrote:

> >> - would be nice if the version number didn't have 'cdh5' in it (eg
> >> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
> >> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
> >
> > Lars is working on this now:
> >
> > https://gerrit.cloudera.org/#/c/4187/3
> >
> > Some possibilities for 2.7.0:
> >
> > 1. Leave it. Pros: expedient; Cons: inaccurate, not Apache
> >
> > 2. Wait until we can change this is master without confusing existing
> > tools. Pros: clean; Cons: delay in releasing
> >
> > 3. Make a commit just to branch-2.7.0. Pros: Doesn't confuse existing
> > tools; Cons: Makes branch-2.7.0 not a pure subset of master.
> >
> > What does everyone think?
>
> Just a reminder that your input on this is valued. I slightly prefer #3.
>

Re: 2.7.0 release blocker: testdata/data/mstr origin

[Jim Apple]

I'm running a test now to see if we can just delete that whole directory.
Full tests take a few hours, so I'll post when I have more info.

'mstr' doesn't seem to appear as a meaningful string in our repo, based on
a cursory grep.

On Mon, Sep 12, 2016 at 10:53 AM, Todd Lipcon <t...@cloudera.com> wrote:

> +Carl Steinbach since it seems he was the original author (and not sure if
> he pays attention to dev@)
>
> On Mon, Sep 12, 2016 at 10:49 AM, Jim Apple <jbap...@cloudera.com> wrote:
>
> > In order to do a 2.7.0 release, we must ensure that everything in the
> > release is OK with http://www.apache.org/legal/resolved.html
> >
> > I am using Apache RAT to do that. It is flagging the files in
> > testdata/data/mstr. Those files do not have a very informative git log
> > history:
> >
> > https://github.com/apache/incubator-impala/commits/
> > master/testdata/data/mstr
> >
> > Can anyone vouch for where they came from? This is a 2.7.0 release
> blocker.
> >
> > Thanks,
> > Jim
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: Config file comments generated by GPL tools

[Jim Apple]

http://www.apache.org/legal/resolved.html#can-we-use-doxygen-generated-config-files

Seems to resolve it for now. I'll strip the doxyfile of all those helpful
comments. Maybe one day this will be resolved by the doxygen authors.

Thanks for your help, Todd and Ryan!

On Mon, Sep 12, 2016 at 10:58 AM, Ryan Blue <rb...@netflix.com.invalid>
wrote:

> We had the same issue last year when we audited Avro's license
> documentation. This is tracked at LEGAL-224 [1] and we did reach out to
> doxygen [2]. The doxygen developer, Dimitri clarified that he doesn't
> intend for the doxy config files to be GPL, but hasn't clarified the
> license to my knowledge. In the end, we created a new config file with all
> of the non-default settings and none of the nice descriptions you get with
> the generated file.
>
> rb
>
>
> [1]: https://issues.apache.org/jira/browse/LEGAL-224
> [2]: https://bugzilla.gnome.org/show_bug.cgi?id=755135
>
> On Mon, Sep 12, 2016 at 10:52 AM, Todd Lipcon <t...@cloudera.com> wrote:
>
> > Maybe it's worth reaching out to the doxygen authors and ask them to add
> a
> > header on that file saying that the prose in the documentation may be
> > licensed under a different more permissive license? (e.g MIT/BSD?)
> >
> > -Todd
> >
> > On Mon, Sep 12, 2016 at 9:38 AM, Jim Apple <jbap...@cloudera.com> wrote:
> >
> >> Apache Impala (incubating)  includes a file that includes substantial
> >> portions containing prose that is only licensed, as far as I can tell,
> in
> >> a
> >> GPL way:
> >>
> >> https://git-wip-us.apache.org/repos/asf?p=incubator-impala.g
> >> it;a=blob;f=be/.impala.doxy;h=4b81af4bab3c04ab60f84e29b70790
> >> 26e9959bf2;hb=fcb5c6821d1a0b2d49212dd791c4556dd5ac6c9e
> >>
> >> https://github.com/doxygen/doxygen/blob/b38efd15eb69b2b61e05
> >> ee09fc9ed6474cc8b1da/src/config.xml
> >>
> >> Can we keep that config file in our project as-is, or do we need to
> remove
> >> the prose, or perhaps some third thing?
> >>
> >> Thanks for your help,
> >> Jim
> >>
> >
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
>
>
> --
> Ryan Blue
> Software Engineer
> Netflix
>

2.7.0 release blocker: testdata/data/mstr origin

[Jim Apple]

In order to do a 2.7.0 release, we must ensure that everything in the
release is OK with http://www.apache.org/legal/resolved.html

I am using Apache RAT to do that. It is flagging the files in
testdata/data/mstr. Those files do not have a very informative git log
history:

https://github.com/apache/incubator-impala/commits/master/testdata/data/mstr

Can anyone vouch for where they came from? This is a 2.7.0 release blocker.

Thanks,
Jim

Config file comments generated by GPL tools

[Jim Apple]

Apache Impala (incubating)  includes a file that includes substantial
portions containing prose that is only licensed, as far as I can tell, in a
GPL way:

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=blob;f=be/.impala.doxy;h=4b81af4bab3c04ab60f84e29b7079026e9959bf2;hb=fcb5c6821d1a0b2d49212dd791c4556dd5ac6c9e

https://github.com/doxygen/doxygen/blob/b38efd15eb69b2b61e05ee09fc9ed6474cc8b1da/src/config.xml

Can we keep that config file in our project as-is, or do we need to remove
the prose, or perhaps some third thing?

Thanks for your help,
Jim

Re: Sending Code Review emails to a different list

[Jim Apple]

This now works. dev@ should no longer see CR comments, reviews@ is the
new list. You can subscribe via reviews-subscribe@.

See also: https://lists.apache.org/list.html?revi...@impala.apache.org

And maybe eventually:
http://mail-archives.apache.org/mod_mbox/incubator-impala-reviews/

Here is how I did it:

1. I sent mail from my moderator account for reviews@ to
reviews-allow-subscribe-gerrit=cloudera@impala.incubator.apache.org

2. I replied to the email that generated, again using my moderator account.

3. I checked that it worked by mailing
reviews-allow-l...@impala.incubator.apache.org.

4. Following the instructions on
https://gerrit-review.googlesource.com/Documentation/user-notify.html,
I changed the gerrit project's refs/meta/config branch to have

[notify "team"]
email = revi...@impala.incubator.apache.org

rather than

[notify "team"]
   email = dev@impala.incubator.apache.org

5. Updated the incubating status website
(http://incubator.apache.org/projects/impala.html) by following the
instructions on http://incubator.apache.org/guides/website.html

Still TODO (lower priority for now): update wiki where appropriate,
get issues@ to actually work, update
http://impala.incubator.apache.org/ where appropriate

On Fri, Sep 9, 2016 at 4:58 PM, Todd Lipcon <t...@cloudera.com> wrote:
> OK, I submitted the request. For Kudu this took a week or so to show up, but
> I think they were dealing with a backlog at that time, and hopefully will go
> faster this time around.
>
> On Fri, Sep 9, 2016 at 4:53 PM, Henry Robinson <he...@cloudera.com> wrote:
>>
>> Sure thing. I already moderate dev@.
>>
>> On 9 September 2016 at 16:51, Todd Lipcon <t...@cloudera.com> wrote:
>>
>> > How about another volunteer to be a moderator?
>> >
>> > -Todd
>> >
>> > On Fri, Sep 9, 2016 at 1:55 PM, Jim Apple <jbap...@cloudera.com> wrote:
>> >
>> > > reviews@ WFM.
>> > >
>> > > On Fri, Sep 9, 2016 at 1:43 PM, Todd Lipcon <t...@cloudera.com> wrote:
>> > >
>> > >> Sorry, I'd accidentally muted this thread and didn't see my name
>> > >> called
>> > >> out :)
>> > >>
>> > >> The form requests a second moderator. Who can I add in addition to
>> > >> Jim?
>> > >> Also, I think more projects are using 'reviews@' than
>> > >> 'code-reviews@'.
>> > >> Any preference for the latter or can I use the shorter for more
>> > consistency
>> > >> with kudu lists, etc?
>> > >>
>> > >> -Todd
>> > >>
>> > >> On Fri, Sep 2, 2016 at 10:15 AM, Jim Apple <jbap...@cloudera.com>
>> > wrote:
>> > >>
>> > >>> Forgot to CC todd.
>> > >>>
>> > >>> On Fri, Sep 2, 2016 at 9:54 AM, Jim Apple <jbap...@cloudera.com>
>> > wrote:
>> > >>> > Tried to set it up here: https://infra.apache.org/offic
>> > >>> ers/mlreq/incubator
>> > >>> >
>> > >>> > But it turns out "Only Members and Officers (this includes all PMC
>> > >>> > chairs) can submit the form."
>> > >>> >
>> > >>> > Todd, I think you are an officer - could you submit this for us?
>> > >>> >
>> > >>> > List name: code-review
>> > >>> > Set Reply-To list header?: no
>> > >>> > Moderation: allow subscribers to post, moderate all others [n.b.:
>> > >>> > we'll need a workaround to allow posting by gerrit, but the
>> > >>> > Hipchat
>> > >>> > ASF infra room just told me that a moderator can send mail to
>> > >>> > code-review-allow-foo=b...@impala.incubator.apache.org to subscribe
>> > >>> > foo@bar to code-rev...@impala.incubator.apache.org]
>> > >>> > Moderators' addresses: jbap...@apache.org
>> > >>> > Notes: The impala code review tool forwards patches for code
>> > >>> > review
>> > >>> > and the comments on those patches to this list.
>> > >>> >
>> > >>> > On Thu, Sep 1, 2016 at 5:21 PM, Harrison Sheinblatt
>> > >>> > <hsheinbl...@cloudera.com> wrote:
>> > >>> >> +1 (nonbinding)
>> > >>> >>
>> > >>> >> On Thu, Sep 1, 2016 at 5:18 PM, Matthew Jacobs <m...@cloudera.com>
>> > >>> wrote:
>&g

[Impala-ASF-CR] IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.
..


Patch Set 1:

This is only a "PREVIEW", as I am seeking feedback on this approach before I 
clean up this patch for further review.

-- 
To view, visit http://gerrit.cloudera.org:8080/4361
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I5bfe77f9a871018e7a67553ed270e2df53006962
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Henry Robinson <he...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.

[Jim Apple (Code Review)]

Jim Apple has uploaded a new change for review.

  http://gerrit.cloudera.org:8080/4361

Change subject: IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.
..

IMPALA-4110: PREVIEW: Make RAT run on Impala tarballs.

While I'm here, fix some of the easier warnings.

Change-Id: I5bfe77f9a871018e7a67553ed270e2df53006962
---
M LICENSE.txt
M be/.astylerc
M be/src/benchmarks/expr-benchmark.cc
M be/src/experiments/data-provider.cc
M be/src/experiments/hashing/cache-hash-test.cc
M be/src/experiments/hashing/growing-test.cc
M be/src/experiments/hashing/interface/cache-hash-test.cc
M be/src/experiments/hashing/interface/growing-test.cc
M be/src/experiments/hashing/multilevel/cache-hash-test.cc
M be/src/experiments/hashing/multilevel/growing-test.cc
M be/src/experiments/hashing/prefetch/cache-hash-test.cc
M be/src/experiments/hashing/prefetch/growing-test.cc
M be/src/experiments/hashing/split-benchmarks/cache-hash-test.cc
M be/src/experiments/hashing/split-benchmarks/growing-test.cc
M be/src/experiments/hashing/split-benchmarks/partitioning-throughput-test.cc
M be/src/experiments/hashing/streaming/cache-hash-test.cc
M be/src/experiments/hashing/streaming/growing-test.cc
M be/src/testutil/certificates-info.txt
M be/src/util/asan.h
M bin/README-RUNNING-BENCHMARKS
A bin/check-rat-report.py
A bin/rat.sh
A bin/rat_exclude_files.txt
M fe/src/main/java/com/cloudera/impala/analysis/AggregateInfoBase.java
M fe/src/main/java/com/cloudera/impala/analysis/ExprSubstitutionMap.java
M fe/src/main/java/com/cloudera/impala/analysis/PartitionSpec.java
M fe/src/main/java/com/cloudera/impala/catalog/ArrayType.java
M fe/src/main/java/com/cloudera/impala/catalog/MapType.java
M fe/src/main/java/com/cloudera/impala/catalog/StructType.java
M fe/src/main/java/com/cloudera/impala/planner/Planner.java
30 files changed, 582 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/4361/1
-- 
To view, visit http://gerrit.cloudera.org:8080/4361
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5bfe77f9a871018e7a67553ed270e2df53006962
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple <jbap...@cloudera.com>

Re: Sending Code Review emails to a different list

[Jim Apple]

reviews@ WFM.

On Fri, Sep 9, 2016 at 1:43 PM, Todd Lipcon <t...@cloudera.com> wrote:

> Sorry, I'd accidentally muted this thread and didn't see my name called
> out :)
>
> The form requests a second moderator. Who can I add in addition to Jim?
> Also, I think more projects are using 'reviews@' than 'code-reviews@'.
> Any preference for the latter or can I use the shorter for more consistency
> with kudu lists, etc?
>
> -Todd
>
> On Fri, Sep 2, 2016 at 10:15 AM, Jim Apple <jbap...@cloudera.com> wrote:
>
>> Forgot to CC todd.
>>
>> On Fri, Sep 2, 2016 at 9:54 AM, Jim Apple <jbap...@cloudera.com> wrote:
>> > Tried to set it up here: https://infra.apache.org/offic
>> ers/mlreq/incubator
>> >
>> > But it turns out "Only Members and Officers (this includes all PMC
>> > chairs) can submit the form."
>> >
>> > Todd, I think you are an officer - could you submit this for us?
>> >
>> > List name: code-review
>> > Set Reply-To list header?: no
>> > Moderation: allow subscribers to post, moderate all others [n.b.:
>> > we'll need a workaround to allow posting by gerrit, but the Hipchat
>> > ASF infra room just told me that a moderator can send mail to
>> > code-review-allow-foo=b...@impala.incubator.apache.org to subscribe
>> > foo@bar to code-rev...@impala.incubator.apache.org]
>> > Moderators' addresses: jbap...@apache.org
>> > Notes: The impala code review tool forwards patches for code review
>> > and the comments on those patches to this list.
>> >
>> > On Thu, Sep 1, 2016 at 5:21 PM, Harrison Sheinblatt
>> > <hsheinbl...@cloudera.com> wrote:
>> >> +1 (nonbinding)
>> >>
>> >> On Thu, Sep 1, 2016 at 5:18 PM, Matthew Jacobs <m...@cloudera.com>
>> wrote:
>> >>
>> >>> +1
>> >>>
>> >>> On Thu, Sep 1, 2016 at 4:18 PM, Henry Robinson <he...@cloudera.com>
>> wrote:
>> >>> > Sounds good.
>> >>> >
>> >>> > On 1 September 2016 at 16:14, Sailesh Mukil <sail...@cloudera.com>
>> >>> wrote:
>> >>> >
>> >>> >> +1
>> >>> >>
>> >>> >> On Thu, Sep 1, 2016 at 4:11 PM, Daniel Hecht <dhe...@cloudera.com>
>> >>> wrote:
>> >>> >>
>> >>> >> > Sounds good to me.
>> >>> >> >
>> >>> >> > On Thu, Sep 1, 2016 at 4:04 PM, Alex Behm <
>> alex.b...@cloudera.com>
>> >>> >> wrote:
>> >>> >> >
>> >>> >> > > Feels much better to have a separate cr list. +1 for this idea
>> >>> >> > >
>> >>> >> > > On Thu, Sep 1, 2016 at 3:57 PM, Jim Apple <
>> jbap...@cloudera.com>
>> >>> >> wrote:
>> >>> >> > >
>> >>> >> > > > Today, all code reviews get send to dev@, causing it to
>> have a
>> >>> huge
>> >>> >> > > > mail volume (1777 messages in August). I feel dev@ might be
>> more
>> >>> >> > > > welcoming if it only included mails sent by humans; that
>> way, new
>> >>> >> > > > contributors don't have to worry about setting up filters
>> right
>> >>> away.
>> >>> >> > > > CR emails could go to code-review@impala.incubator.a
>> pache.org or
>> >>> >> > > > something else that the Apache infra people are willing to
>> set up.
>> >>> >> > > >
>> >>> >> > > > What does everyone else think?
>> >>> >> > > >
>> >>> >> > >
>> >>> >> >
>> >>> >>
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Henry Robinson
>> >>> > Software Engineer
>> >>> > Cloudera
>> >>> > 415-994-6679
>> >>>
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

[Impala-ASF-CR] IMPALA-4096: Allow clean.sh to work from snapshots

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-4096: Allow clean.sh to work from snapshots
..


Patch Set 3:

> Build failed: 
> http://sandbox.jenkins.cloudera.com/job/impala-external-gerrit-verify-merge-ASF/181/

Waiting on https://gerrit.cloudera.org/#/c/4337/ 
https://issues.cloudera.org/browse/IMPALA-4097

-- 
To view, visit http://gerrit.cloudera.org:8080/4336
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I12dd9035298151557491009680d66d25c8f58c1d
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-HasComments: No

Re: Prepare to vote on release candidate 1

[Jim Apple]

>> - would be nice if the version number didn't have 'cdh5' in it (eg
>> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
>> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
>
> Lars is working on this now:
>
> https://gerrit.cloudera.org/#/c/4187/3
>
> Some possibilities for 2.7.0:
>
> 1. Leave it. Pros: expedient; Cons: inaccurate, not Apache
>
> 2. Wait until we can change this is master without confusing existing
> tools. Pros: clean; Cons: delay in releasing
>
> 3. Make a commit just to branch-2.7.0. Pros: Doesn't confuse existing
> tools; Cons: Makes branch-2.7.0 not a pure subset of master.
>
> What does everyone think?

Just a reminder that your input on this is valued. I slightly prefer #3.

Re: Prepare to vote on release candidate 1

[Jim Apple]

Do you mean of the tree, ala

d4b8675dd7a717c80305f5c35ef0179e9c2034b3 is the hash of the tree

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tree;h=d4b8675dd7a717c80305f5c35ef0179e9c2034b3;hb=fcb5c6821d1a0b2d49212dd791c4556dd5ac6c9e

which I got by looking at this commit

https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=commit;h=fcb5c6821d1a0b2d49212dd791c4556dd5ac6c9e

which is the latest commit to branch-2.7.0


On Fri, Sep 9, 2016 at 2:04 AM, Tom White <t...@cloudera.com> wrote:
> On Thu, Sep 8, 2016 at 5:27 PM, Jim Apple <jbap...@cloudera.com> wrote:
>> Sure, I can tag the tree where I git archive'd my next RC. Just to be
>> sure I understand you - what exactly do you mean by the "git SHA"?
>
> I just mean git hash.
>
>>
>> On Thu, Sep 8, 2016 at 2:24 AM, Tom White <t...@cloudera.com> wrote:
>>> Jim,
>>>
>>> Can you provide a git tag and git SHA for the release when you send
>>> out the vote so we can check the tarball against the tag.
>>>
>>> Thanks!
>>> Tom
>>>
>>> On Thu, Sep 8, 2016 at 1:37 AM, Todd Lipcon <t...@cloudera.com> wrote:
>>>> - agree with Tim that it would be nice if the tarball extracted into a
>>>> directory named apache-impala-incubating-2.7.0 (to match the tarball 
>>>> prefix)
>>>>
>>>> - A couple places seem to keep a Cloudera copyright/license - eg
>>>> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped for
>>>> Cloudera and found these). Should probably check over these and fix before
>>>> the "real" RC.
>>>>
>>>> - I tried building using 'buildall.sh' but it failed with 'fatal: Not a git
>>>> repository' when trying to do a git clean. Maybe some tweaks need to be
>>>> made so that Impala can be built from a tarball? I worked around it using
>>>> 'git init' inside my extracted directory.
>>>>
>>>> - would be nice if the version number didn't have 'cdh5' in it (eg
>>>> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
>>>> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
>>>>
>>>> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
>>>> Starting hdfs (Web UI - http://localhost:5070)
>>>> Failed to start hdfs-datanode. The end of the log
>>>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/hdfs-datanode.out)
>>>> is:
>>>> Failed to start hdfs-datanode. The end of the log
>>>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/hdfs-datanode.out)
>>>> is:
>>>> Failed to start hdfs-datanode. The end of the log
>>>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/hdfs-datanode.out)
>>>> is:
>>>>
>>>> That said, it appears that impalad built OK (with build-all.sh -notests)
>>>>
>>>>
>>>> some nits/suggestions (ok to address in a later release:
>>>> - a few mentions of 'Cloudera Impala' in the various pom files
>>>> - would be great if buildall.sh could check if there is enough remaining
>>>> space on the drive before building. I had 20GB free but still ran out of
>>>> space trying to do the default build (had to rebuild with -notests)
>>>> - consider setting up a cloudfront distribution in front of the
>>>> native-toolchain S3 bucket? It downloads pretty slowly for me
>>>> -- related: consider stripping debug info from some of the built deps? eg
>>>> Kudu is 500+MB which seems unnecessary for a test dependency.
>>>>
>>>>
>>>> On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <tarmstr...@cloudera.com>
>>>> wrote:
>>>>
>>>>> I'm in the process of testing this.
>>>>>
>>>>> One nit that I think we should fix: apache-impala-incubating-2.7.
>>>>> 0-rc1.tar.gz
>>>>> unpacks to incubator-impala. I think we should stick with the normal
>>>>> convention of unpacking to a directory with the same name as the tarball
>>>>> (or maybe apache-impala-incubating-2.7.0).
>>>>>
>>>>> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com> wrote:
>>>>>
>>>>> > Jim, this is great work (by everyone) and the culmination of a ton of
>>>>> > effort. Thanks for getting us to this stage!
>>>>> >
>>>>> > On 7 September 2016 at 

[Impala-ASF-CR] IMPALA-4096: Allow clean.sh to work from snapshots

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-4096: Allow clean.sh to work from snapshots
..


Patch Set 3: Code-Review+2

-- 
To view, visit http://gerrit.cloudera.org:8080/4336
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I12dd9035298151557491009680d66d25c8f58c1d
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-HasComments: No

Re: Contributions to Cloudera Impala

[Jim Apple]

I do not know if any such method is possible. Did you read
https://review.openstack.org/Documentation/user-upload.html? Did you try
anything on it?

Have you tried debugging your ssh issues? Did you talk to your network
config people?

On Thu, Sep 8, 2016 at 9:02 PM, Valencia Serrao <vser...@us.ibm.com> wrote:

> Hi Jim,
>
> Thanks for you concern.
>
> Looks like we are having some issues while accessing the ssh URL, could
> you please provide us the equivalent https link for the asf-gerrit remote ?
>
>
> Regards,
> Valencia
>
>
> [image: Inactive hide details for Jim Apple ---09/07/2016 01:58:53
> AM---Any luck with this? On Thu, Aug 25, 2016 at 10:47 AM, Jim Apple]Jim
> Apple ---09/07/2016 01:58:53 AM---Any luck with this? On Thu, Aug 25, 2016
> at 10:47 AM, Jim Apple <jbap...@cloudera.com> wrote:
>
> From: Jim Apple <jbap...@cloudera.com>
> To: Nishidha Panpaliya/Austin/Contr/IBM@IBMUS
> Cc: "dev@impala" <dev@impala.incubator.apache.org>, Manish
> Patil/Austin/Contr/IBM@IBMUS, Silvius Rus <s...@cloudera.com>, Valencia
> Serrao/Austin/Contr/IBM@IBMUS, Sudarshan Jagadale/Austin/Contr/IBM@IBMUS,
> Zhi Zhi NY Yang <zhiz...@cn.ibm.com>
> Date: 09/07/2016 01:58 AM
> Subject: Re: Contributions to Cloudera Impala
> --
>
>
>
> Any luck with this?
>
> On Thu, Aug 25, 2016 at 10:47 AM, Jim Apple <*jbap...@cloudera.com*
> <jbap...@cloudera.com>> wrote:
>
>That sounds to me like a network config issue.
>
>As far as what the code should be based on, the Apache repo and the
>gerrit repo should be close, with apache being a few commits behind at
>most. I use gerrit almost exclusively, because that way I know my patches
>will be based on the most recent tree.
>
>On Mon, Aug 22, 2016 at 5:20 AM, Nishidha Panpaliya <
>*nishi...@us.ibm.com* <nishi...@us.ibm.com>> wrote:
>Hi Jim,
>
>We tried the steps mentioned at
>
> *https://cwiki.apache.org/confluence/display/IMPALA/How+to+switch+to+Apache-hosted+git*
>
> <https://cwiki.apache.org/confluence/display/IMPALA/How+to+switch+to+Apache-hosted+git>
>.
>On Cloudera's Impala source tree ->
>$git remote add apache
>*https://git-wip-us.apache.org/repos/asf/incubator-impala.git*
><https://git-wip-us.apache.org/repos/asf/incubator-impala.git>
>$git remote add asf-gerrit ssh://
>*npanpal...@gerrit.cloudera.org:29418/Impala-ASF*
><http://npanpal...@gerrit.cloudera.org:29418/Impala-ASF>
>(Here, npanpaliya is my github id which I also used for gerrit login,
>also added my ssh key to my gerrit settings)
>$git fetch asf-gerrit
>ssh: Could not resolve hostname *gerrit.cloudera.org:29418*
><http://gerrit.cloudera.org:29418/>: Name or service not known
>   fatal: Could not read from remote repository.
>
>Please make sure you have the correct access rights
>and the repository exists.
>
>Could you please help us with this error?
>
>Our understanding is we need repo referred by "asf-gerrit" only for
>uploading gerrit code reviews. To work (build, modify, port, test) on
>actual Impala code base, we still have to follow the steps at
>*https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala*
><https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala>.
>that uses code base referred by "apache" remote.
>
>Thanks,
>Nishidha
>
>Jim Apple ---08/19/2016 08:58:00 PM---> [Nishidha] We found two new
>source code URLs as one mentioned in Confluence >
>*https://git-wip-us.a* <https://git-wip-us.a/>
>
>From: Jim Apple <*jbap...@cloudera.com* <jbap...@cloudera.com>>
>To: Nishidha Panpaliya/Austin/Contr/IBM@IBMUS
>Cc: "dev@impala" <*dev@impala.incubator.apache.org*
><dev@impala.incubator.apache.org>>, Manish Patil/Austin/Contr/IBM@IBMUS,
>Silvius Rus <*s...@cloudera.com* <s...@cloudera.com>>, Valencia
>Serrao/Austin/Contr/IBM@IBMUS, Sudarshan Jagadale/Austin/Contr/IBM@IBMUS,
>Zhi Zhi NY Yang <*zhiz...@cn.ibm.com* <zhiz...@cn.ibm.com>>
>Date: 08/19/2016 08:58 PM
>Subject: Re: Contributions to Cloudera Impala
>--
>
>
>
>> [Nishidha] We found two new source code URLs as one mentioned in
>Confluence
>> *https://git-wip-us.apache.org/repos/asf/incubator-impala.git*
><https://git-wip-us.apache.org/repos/asf/incubator-impala.git> and
>another to
>> be *https://github.com/apache/incubator-impala*

[Impala-ASF-CR] IMPALA-4096: Allow clean.sh to work from snapshots

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-4096: Allow clean.sh to work from snapshots
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4336/2/bin/clean.sh
File bin/clean.sh:

PS2, Line 55: git rev-parse &>dev/null
> Here and elsewhere, you're missing the leading / for /dev/null unless this 
1. Done

2. Changed to 2>, as (a) this is a more common idiom, (b) git rev-parse 
produces no output in my testing, and (c) I don't have easy access to a CentOS 
5 installation.


-- 
To view, visit http://gerrit.cloudera.org:8080/4336
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I12dd9035298151557491009680d66d25c8f58c1d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-4096: Allow clean.sh to work from snapshots

[Jim Apple (Code Review)]

Jim Apple has uploaded a new patch set (#3).

Change subject: IMPALA-4096: Allow clean.sh to work from snapshots
..

IMPALA-4096: Allow clean.sh to work from snapshots

buildall.sh calls bin/clean.sh, which fails when not in a git
directory. This skips the "git clean" steps when not in a git
checkout.

Todd Lipcon found this when testing a snapshot created using git
archive.

Change-Id: I12dd9035298151557491009680d66d25c8f58c1d
---
M bin/clean.sh
1 file changed, 5 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/36/4336/3
-- 
To view, visit http://gerrit.cloudera.org:8080/4336
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I12dd9035298151557491009680d66d25c8f58c1d
Gerrit-PatchSet: 3
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Michael Brown <mi...@cloudera.com>

[Impala-ASF-CR] IMPALA-4096: Allow clean.sh to work from snapshots

[Jim Apple (Code Review)]

Jim Apple has posted comments on this change.

Change subject: IMPALA-4096: Allow clean.sh to work from snapshots
..


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/4336/2/bin/clean.sh
File bin/clean.sh:

PS2, Line 77: -fdx
> I just noticed this one is different from the other two. I think since you'
I am reluctant to change this in this commit, which I would like to be a no-op 
for current developers, without understanding why the options are written the 
way they are in the first place.


-- 
To view, visit http://gerrit.cloudera.org:8080/4336
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-MessageType: comment
Gerrit-Change-Id: I12dd9035298151557491009680d66d25c8f58c1d
Gerrit-PatchSet: 2
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Jim Apple <jbap...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-HasComments: Yes

Re: Prepare to vote on release candidate 1

[Jim Apple]

Sure, I can tag the tree where I git archive'd my next RC. Just to be
sure I understand you - what exactly do you mean by the "git SHA"?

On Thu, Sep 8, 2016 at 2:24 AM, Tom White <t...@cloudera.com> wrote:
> Jim,
>
> Can you provide a git tag and git SHA for the release when you send
> out the vote so we can check the tarball against the tag.
>
> Thanks!
> Tom
>
> On Thu, Sep 8, 2016 at 1:37 AM, Todd Lipcon <t...@cloudera.com> wrote:
>> - agree with Tim that it would be nice if the tarball extracted into a
>> directory named apache-impala-incubating-2.7.0 (to match the tarball prefix)
>>
>> - A couple places seem to keep a Cloudera copyright/license - eg
>> common/thrift/beeswax.thrift, FrontEndTestBase.java, etc. (I grepped for
>> Cloudera and found these). Should probably check over these and fix before
>> the "real" RC.
>>
>> - I tried building using 'buildall.sh' but it failed with 'fatal: Not a git
>> repository' when trying to do a git clean. Maybe some tweaks need to be
>> made so that Impala can be built from a tarball? I worked around it using
>> 'git init' inside my extracted directory.
>>
>> - would be nice if the version number didn't have 'cdh5' in it (eg
>> impala-shell-2.7.0-cdh5-INTERNAL, seems to come from bin/version.info,
>> bin/save-version.sh, etc). Should probably be '2.7.0-incubating'
>>
>> - can't seem to use testdata/bin/run-all.sh to start DFS. I get:
>> Starting hdfs (Web UI - http://localhost:5070)
>> Failed to start hdfs-datanode. The end of the log
>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-2/var/log/hdfs-datanode.out)
>> is:
>> Failed to start hdfs-datanode. The end of the log
>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-3/var/log/hdfs-datanode.out)
>> is:
>> Failed to start hdfs-datanode. The end of the log
>> (/tmp/incubator-impala/testdata/cluster/cdh5/node-1/var/log/hdfs-datanode.out)
>> is:
>>
>> That said, it appears that impalad built OK (with build-all.sh -notests)
>>
>>
>> some nits/suggestions (ok to address in a later release:
>> - a few mentions of 'Cloudera Impala' in the various pom files
>> - would be great if buildall.sh could check if there is enough remaining
>> space on the drive before building. I had 20GB free but still ran out of
>> space trying to do the default build (had to rebuild with -notests)
>> - consider setting up a cloudfront distribution in front of the
>> native-toolchain S3 bucket? It downloads pretty slowly for me
>> -- related: consider stripping debug info from some of the built deps? eg
>> Kudu is 500+MB which seems unnecessary for a test dependency.
>>
>>
>> On Wed, Sep 7, 2016 at 5:01 PM, Tim Armstrong <tarmstr...@cloudera.com>
>> wrote:
>>
>>> I'm in the process of testing this.
>>>
>>> One nit that I think we should fix: apache-impala-incubating-2.7.
>>> 0-rc1.tar.gz
>>> unpacks to incubator-impala. I think we should stick with the normal
>>> convention of unpacking to a directory with the same name as the tarball
>>> (or maybe apache-impala-incubating-2.7.0).
>>>
>>> On Wed, Sep 7, 2016 at 2:16 PM, Henry Robinson <he...@cloudera.com> wrote:
>>>
>>> > Jim, this is great work (by everyone) and the culmination of a ton of
>>> > effort. Thanks for getting us to this stage!
>>> >
>>> > On 7 September 2016 at 14:08, Jim Apple <jbap...@cloudera.com> wrote:
>>> >
>>> > > I have put a release candidate, along with checksums and a
>>> > > cryptographic signature, in
>>> > >
>>> > > https://dist.apache.org/repos/dist/dev/incubator/impala/2.7.0/
>>> > >
>>> > > I will be calling for a vote from the PPMC soon. This thread is not
>>> > > the vote thread.
>>> > >
>>> > > That vote will only pass, according to our bylaws, if it has 3 binding
>>> > > +1 votes and more binding +1 votes than -1 votes. Only the votes of
>>> > > PPMC members are binding, but anyone may vote.
>>> > >
>>> > > If that vote passes, I will ask the Incubator PMC to approve the
>>> > > release candidate, following the rules on
>>> > >
>>> > > http://incubator.apache.org/incubation/Incubation_Policy.html#Releases
>>> > >
>>> > > To +1 a release candidate, you will need to verify it. This includes:
>>> > >
>>> > > 1. Verifying the signature. You can import my code-signin