Re: Impalad Process Crash

2017-11-22 Thread Tim Armstrong
Crashes in tcmalloc::CentralFreeList::FetchFromSpans() or in TCMalloc functions in general usually indicate a heap corruption bug (e.g. a double-free or a free of an invalid pointer). The stack trace often isn't useful because the corruption happened on a different thread. These are among the

Re: [DISCUSS] 2.11.0 release

2017-12-07 Thread Tim Armstrong
Actually it looks like we have a new blocker that Taras filed: https://issues.apache.org/jira/browse/IMPALA-6292 On Thu, Dec 7, 2017 at 10:03 AM, Tim Armstrong <tarmstr...@cloudera.com> wrote: > I think that makes sense. We'll have to go through the fix versions of > recent JIRAs a

Re: build broken

2017-12-05 Thread Tim Armstrong
Thanks for investigating Jin Chul. I came to the same conclusion and merged the change. On Tue, Dec 5, 2017 at 5:57 PM, Jin Chul Kim wrote: > Hi, > > Currently Impala build is broken because the child change was merged > without the merge of the parent change. > > (parent)

Re: [DISCUSS] 2.11.0 release

2017-12-07 Thread Tim Armstrong
It looks like https://issues.apache.org/jira/browse/IMPALA-5017 caused https://issues.apache.org/jira/browse/IMPALA-6284, so maybe we should not include IMPALA-5017 in the 2.11 release also. On Thu, Dec 7, 2017 at 9:43 AM, Jim Apple wrote: > I think it would be great to

Re: build broken

2017-12-06 Thread Tim Armstrong
Chul for investigating the problem. > Sorry > > for missing that during code review. > > > > On Tue, Dec 5, 2017 at 6:00 PM, Tim Armstrong <tarmstr...@cloudera.com> > > wrote: > > > > > Thanks for investigating Jin Chul. I came to the same conclusion and

Re: [ANNOUNCE] Apache Impala 3.0.0 release

2018-05-10 Thread Tim Armstrong
Thanks Sailesh! On Wed, May 9, 2018 at 10:09 PM, Jim Apple wrote: > Thanks to Sailesh for managing the release, and thanks to the many, many > others who contributed to creating this new branch and maintaining the > relationship between the two branches. > > On Wed, May 9,

Re: New message on your code reviews when cherry-picking your change fails.

2018-05-11 Thread Tim Armstrong
Thanks for setting this up, this should make things a lot easier. On Fri, May 11, 2018 at 12:02 PM, Philip Zeyliger wrote: > Hey folks, > > I just updated https://jenkins.impala.io/job/cherrypick-2.x-and-test/ to > comment on a code review the first time a change causes the

Re: jenkins.impala.io's ubuntu-16.04-from-scratch will now have Junit test results

2018-05-15 Thread Tim Armstrong
I think it's trying to interpret our docs as test results: https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2186/#showFailuresLink - - Test Result (250 failures

Re: Automatically rebase changes before GVO?

2018-06-11 Thread Tim Armstrong
pala.io/job/gerrit-verify-dryrun/jobConfigHistory/ > showDiffFiles?timestamp1=2018-06-11_22-09-04= > 2018-06-12_02-47-10 > > On Mon, Jun 11, 2018 at 3:09 PM, Tim Armstrong > wrote: > > > Ok, I applied the changes. Let me know if you run into any issues. > > > &

Re: Broken/Flaky Tests

2018-06-07 Thread Tim Armstrong
that it won't break any other configurations (S3, Local, etc). On Wed, Jun 6, 2018 at 2:44 PM, Tim Armstrong wrote: > We ran into some test issues cherry-picking the latest set of changes to > 2.x. I pushed out a fix and I'm merging now. Once that is done the main > build fixes should be

Re: Automatically rebase changes before GVO?

2018-06-07 Thread Tim Armstrong
gt; > Seems fine, especially since we do the rebase as our submission strategy > > anyway, so we're already accepting/testing something that's likely to get > > rebased, and we may as well minimize that window. > > > > I'd be in favor of the bot also carrying the votes. >

Re: Branch 2.x

2018-06-18 Thread Tim Armstrong
This just came up - do we want to backport bug or test fixes to 2.x or nothing at all? https://gerrit.cloudera.org/#/c/10722/ On Mon, Jun 18, 2018 at 10:48 AM, Tim Armstrong wrote: > Thanks Phil, I think it will be good not to have to maintain this. > > On Mon, Jun 18, 2018 at 9:40 A

Re: Branch 2.x

2018-06-18 Thread Tim Armstrong
Thanks Phil, I think it will be good not to have to maintain this. On Mon, Jun 18, 2018 at 9:40 AM, Philip Zeyliger < phi...@cloudera.com.invalid> wrote: > I've not heard anything, so I've removed the "trigger" for job > https://jenkins.impala.io/job/cherrypick-2.x-and-test/. It used to "Poll >

Future of unsupported formats?

2018-06-18 Thread Tim Armstrong
For a few years now we've had write support for Sequence, Avro and compressed text hidden behind a query option. We haven't really made any progress on turning it into a supported feature, so I'm wondering if we should remove the code and save some overhead of building, testing and code

Re: Configuring Impala to exit on OutOfMemoryError

2018-06-14 Thread Tim Armstrong
It definitely seems like it would result in more predictable behaviour. I think the other reasonable alternative is that we actually test the behaviour of the system when it gets into an OOM state and make sure that it can recover (which might not be possible with a reasonable amount of effort).

Apache jira-bot stopped working

2018-06-15 Thread Tim Armstrong
Just FYI, it decided to take a day off, in case you were waiting for a comment on your JIRA: https://issues.apache.org/jira/browse/INFRA-16651 Hopefully it's back up now.

Re: Broken/Flaky Tests

2018-06-14 Thread Tim Armstrong
I'm concerned that for the last couple of days we've been finding new issues faster than we're fixing them. I'll start pushing back on some higher-risk changes (e.g. in race-prone parts of the code) until we've cleared up some of the issues. On Tue, Jun 12, 2018 at 1:31 PM, Tim Armstrong wrote

Re: Automatically rebase changes before GVO?

2018-06-11 Thread Tim Armstrong
I've tried my job a few times and it's working as expected. Any objections to me switching over gerrit-verify-dryrun to my approach? On Thu, Jun 7, 2018 at 2:42 PM, Tim Armstrong wrote: > Ok, I was able to put together a test job that does the automatic rebase > and carries a +2 here:

Re: Automatically rebase changes before GVO?

2018-06-11 Thread Tim Armstrong
me. > > On Mon, Jun 11, 2018 at 12:06 PM, Tim Armstrong > wrote: > > > > On nit: as GVD gets more complex, it becomes harder for new people to > > understand the messages and +Ns applied to their patches. That doesn't > mean > > we shouldn't do this, only that it's

Re: Automatically rebase changes before GVO?

2018-06-11 Thread Tim Armstrong
Ok, I applied the changes. Let me know if you run into any issues. On Mon, Jun 11, 2018 at 3:05 PM, Sailesh Mukil wrote: > +1 > > On Mon, Jun 11, 2018 at 3:02 PM, Jim Apple wrote: > > > No objection from me. > > > > On Mon, Jun 11, 2018 at 12:06

Re: Future of unsupported formats?

2018-06-19 Thread Tim Armstrong
rth taking a survey wider than dev@. User@ or the customers > of Impala packagers might be good places to start. > > On Mon, Jun 18, 2018 at 5:10 PM Tim Armstrong > wrote: > > > For a few years now we've had write support for Sequence, Avro and > > compressed text hidd

Re: Future of unsupported formats?

2018-06-19 Thread Tim Armstrong
Lars, Even if people were using it, would that affect our decision if there's noone to maintain it? I don't think we were ambiguous about whether writing those formats was supported or not. On Tue, Jun 19, 2018 at 10:04 AM, Tim Armstrong wrote: > Hi Edward, > I was talking about

Re: Future of unsupported formats?

2018-06-19 Thread Tim Armstrong
r > > communication channels they might have access to. > > > > Cheers, Lars > > > > On Tue, Jun 19, 2018 at 8:26 AM Tim Armstrong > > wrote: > > > > > I don't think we need to bump a major version to remove something that >

Re: Broken/Flaky Tests

2018-06-05 Thread Tim Armstrong
Things are starting to look healthier now. I went through the broken-build JIRAs and downgraded some of the infrequent infrastructure issues to critical so we have a clearer idea of what's actually breaking the build now versus what's an occasional infra issue:

Re: UDA debugging, was Re: Broken/Flaky Tests

2018-06-05 Thread Tim Armstrong
so 2/3 of those fixes are merged and the other is being merged. > > > > We still have a long list of flaky issues but I went through and we've > > either mitigated them or we're blocked on being able to repro them. > > > > I'll see how things look tomorrow, but if you have

Re: Broken/Flaky Tests

2018-06-06 Thread Tim Armstrong
We ran into some test issues cherry-picking the latest set of changes to 2.x. I pushed out a fix and I'm merging now. Once that is done the main build fixes should be on both 2.x and master. On Tue, Jun 5, 2018 at 6:08 PM, Tim Armstrong wrote: > Ok, so 2/3 of those fixes are mer

Jenkins upgrade

2018-06-04 Thread Tim Armstrong
I'll be upgrading Jenkins today in response to a security advisory. There was only one job running and I've talked to mikeb about it already. I'll reply to this thread once it's back up.

Need reviewer for SHUTDOWN patch

2018-07-02 Thread Tim Armstrong
Does anyone have time to review this patch? It's a pretty big step towards being able to shrink an Impala cluster without disrupting workloads. https://gerrit.cloudera.org/#/c/10744/

Re: Build error

2018-01-09 Thread Tim Armstrong
The error is probably higher up in your console output. The lines you have there look like the side-effects of parallel build tasks being killed off after an earlier failure. The below line hints that the failure may have been in building the frontend Java: fe/CMakeFiles/fe.dir/build.make:57:

Re: Re: Cancellation logic in HdfsScanners

2018-01-16 Thread Tim Armstrong
Would you be able to file a JIRA? On Tue, Jan 16, 2018 at 3:00 PM, Tim Armstrong <tarmstr...@cloudera.com> wrote: > Looks to me like you found a bug. I think the scanners should be checking > both cancellation conditions, i.e. RuntimeState::is_cancelled_ for MT and >

Re: Re: Cancellation logic in HdfsScanners

2018-01-16 Thread Tim Armstrong
--+ > > When errors happen in F00, cancellation rpc will be sent to F01. However, the > hdfs scanner in F01 does not notice it in time and pass up all the row > batches. Then the DataStreamSender will try to send these r

Re: [Ready for Review] IMPALA-5717: Support reading from ORC format files

2018-01-26 Thread Tim Armstrong
Thank you! I had few higher-level questions or thoughts: * Assuming we end up using the ORC C++ library, we probably want to manage it in the same way that we do Avro by building it externally and then linking against it (we use the native-toolchain project for convenience). Importing the code

Re: Re: Re: Cancellation logic in HdfsScanners

2018-01-16 Thread Tim Armstrong
don't they just check > about RuntimeState::is_cancelled()? Are there any reasons that they > should go ahead until HdfsScanNode::done()? > > At 2018-01-17 07:00:51, "Tim Armstrong" <tarmstr...@cloudera.com> wrote: > > Looks to me like you found a bug. I think the

Re: .test file format

2018-01-12 Thread Tim Armstrong
It is possible to get the test infra to generate .test files based on the actual output. For planner tests, they automatically get written to logs/fe_tests/PlannerTest/. If you're running end-to-end tests with impala-py.test, you can use the --update_results flag to get it to generate similar

Re: Query compilation approach

2018-01-30 Thread Tim Armstrong
Thanks for sharing. I think we have some of the infrastructure required to do something similar - we have an interpreted path already and we could swap in compiled versions of functions by updating function pointers. Michael Ho and I have talked about doing async codegen before or even having a

Re: ERROR: NotImplementedException: MT_DOP not supported

2018-02-05 Thread Tim Armstrong
o through, I was not aware > that. basically, I was trying to insert some values in a partitioned table > and trying to find a way set a specific row's numDVs and the error > happened. > > 2018-02-05 15:24 GMT-06:00 Tim Armstrong <tarmstr...@cloudera.com>: > > > We don

Re: Re: ORC scanner - points for discussion

2018-02-12 Thread Tim Armstrong
Putting it behind a flag sounds good to me too. Hopefully we can get feedback from Hulu and other users of Impala that will try out the experimental version. On Mon, Feb 12, 2018 at 10:26 AM, Dimitris Tsirogiannis < dtsirogian...@cloudera.com> wrote: > Does the patch also implement an ORC

Re: Any design docs about the buffer pool

2018-02-09 Thread Tim Armstrong
I put together this summary to explain to reviewers the overall scope of the work: https://docs.google.com/presentation/d/1zSVATeUmhAmA6JvIWH4ODbhjsnx554WLeupigmUwi9g/edit#slide=id.p This was the top-level JIRA that tracked all of the work: https://issues.apache.org/jira/browse/IMPALA-3200 On

ORC scanner - points for discussion

2018-02-09 Thread Tim Armstrong
Quanlong has done a bunch of work implementing an ORC scanner. I've been playing around with it and it works pretty nicely - I can load and run TPC-H with no problem! It's a big addition to Impala and the integration with the external library has caused some implementation challenges, so I wanted

Re: Re: Re: Build fail in native-toolchain project

2018-02-14 Thread Tim Armstrong
There are. I tried to document the list at some point here: https://cwiki.apache.org/confluence/display/IMPALA/Building+native-toolchain+from+scratch+and+using+with+Impala I believe that covers most of them, but it may be slightly stale. Please feel free to update the list or let us know if

Re: minicluster with SSL

2018-02-22 Thread Tim Armstrong
If you figured this out, it would be helpful to add it to the developer wiki: https://cwiki.apache.org/confluence/display/IMPALA/Impala+Home . I gave you permissions to edit pages on it. On Wed, Feb 21, 2018 at 6:36 PM, Vincent Tran wrote: > Please disregard. It turns out

Re: Inconsistent float/double sort order in spec and implementations can lead to incorrect results

2018-02-16 Thread Tim Armstrong
There is an extensibility mechanism with the ColumnOrder union - I think that was meant to avoid the need to add new stat fields? Given that the bug was in the Parquet spec, we'll need to make a spec change anyway, so we could add a new ColumnOrder - FloatingPointTotalOrder? at the same time as

Re: Inconsistent float/double sort order in spec and implementations can lead to incorrect results

2018-02-16 Thread Tim Armstrong
> > On Fri, Feb 16, 2018 at 8:38 AM, Tim Armstrong <tarmstr...@cloudera.com> > wrote: > > > There is an extensibility mechanism with the ColumnOrder union - I think > > that was meant to avoid the need to add new stat fields? > > > > Given that the bug was

Re: Inconsistent float/double sort order in spec and implementations can lead to incorrect results

2018-02-16 Thread Tim Armstrong
t; > > > That fix does not preclude a more thorough solution in the future, but it > > addresses the common case quickly. > > > > For existing data files we could check the writer version ignore filters > on > > float/double. I don't know whether min/max

Re: Inconsistent float/double sort order in spec and implementations can lead to incorrect results

2018-02-16 Thread Tim Armstrong
; Yeah, I missed that. We set it per column, so all other types could keep > TypeDefinedOrder and floats could have something like NanAwareDoubleOrder. > > On Fri, Feb 16, 2018 at 9:18 AM, Tim Armstrong <tarmstr...@cloudera.com> > wrote: > > > We wouldn't need to rev th

Re: Inconsistent float/double sort order in spec and implementations can lead to incorrect results

2018-02-15 Thread Tim Armstrong
We could also consider treating NaN similar to NULL and having a separate piece of information with a count of NaN values (or just a bit indicating presence/absence of NaN). I'm not sure if that is easier or harder to implement than a total order. On Thu, Feb 15, 2018 at 9:12 AM, Laszlo Gaal

Working on cherry-picks to 2.x

2018-02-23 Thread Tim Armstrong
Just an FYI, there were some merge conflicts causing a logjam with the cherry-pick job. Alex fixed the first set, now i'm working on cherry-picking the remaining patches.

Re: test_semi_joins_exhaustive error?

2018-02-23 Thread Tim Armstrong
Looks like https://issues.apache.org/jira/browse/IMPALA-4857 . If there are rows with duplicate keys on the right side of a hash join, we have to be able to fit all rows for a key in memory to process it. I think we could avoid the problem by modifying the test so that it's joining on a column

Re: Setting up ASF Git Bot?

2018-02-21 Thread Tim Armstrong
Perhaps file an INFRA jira again? > > Thanks! > > -- Philip > > > > On Wed, Feb 21, 2018 at 9:52 AM, Tim Armstrong <tarmstr...@cloudera.com> > wrote: > > > What happened to git bot? I noticed that it stopped posting on JIRAs. > E.g. > > I had to co

Re: Setting up ASF Git Bot?

2018-02-21 Thread Tim Armstrong
What happened to git bot? I noticed that it stopped posting on JIRAs. E.g. I had to copy and paste the commit message manually on this one: https://issues.apache.org/jira/browse/IMPALA-6497 On Thu, Feb 1, 2018 at 10:17 AM, Philip Zeyliger wrote: > I filed

Re: Inconsistent float/double sort order in spec and implementations can lead to incorrect results

2018-02-19 Thread Tim Armstrong
We could drop NaNs and require that -0 be normalised to +0 when writing out stats. That would remove any degrees of freedom from the writer and then straightforward comparison with =, <, >, >=, <=, != would work as expected. On Mon, Feb 19, 2018 at 8:04 AM, Zoltan Ivanfi

Re: Joining Impala mailing list

2018-01-03 Thread Tim Armstrong
Hi Xinran, That looks like a good JIRA to start with. I wasn't able to figure out your Apache JIRA username but please feel free to assign it to yourself (or let us know if you need permissions to do so) Cheers, Tim On Tue, Jan 2, 2018 at 4:30 PM, kenneth mcfarland <

Re: Could anybody help to explain why there is such a big gap in 'ProbeTime' and How to fix this gap?

2018-01-03 Thread Tim Armstrong
That's a tricky one. I have a couple of ideas but it's a bit difficult to confirm since the profile isn't really designed to easily answer questions like this. ProbeTime measures wall-clock time rather than actual time spent executing on the CPU. My first guess is that it's because the Kudu scan

Re: Joining Impala mailing list

2018-01-03 Thread Tim Armstrong
and corner of the > "Contributors" box. A text box will appear to the right of the > contributor box. > > 3. Type the name of the person in that box to attempt to find their > Apache JIRA username. > > 4. Once you have it, press "Update" > > > On We

Re: Enabling automatic code review precommit job

2018-07-31 Thread Tim Armstrong
e backslash is redundant between brackets > 368 : E302 expected 2 blank lines, found 1 >1295 : E128 continuation line under-indented for visual indent > > Maybe worth just disabling some of the indentation-related ones to start? > > > On Mon, Jul 30, 2018 at 4:09 PM

Re: Enabling automatic code review precommit job

2018-07-31 Thread Tim Armstrong
lready 4 spaces in at the start of the region, you don't want to revert > back to 0 or 2 spaces in. That said, it can still be helpful. I don't think > this is as sophisticated as clang-format.el. > > On Tue, Jul 31, 2018 at 9:09 AM, Todd Lipcon > wrote: > > > On Tu

Re: Assign JIRA

2018-08-06 Thread Tim Armstrong
It looks like you're already an admin. I think you need to add the person as a "Contributor". I added "asherman" as a contributor. On Mon, Aug 6, 2018 at 12:54 PM, Thomas Tauber-Marshall < tmarsh...@cloudera.com.invalid> wrote: > I'm trying to assign a JIRA to a new contributor (Andrew Sherman),

Re: #pragma once?

2018-08-06 Thread Tim Armstrong
; > > potentially making the code even less portable. > > > > > > That being said, since Kudu has been using it for a while and has had > no > > > issues, we can do the same since the potential benefits outweigh the > > cons. > > > > > > On Wed, Aug 1,

Re: Enabling automatic code review precommit job

2018-08-16 Thread Tim Armstrong
you can do it here: https://jenkins.impala.io/ > gerrit_manual_trigger/ > " > > Zoltan > > > On Thu, Aug 16, 2018 at 7:19 PM Tim Armstrong > wrote: > > > Thanks for letting me know and sorry for the confusion. It's fine to go > > ahead and do a gerrit-verify-d

New Impala committer - Quanlong Huang

2018-08-17 Thread Tim Armstrong
The Project Management Committee (PMC) for Apache Impala has invited Quanlong Huang to become a committer and we are pleased to announce that they have accepted. Congratulations and welcome, Quanlong Huang!

New Impala PMC member: Joe McDonnell

2018-08-21 Thread Tim Armstrong
The Project Management Committee (PMC) for Apache Impala has invited Joe McDonnell to become a PMC member and we are pleased to announce that they have accepted. Congratulations and welcome, Joe!

Re: Improving Kudu Build Support

2018-08-21 Thread Tim Armstrong
Is there a path to building a version of Kudu locally for an arbitrary linux distro? Personally I am less concerned about 14.04 support and more concerned about what the path to upgrading to 18.04. It would also be nice for it to be at least possible to develop on RedHat-derived distros even if

Re: Improving latency of catalog update propagation?

2018-08-21 Thread Tim Armstrong
ency or doing a "push" to the statestore, > etc. > > I > > > probably won't work on this in the near term to avoid complicating the > > > ongoing changes with catalog. > > > > > > -Todd > > > > > > On Tue, Aug 21, 2018 at 10

Re: Improving latency of catalog update propagation?

2018-08-21 Thread Tim Armstrong
This is somewhat relevant for admission control too - I had thought about some of these issues in that context, because reducing the latency of admission controls state propagation helps avoid overadmission but having a very low statestore frequency is very inefficient and doesn't scale well to

Re: thrift dependency error - pip

2018-07-19 Thread Tim Armstrong
It's weird that it thinks that thrift_sasl-0.1.0.tar.gz is a version of thrift. Do you have a thrift tarball in infra/python/deps/? You could try rerunning ./infra/python/deps/download_requirements On Thu, Jul 19, 2018 at 5:30 PM, Bharath Vissapragada < bhara...@cloudera.com.invalid> wrote: >

Enabling automatic code review precommit job

2018-07-23 Thread Tim Armstrong
Hi All, I'm enabling an automatic precommit job for code reviews uploaded to gerrit that will run RAT, clang-tidy and a GCC debug compilation. This is to provide faster feedback on code reviews: https://issues.apache.org/jira/browse/IMPALA-7317 . I'll add some more checks but I'm wanting to test

Re: JSON support: Hive compatibility or ANSI SQL standard

2018-09-10 Thread Tim Armstrong
I think I agree with Lars. Having the (mostly) Hive-compatible version is useful if there are shared views between Hive/Impala and for people migrating queries from Hive or some of the Impala JSON UDFs that I've seen floating around. On Mon, Sep 10, 2018 at 8:20 AM, Lars Volker wrote: > Thanks

Re: boost::scoped_ptr vs std::unique_ptr

2018-07-05 Thread Tim Armstrong
the subtle difference between scoped_ptr and unique_ptr > off > > > the > > > > top of their head anyways. > > > > > > > > Fwiw, I was under the impression from talking with people in the past > > > that > > > > we were already

Re: Impala OS and Java version support

2018-07-06 Thread Tim Armstrong
Sounds like a good idea to me. Java 8 makes sense to me too since the major Java implementations (OpenJDK and Oracle) no longer support the older JDKs. On Fri, Jul 6, 2018 at 9:58 AM, Sailesh Mukil wrote: > I'm in favor of the minimum OS version support proposal. From 3.x, I don't > see any

Re: boost::scoped_ptr vs std::unique_ptr

2018-07-06 Thread Tim Armstrong
gt; the globe weigh in. > > BR, > Zoltan > > > On Fri, Jul 6, 2018 at 3:31 AM Jim Apple > wrote: > > > SGTM > > > > On Thu, Jul 5, 2018 at 6:13 PM, Tim Armstrong < > > tarmstr...@cloudera.com.invalid> wrote: > > > > > Sounds like un

Re: Inconsistent handling of schema in Avro tables

2018-07-11 Thread Tim Armstrong
The behaviour of Avro schemas in all these cases has always been rather mysterious to me. Before you wrote this email I would have assumed that Impala's behaviour would be like Hive's behaviour. I agree with the principle that the creation of a partition without changes to table metadata

impala-4835 gerrit branch

2018-03-15 Thread Tim Armstrong
You might see some gerrit notifications about commits to this branch. I created this temporary branch to make it easier to incrementally review fixes to my original IMPALA-4835 patches. I won't merge the changes to master without consultation with others. - Tim

Re: Maintenance for jenkins.impala.io today

2018-04-11 Thread Tim Armstrong
Is it possible that the upgrade broke some of our builds? I'm seeing some weird errors like the below, e.g. on this build https://jenkins.impala.io/job/clang-tidy-ub1604/1327/consoleFull *17:38:46* FATAL: java.lang.RuntimeException: Failed to serialize hudson.model.Actionable#actions for class

Re: Re: Re: ORC scanner - points for discussion

2018-03-28 Thread Tim Armstrong
-existing issues as a result of the testing done on this patch: https://issues.apache.org/jira/issues/?jql=project%20%3D%20ORC%20AND%20reporter%20in%20(stiga-huang) On Mon, Feb 12, 2018 at 1:52 PM, Tim Armstrong <tarmstr...@cloudera.com> wrote: > Maybe it would make sense to create an Epi

Re: Re: [DISCUSS] Apache Impala 2.12.0 and 3.0 releases

2018-04-02 Thread Tim Armstrong
I don't think we should hold the release for the ORC scanner patch - it's a great feature, but there are many other equally important features and improvements that have been waiting for months to be released. Generally, there are always going to be big changes that are "nearly there" and I think

Re: Unit testing framework to test impala queries

2018-10-31 Thread Tim Armstrong
Are you wanting to add new tests to the Impala codebase, or are you trying to do something different? We might be able to help if you're more specific about what you're trying to achieve. On Wed, Oct 31, 2018 at 8:38 AM Giridhar Yalavarthi < giridhar.yalavar...@gmail.com> wrote: > Hello Team, >

Re: [DISCUSS] 3.1.0 release

2018-11-05 Thread Tim Armstrong
+1 I know that I and some other people have been pushing to burn down the critical bugs remaining targeted for 3.1 and push out less critical work to 3.2 so we will be in a good position to do a release (it's been a while since 3.0!). I have this filter to track the open JIRAs targeted for 3.1:

Re: parallel-all-tests update

2018-09-27 Thread Tim Armstrong
I also added it to the pre-review tests. Let me know if you see any issues. On Thu, Sep 27, 2018 at 12:33 PM Philip Zeyliger wrote: > Hi folks, > > To address IMPALA-6543, there's a new test in parallel-all-tests that makes > sure that any Python scripts use Python2.6-compatible syntax. Note

Re: parallel-all-tests update

2018-09-27 Thread Tim Armstrong
Is it worth adding some regexes or similar to the gerrit bot to catch itertools.count() usage? Or do we not expect repeated bugs of this form? On Thu, Sep 27, 2018 at 12:59 PM Tim Armstrong wrote: > I also added it to the pre-review tests. Let me know if you see any issues. > > On Th

Re: jenkins.impala.io maintenance

2018-09-25 Thread Tim Armstrong
Excellent, the restart is done, let me know if you see any issues. On Tue, Sep 25, 2018 at 10:09 AM Jim Apple wrote: > You can kill it. Actually, I'll kill it, and you can restart at your > leisure. > On Tue, Sep 25, 2018 at 10:05 AM Tim Armstrong > wrote: > > > &g

jenkins.impala.io maintenance

2018-09-25 Thread Tim Armstrong
I'm going to restart Jenkins to upgrade some plugins. I'll try to restart any running jobs. Thanks, Tim

Re: jenkins.impala.io maintenance

2018-09-25 Thread Tim Armstrong
It looks like Jim has a running perf A/B test that is nearly done, so I'll let that run to completion. On Tue, Sep 25, 2018 at 10:02 AM Tim Armstrong wrote: > I'm going to restart Jenkins to upgrade some plugins. I'll try to restart > any running jobs. > > Thanks, > Tim >

Re: jenkins.impala.io maintenance coming up

2018-12-08 Thread Tim Armstrong
I was able to kick one off - let us know if you're still having issues. On Thu, Dec 6, 2018 at 11:53 PM Gabor Kaszab wrote: > Hey Lars, > > Is everything up and running? I'm asking because I'm unable to launch a > GVO. Every time I hit the build/rebuild button it times out with 504 > (Gateway

Re: [NOTICE] Mandatory migration of git repositories to gitbox.apache.org

2019-01-03 Thread Tim Armstrong
It would be good to give everyone at least a few days to plan their migration and do it at a specific time for the convenience of others that might have jobs or infrastructure using git-wip-us.apache.org On Thu, Jan 3, 2019 at 9:26 AM Jim Apple wrote: > And some jenkins.impala.io jobs > > On

New Impala PMC member - Zoltán Borók-Nagy

2019-01-04 Thread Tim Armstrong
The Project Management Committee (PMC) for Apache Impala has invited Zoltán Borók-Nagy to become a PMC member and we are pleased to announce that they have accepted. Congratulations and welcome, Zoltán!

Re: Deprecating file based authorization.

2018-09-13 Thread Tim Armstrong
This sounds ok to me, I'm not aware of any users of this mode. Maybe also email the user@ list to let people know of this change. On Tue, Sep 4, 2018 at 7:04 AM, Adam Holley wrote: > Hey all, > > Sentry has deprecated file based authorization. Impala currently has > options to enable this, so

Re: A problem of left outer join when statement contains two aggregation for the same column

2019-04-04 Thread Tim Armstrong
This also reminds me of https://issues.apache.org/jira/browse/IMPALA-7957 since there's an extra predicate added after the LEFT JOIN. On Thu, Apr 4, 2019 at 8:37 AM Quanlong Huang wrote: > Yes. I'm interested in this and going to look deeper in it tomorrow. Just > filed a JIRA:

Remote read testing in precommit

2019-04-05 Thread Tim Armstrong
This is really about testing the dockerised minicluster, but gives us coverage of remote read code paths for free, and more people care about that right now. I got the core end-to-end tests passing locally as part of https://issues.apache.org/jira/browse/IMPALA-7995. That change is up for review

Re: Jenkins update and restart

2019-03-06 Thread Tim Armstrong
l completion: > >- https://gerrit.cloudera.org/#/c/12542/ initially started by Fredy > Wijaya >- https://gerrit.cloudera.org/#/c/12640/ initially started by Tim > Armstrong > > Both jobs were restarted. > > On Wed, Mar 6, 2019 at 1:09 PM Bikramjeet Vig > > wr

Re: Next round of the Impala community meeting

2019-03-04 Thread Tim Armstrong
WFM. I think we should also consider a future meeting time (not this one) that will make it easy for people in Europe to join as well. On Mon, Mar 4, 2019 at 11:08 AM Lars Volker wrote: > Hi All, > > I propose to have the next Impala community meetings on > > Thursday, March 14th, 4pm PST/San

Re: [DISCUSS] 3.2.0 release

2019-03-04 Thread Tim Armstrong
+1 - what Jim said! On Mon, Mar 4, 2019 at 9:53 AM Jim Apple wrote: > +1 to both a release and a habit of curated release notes as Quanlong > suggests. > > On Mon, Mar 4, 2019 at 6:54 AM Quanlong Huang > wrote: > > > > +1. Thanks for your volunteer! > > > > If we decide to release, could we

Re: Impala Video Meeting

2019-02-06 Thread Tim Armstrong
That would work for me too On Wed., 6 Feb. 2019, 13:11 Quanlong Huang, wrote: > Thank you, Lars! I'm good for this time. Looking forward to it! > > Thanks, > Quanlong > > On Wed, Feb 6, 2019 at 6:32 PM Lars Volker wrote: > > > Thanks for the feedback. I looked at Beijing and San Francisco

Re: Could anyone assign IMPALA-8108 to me?

2019-02-14 Thread Tim Armstrong
I made you a contributor - that should let you assign it to yourself. On Thu, Feb 14, 2019 at 3:43 PM Robbie Zhang wrote: > Hi, > > I made a patch for it and tested on my cluster already. Could you please > assigned it to me so that I can send the patch for review? My Apache Jira > username is

Re: Request to be added as a contributor

2019-01-29 Thread Tim Armstrong
Ok, you should be added now. On Tue, Jan 29, 2019 at 6:49 PM Fang-Yu Rao wrote: > Hi Tim, > > My user name on the Apache JIRA is fangyurao. Thanks! > > Best regards, > Fang-Yu > > > On Mon, Jan 28, 2019 at 11:52 PM Tim Armstrong > wrote: > > > H

Re: Request to be added as a contributor

2019-01-28 Thread Tim Armstrong
Hi Fang-Yu, What is your user name on the Apache JIRA instance? Thanks, Tim On Tue, Jan 29, 2019 at 1:36 AM Fang-Yu Rao Rao wrote: > Hi, > > This is Fang-Yu Rao, currently an employee working on the Impala team at > Cloudera. I was wondering if you could add me as a contributor on the >

Re: Create version id for impala-2.13 in JIRA

2019-02-03 Thread Tim Armstrong
I unarchived the 2.13.0 version. Hopefully that should work for you. It looks like only the PMC has permissions to create versions. On Sat, Feb 2, 2019 at 12:34 AM Quanlong Huang wrote: > Hi, > > Could anyone help to create a "Fix version" for "Impala 2.13.0" or "Impala > 2.13"? I don't have

Re: Runtime filter publishing from a join node to its cousin

2019-01-30 Thread Tim Armstrong
I suspect the motivation might be something related to the order in which the join builds execute. I think the mental model is that they happen "top down", but in most cases in Impala they actually happen concurrently because either they're in different fragments or we start up an async build

Re: Impala Video Meeting

2019-02-04 Thread Tim Armstrong
I would also like to do that at some point if we can figure out the time zones :). - Tim On Wed, Jan 9, 2019 at 2:55 AM Quanlong Huang wrote: > +1. It's quite interesting and exciting if we can know each other and > everyone's focus area! > > I'm in Beijing timezone and I'm awake during 7 AM -

Re: Create version id for impala-2.13 in JIRA

2019-02-05 Thread Tim Armstrong
Quanlong was interested in continuing to add to the 2.x branch. I don't think we've discussed a release but this is needed to track things in 2.x that are not in 2.12 On Tue., 5 Feb. 2019, 06:29 Quanlong Huang Thanks, Tim! > > On Mon, Feb 4, 2019 at 4:53 AM Tim Armstrong > wrote

Does anyone use make_{asan,debug,impala,release}.sh?

2019-01-25 Thread Tim Armstrong
Currently one of the biggest pain points with the build system is the number of layered shell scripts that interact with each other in non-obvious ways. The easiest way to simplify the situation, to start off with, is to delete or deprecate some of them as entry points. I deleted some in

  1   2   3   >