Re: Impala 4.x (and 3.x)

Do we plan to switch the default catalog implementation to the local catalog as well? On Wed, Jan 29, 2020 at 3:41 PM Tim Armstrong wrote: > We haven't generally had clear guarantees about what OS versions we're > going to support. I ran into issues with > https://issues.apa

Re: Impala 4.x (and 3.x)

ata a lot. > > On Wed, Feb 12, 2020 at 3:46 PM Tim Armstrong > wrote: > > > Do we plan to switch the default catalog implementation to the local > > catalog as well? > > > > On Wed, Jan 29, 2020 at 3:41 PM Tim Armstrong > > wrote: > > > > > We h

Re: Impala 4.x (and 3.x)

, and 20.04 is > expected later this spring. > > I wonder what the community's opinion is about SLES 12 (mentioned in > IMPALA-7273). > > We could also express our intention to support CentOS 8 and Ubuntu 20.04 > (when it becomes > available), but these are just additio

Re: [DISCUSS] Impala 3.4.0 release

rent count the following as blockers for Impala 3.4 until > > determined otherwise: > > https://issues.apache.org/jira/browse/IMPALA-9357 > > https://issues.apache.org/jira/browse/IMPALA-9351 > > https://issues.apache.org/jira/browse/IMPALA-8533 > > > > Thanks, >

Re: Impala 4.0 breaking changes

I think I generally support this. A few specific comments. > Proposal 3: Impala-lzo > Drop support for Impala-lzo/hadoop-lzo Does this mean dropping the plugin text scanner interface entirely? LZO is the only implementation of that that I'm aware of (and we rely on it to test the interface) so se

Re: Impala 4.0 breaking changes

On Tue, Mar 17, 2020 at 11:50 AM Tim Armstrong wrote: > I think I generally support this. A few specific comments. > > > Proposal 3: Impala-lzo > > Drop support for Impala-lzo/hadoop-lzo > > Does this mean dropping the plugin text scanner interface entirely? LZO is > t

Re: Impala 4.0 breaking changes

also consider bumping RUNTIME_FILTER_WAIT_TIME_MS to a higher value, since I think generally higher values have proven to be more robust for complex queries (TPC-DS, etc). On Tue, Mar 17, 2020 at 11:56 AM Tim Armstrong wrote: > > - Do we still need the DECIMAL_V2 query option? Seems like th

Re: Impala 4.0 breaking changes

I think ARM support can ship in whatever release it's reading in, since it's not a breaking change. On Wed, Mar 18, 2020 at 9:43 PM 赵 仁海 wrote: > Thanks > I will work hard on this ^_^ > > > 发件人: Jim Apple > 发送时间: 2020年3月19日 10:21 > 收件人: dev@impala.apache.org >

Re: [VOTE] 3.4.0 release candidate 2

+1 (binding) Did the following on my own system: - Verified asc signature and sha512 - Checked the tree hash - Build from the tarball, ran some queries, including TPC-DS queries. - Diffed the git checkout and the tarball Ran https://jenkins.impala.io/job/release-test-ub1604/34/ On

Re: Impala 4 Breaking Changes

An addendum to this - we're also considering whether to increase the minimum CPU version so that we can simplify development and focus more on optimising for the most common CPUs - see https://issues.apache.org/jira/browse/IMPALA-9690 Running on commodity hardware is important and we want people t

Re: apache impala jira + gerrit rights

Hi Zoltan, I added you as a contributor in JIRA. You should be able to set up your gerrit account yourself - you need a github account - see https://cwiki.apache.org/confluence/display/IMPALA/Using+Gerrit+to+submit+and+review+patches On Mon, Apr 27, 2020 at 9:21 AM Zoltan Garaguly wrote: > Hi,

Re: Access to Jira

Hi Akos, I added you as a contributor in the Jira project. Welcome! On Wed, Apr 29, 2020 at 10:35 AM Ákos Kovács wrote: > Hi! > > I would like to ask for access to Impala Jira to be able to assign a ticket > to myself. > My username is akovacs. > Thanks in advance! > > Best, > Akos >

Re: Apache Jira Permissions

Added you as a contributor. Welcome! On Thu, Apr 30, 2020 at 9:19 AM Martin Zink wrote: > Hi, > > I am working at Cloudera Impala Sustaining team, and would like to assign > to me the Impala issue > (https://issues.apache.org/jira/browse/IMPALA-452) which I > am currently working on, but I seem

Re: Impala 4.0 breaking changes

n these? On Thu, Mar 19, 2020 at 4:44 PM Tim Armstrong wrote: > I think ARM support can ship in whatever release it's reading in, since > it's not a breaking change. > > On Wed, Mar 18, 2020 at 9:43 PM 赵 仁海 wrote: > >> Thanks >> I will work hard on this ^_

Re: Impala 4.0 breaking changes

it seems to be relatively stable. > > > > On Thu, May 7, 2020 at 11:41 AM Tim Armstrong > > wrote: > > > > > I'm going to revive this thread. I thought of a few more defaults that > we > > > might want to change. These are default changes we (pu

Re: Who took impalad's memory

That is strange, I would expect TCMalloc's unmapped bytes to be *not* counted as part of the process RSS. We depend on TCMalloc's tcmalloc.aggressive_memory_decommit setting to release all freed memory to the OS. If the memory page size on your system is bigger than TCMalloc expects, or something a

Re: Who took impalad's memory

ssues/780 > By the way, is it necessary to disable MADV_FREE while the linux > kernel version is >= 4.5? If yes, I will add a patch to native-toolchain :) > > > Tim Armstrong 于2020年5月16日周六 上午9:21写道： > > > That is strange, I would expect TCMalloc's unmapped byt

Re: Impala 4.0 breaking changes

> > Btw, do we want to achieve consistent default behavior with an upstream > > > Hive version? > > > > > > That said, I think creating non-transactional tables is a good default. > > > Especially because Impala will probably support Hudi and Iceberg in the >

Re: Impala 4 Breaking Changes

Another thing that we're looking into is changing the encoded runtime profile representation to be more efficient - see https://issues.apache.org/jira/browse/IMPALA-9378. So that might be a default we will try to change in Impala 4.0. On Fri, Apr 24, 2020 at 12:37 PM Tim Armstrong wrote:

Re: jenkins.impala.io downtime

Thanks for taking care of this! On Sat, Aug 1, 2020 at 11:45 AM Laszlo Gaal wrote: > The service is back up and running, > > Thank you, > > - LaszloG > > On Sat, Aug 1, 2020 at 6:33 PM Laszlo Gaal > wrote: > > > The Impala public Jenkins instance will be taken offline for scheduled > > backup

Re: Precommit test failures in the last 24 hours caused by service misconfiguration

Thanks for letting us know Laszlo, we appreciate all your work on this. On Tue, Aug 11, 2020 at 11:00 AM Laszlo Gaal wrote: > Hi > > Precommit tests have failed consistently for about 24 hours, since about > 2020-08-10 14:30 UTC). The failures were caused by an infrastructure > change: I updated

Re: Run pre commit tests failed

The logs in /home/ubuntu/Impala/logs/data_loading/ might have some hints, e.g. /home/ubuntu/Impala/logs/data_loading/impalad.INFO On Fri, Aug 14, 2020 at 7:39 AM 何献青 wrote: > Hi, all, > When I run pre commit tests, it fails in starting impalad, anyone knows > how to fix it. > The failed info: >

Re: jenkins.impala.io going down for upgrade

Thanks for taking care of this Norbert! On Wed, Aug 19, 2020 at 1:29 AM Norbert Luksa wrote: > Hi, > > the upgrade is now complete. > Let me know if you see any issues. > > Thanks, > Norbert > > On Wed, Aug 19, 2020 at 10:05 AM Norbert Luksa > > wrote: > > > Hi, > > > > jenkins.impala.io will b

New committer - Aman Sinha

The Project Management Committee (PMC) for Apache Impala has invited Aman Sinha to become a committer and we are pleased to announce that they have accepted. Congratulations and welcome, Aman!

Re: Breaking change in impala-shell

Thanks for the update and for fixing this bit of legacy! On Tue, Sep 22, 2020 at 8:52 AM Thomas Tauber-Marshall < tmarsh...@cloudera.com> wrote: > Impala devs, > > There is a patch that has just gone into master which switches the default > protocol for impala-shell from beeswax to hiveserver2, a

Re: Discuss Impala ARM CI integration

I think we got stalled here. I never got anywhere with configuring commits@ to accept emails from openlab and not sure that it's possible. In order to get something going, my suggestion is to copy what Kudu did and have this go to a custom mailing list. If you follow the thread here it describes w

Re: Discuss Impala ARM CI integration

; all others are rejected", so no moderation messages > should show up. > > The form said it will be acted on in 24 hours and the PMC will be informed > when the mailing list is ready. > > Also, FWIW, aside from Docker, AWS supports some ARM EC2 types. > > On Thu, Oct 1,

Re: Discuss Impala ARM CI integration

; > > Tim, I made you and I moderators. You're welcome to decline. :-) > > > Volunteers are welcome, but I expect zero moderation work - I set the > > list > > > to "subscribers can post; all others are rejected", so no moderation > > > messag

Re: jenkins.impala.io maintenance

Thank you for the advance notice! On Mon, Oct 12, 2020 at 6:18 AM Quanlong Huang wrote: > Hi all, > > The public Jenkins instance will be taken offline for a short period of > time for maintenance and upgrades on Oct. 16 (China time). Mail will be > sent when the service is going to be offline.

New Apache Impala committer - Shant Hovsepian

The Project Management Committee (PMC) for Apache Impala has invited Shant Hovsepian to become a committer and we are pleased to announce that they have accepted. Congratulations and welcome, Shant!

Re: FE Compilation Error in KuduCatalogOpExecutor.java

Thanks for sharing this. I think your explanation makes sense - SNAPSHOT jars are really a disaster for build stability and reproducibility. On Sun, Oct 25, 2020 at 8:24 PM Quanlong Huang wrote: > Hi all, > > I just noticed the following compilation error when rebasing to the current > master br

Re: Queries Stuck on Failed HDFS Calls and not Timing out

https://issues.apache.org/jira/browse/IMPALA-7738 can mitigate this to a large degree (it will cancel the query and log the error if the timeout is exceeded). It is not a 100% solution since the thread pool is fixed in size and can in theory get exhausted. On Sun, Nov 1, 2020 at 9:18 PM hexianqing

Re: [Black|White]List rename to [Ban|Allow]

Makes sense to me. We should keep current users in mind and allow a graceful migration but don't see any reason why we need to stick with these terms. On Mon, Nov 9, 2020 at 5:51 PM skyyws wrote: > +1 > > > > > On 11/9/2020 23:00，Shant Hovsepian wrote： > Hi All, > > Any thoughts on more inclusiv

Re: why impala memory layout is row base

One of the main problems that a columnar internal row batch format solves is interpretation overhead - if you switch based on type/expression/etc once per value, it is quite slow. But if you switch once per vector of values, the overhead is minimized. Impala uses LLVM query compilation to solve th

Re: 3.4.1 branch

We did manage to switch to ASF Kudu master via native-toolchain by default, but that was probably the easiest switch. I don't think we've tried pinning to Kudu release for our official release, but it's probably doable. I think the main concern would be is if there wasn't a Kudu release available w

New committer - Abhishek Rawat

The Project Management Committee (PMC) for Apache Impala has invited Abhishek Rawat to become a committer and we are pleased to announce that they have accepted. Congratulations and welcome, Abhishek!

Re: jenkins.impala.io maintenance

Agreed, thank you! On Wed, Jan 13, 2021 at 3:40 PM Laszlo Gaal wrote: > Thank you for taking care of this, Thomas! > > - LaszloG > > On Wed, Jan 13, 2021 at 7:52 PM Thomas Tauber-Marshall < > tmarsh...@cloudera.com> wrote: > > > The update has been applied and Jenkins should be available again

Re: why tuple memory need sorted by solt size

I think it made more sense when we tried to pad the tuple so that slots were aligned, since it reduced the amount of padding. But now we don't add any padding the order of slots doesn't affect the tuple size. On amd64 I don't think we've tended to see any real penalty from unaligned accesses - it's

Re: Switching Jenkins to Ubuntu 20.04?

I started developing on Ubuntu 20.04 mid-way through last year and didn't have any major problems. It's always possible the job scripts have some dependency or that the timing or resource consumption of the tests will change, but I would guess it's not a big change. On Sat, 3 Apr 2021 at 12:03, Ji

Re: Impala 4 Breaking Changes

; > huangquanl...@gmail.com> > > > wrote: > > > > > > > Reviving this thread. I think it's time to create the 4.0.0 branch > and > > > > prepare for the release now. Any breaking change is landing and we > > should > > > > wai

Re: Impala 2.9.0 CDH5.12.1 Build

That's an old unmaintained branch. There were a lot of Python-related build fixes over the last few years to deal with more recent versions of python and changes to the pypi infrastructure, but I don't immediately recognise this issue. Why do you want to build that version, out of curiousity? On

Re: please add the permission to assign a jira

I went ahead and added you On Fri, 14 May 2021 at 04:09, yong.sunny wrote: > Hi impala dev, > > > Please assign me the permission to assign a jira to myself. I am going to > handle jira:https://issues.apache.org/jira/browse/IMPALA-10695 > I am the reporter: > Reporter: Yong Yang > > > Thanks and

Re: Apache Impala contributor permission

I added you as a contributor in apache JIRA. Let me know if that solves it for you. On Thu, 27 May 2021 at 12:40, Kishen Das wrote: > Dear All, > > I have created this Jira -> > https://issues.apache.org/jira/browse/IMPALA-10721 and have sent a PR for > the same -> https://gerrit.cloudera.org/#

Re: Impala Iceberg partition DDL syntax

FWIW it looks like Spark supports multiple syntaxes for Parquet at least - both "using" and "stored as" - https://spark.apache.org/docs/2.3.1/sql-programming-guide.html. I don't know if that works the same for iceberg. Agree that the Spark SQL syntax is reasonable and it probably makes Iceberg ado

Re: New PMC member: Vihang Karajgaonkar

Congrats! On Sun, Jun 6, 2021, 06:44 Shant Hovsepian wrote: > Woohoo! Congrats Vihang! > > On Sun, Jun 6, 2021, 5:04 AM skyyws wrote: > > > Congrats Vihang! > > > > > > > > > > On 06/5/2021 06:30，Vihang Karajgaonkar wrote： > > Thank you everyone! It is really great to be part of this wonderful

Re: New committer - Wenzhe Zhou

Congrats Wenzhe, great to have you as a committer On Fri, 9 Jul 2021 at 09:05, Fang-Yu Rao wrote: > Congrats, Wenzhe! > > On Fri, Jul 9, 2021 at 8:51 AM Quanlong Huang > wrote: > > > Congratulations, Wenzhe! > > > > > > On Fri, Jul 9, 2021 at 11:13 PM Joe McDonnell > > > wrote: > > > > > Congr

Re: [ANNOUNCE] Apache Impala 4.0.0 release

This is a big deal! So much good stuff in the release. On Mon, 12 Jul 2021 at 07:24, Zoltán Borók-Nagy wrote: > Great job! Thanks a lot for driving this, Quanlong! > > > On Mon, Jul 12, 2021 at 4:07 PM Quanlong Huang > wrote: > > > The Apache Impala team is pleased to announce the release of Im

Re: Discussion about IMPALA-10789: Early materialize expressions in ScanNode

I believe that this optimisation could be very beneficial. It is difficult to cost - late materialisation can be a lot cheaper if there are predicates. Pruning columns would be very useful for some queries just on its own. We tried to implement a related optimisation - https://issues.apache.org/ji

Re: Re: Discussion about IMPALA-10789: Early materialize expressions in ScanNode

expression after the > predicates is executed. For example,select count(expr), sum(expr) from > (select a+b+c expr from T where a>10) T where T.expr>201. Scan > column a and execute a>10, then scan b, c2. Then Calculate a + b + > c and execute expr>20 > At 2021-07-20 12:14:18

Re: Untracked memory is huge like memory leak

Nice catch! This definitely seems like an issue we should fix. On Sat, 31 Jul 2021 at 02:17, Xianqing wrote: > > > Hi all, > An abnormal connection to port 21050 may cause a large untracked memory. > The reason is that the Thrift protocol parses messages sent by abnormal > connections as strings

Re: Proposal for IMPALA-9873: Avoid materialisation in filtered out tuples

This makes a lot of sense and I think works out some of the details that we hadn't figured out earlier. On Fri, 6 Aug 2021 at 04:46, Amogh Margoor wrote: > Hi, > As a part of Late materialization, Tim and Abhisekh had few ideas for > IMPALA-9873 to prevent materialization of filtered out tuples.

Re: UNNEST() operation for arrays

The syntax/behaviour seems kinda unusual to me - do any other systems aside from postgres implement this? UNNEST() in the from clause seems more common, and it is generally not consistent with the rest of SQL for SELECT clause entries to affect the set of rows returned. I'm not sure but it seems l

Re: Re: UNNEST() operation for arrays

> > > https://stackoverflow.com/questions/33220916/explode-transpose-multiple-columns-in-spark-sql-table > > >https://spark.apache.org/docs/latest/api/sql/index.html#arrays_zip > > > > > >On a separate note, what about using the same name(explode) as Hive? I >

Re: Re: UNNEST() operation for arrays

a kind > of TableRef (e.g. UnnestTableRef?, ZippingUnnestTableRef) but there I found > that each TableRef has a single rawPath_ and this might not be the best for > a single TableRef descendant holding multiple arrays with multiple > rawPath_s. But this is not a big issue, I'm sure

Re: New committer - Amogh Margoor

Congratulations! On Wed, 24 Nov 2021 at 10:36, Vihang Karajgaonkar wrote: > Congrats Amogh! > > On Wed, Nov 24, 2021 at 8:39 AM David Rorke wrote: > > > Congrats Amogh! > > > > On Wed, Nov 24, 2021 at 8:31 AM Wenzhe Zhou wrote: > > > > > Congratulations Amogh! > > > > > > Wenzhe Zhou > > > wz.

Re: Re: Re: New committer - Riza Suminto

Congrats Riza! On Tue, 31 May 2022 at 09:18, Fang-Yu Rao wrote: > Congrats Riza! > > On Tue, May 31, 2022 at 8:15 AM Andrew Sherman > wrote: > > > Well deserved Riza! > > > > On Tue, May 31, 2022 at 8:04 AM Yida Wu wrote: > > > > > Congratulations, Riza > > > > > > On Tue, May 31, 2022 at 7:58

Re: New PMC member: Tamás Máté

Congrats Tamas! On Mon, 11 Jul 2022 at 08:53, Xiaoqing Gao <285814...@qq.com> wrote: > Congratulations! > > -- Original -- > From: Zoram Thanga Date: Mon,Jul 11,2022 9:49 PM > To: dev Subject: Re: New PMC member: Tamás Máté > > > > Congratulations, Tamas! > > On

Re: Apache Jira Account

Jim Apple replied but it looks like you're probably not signed up to the mailing list: > What do you want your username to be? On Wed, 23 Nov 2022 at 09:53, Jason Fehr wrote: > Following up on this request. Per this Apache blog >

Re: Apache Jira Account

That's also unfortunate that they disabled public signups, what a pain. On Wed, 23 Nov 2022 at 09:55, Tim Armstrong wrote: > Jim Apple replied but it looks like you're probably not signed up to the > mailing list: > > What do you want your username to be? > > On Wed,

Re: New PMC member: Daniel Becker

Congrats Daniel! On Thu, 19 Jan 2023 at 06:18, Noémi Pap-Takács wrote: > Congratulations, Daniel! > > On Thu, Jan 19, 2023 at 12:02 PM Laszlo Gaal > wrote: > > > Congrats, Daniel, > > > > way to go! > > > > On Thu, Jan 19, 2023 at 2:21 AM Quanlong Huang > > wrote: > > > > > Congratulations, Da

Re: New PMC member: Andrew Sherman

Congrats! On Fri, 21 Apr 2023 at 13:34, Deepti Sehrawat wrote: > Congratulations andrew!! > > On Fri, Apr 21, 2023 at 1:32 PM Laszlo Gaal > wrote: > > > Congratulations, Andrew! > > > > On Fri, Apr 21, 2023 at 10:31 PM Michael Smith < > michael.sm...@cloudera.com > > > > > wrote: > > > > > Cong

Re: New committer - Kurt Deschler

Congrats Kurt! On Thu, 13 Jul 2023 at 20:23, Wenzhe Zhou wrote: > Congratulations. > > Wenzhe Zhou > wz...@cloudera.com > 408-568-0101 > > > On Fri, Jul 14, 2023 at 10:58 AM Yida Wu wrote: > > > Congratulations, Kurt!! > > > > On Thu, Jul 13, 2023 at 10:49 PM Joe McDonnell < > joemcdonn...@clou

[hs2client-CR] Add contribution guidelines

Tim Armstrong has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11189 Change subject: Add contribution guidelines .. Add contribution guidelines Change-Id: Iabf89b0f2d5246adc61f7d0f0e1c337561548379 --- M

[hs2client-CR] Add contribution guidelines

Tim Armstrong has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/11189 ) Change subject: Add contribution guidelines .. Add contribution guidelines Change-Id

[hs2client-CR] Add contribution guidelines

Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11189 ) Change subject: Add contribution guidelines .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/11189 To

[hs2client-CR] Add contribution guidelines

Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11189 ) Change subject: Add contribution guidelines .. Patch Set 1: Ah, great - this didn't replicate because it's out of sync w

<    1   2   3