[jira] [Created] (IMPALA-6263) Assert hit during service restart Mutex.cpp:130: apache::thrift::concurrency::Mutex::impl::~impl(): Assertion `ret == 0' failed

2017-11-30 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6263: --- Summary: Assert hit during service restart Mutex.cpp:130: apache::thrift::concurrency::Mutex::impl::~impl(): Assertion `ret == 0' failed Key: IMPALA-6263 URL:

[jira] [Created] (IMPALA-6238) Add source and destination hosts to TErrorCode::DATASTREAM_SENDER_TIMEOUT

2017-11-23 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6238: --- Summary: Add source and destination hosts to TErrorCode::DATASTREAM_SENDER_TIMEOUT Key: IMPALA-6238 URL: https://issues.apache.org/jira/browse/IMPALA-6238

[jira] [Created] (IMPALA-6347) Monitor queue depth size for outgoing RPCs for Reactor threads

2017-12-19 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6347: --- Summary: Monitor queue depth size for outgoing RPCs for Reactor threads Key: IMPALA-6347 URL: https://issues.apache.org/jira/browse/IMPALA-6347 Project: IMPALA

[jira] [Created] (IMPALA-6218) Crash due to memory allocation failure on secure cluster

2017-11-17 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6218: --- Summary: Crash due to memory allocation failure on secure cluster Key: IMPALA-6218 URL: https://issues.apache.org/jira/browse/IMPALA-6218 Project: IMPALA

[jira] [Created] (IMPALA-6294) Concurrent hung with lots of fragments and spilling make slow progress due to blocking in DataStreamRecvr and DataStreamSender

2017-12-07 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6294: --- Summary: Concurrent hung with lots of fragments and spilling make slow progress due to blocking in DataStreamRecvr and DataStreamSender Key: IMPALA-6294 URL:

[jira] [Created] (IMPALA-6285) Avoid printing the stack as part of DoTransmitDataRpc as it leads to burning lots of kernel CPU

2017-12-06 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6285: --- Summary: Avoid printing the stack as part of DoTransmitDataRpc as it leads to burning lots of kernel CPU Key: IMPALA-6285 URL:

[jira] [Created] (IMPALA-7020) Order by expressions in Analytical functions are not materialized causing severe slow down

2018-05-13 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-7020: --- Summary: Order by expressions in Analytical functions are not materialized causing severe slow down Key: IMPALA-7020 URL: https://issues.apache.org/jira/browse/IMPALA-7020

[jira] [Created] (IMPALA-7015) Insert into Kudu table returns with Status OK even if there are Kudu errors

2018-05-11 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-7015: --- Summary: Insert into Kudu table returns with Status OK even if there are Kudu errors Key: IMPALA-7015 URL: https://issues.apache.org/jira/browse/IMPALA-7015

[jira] [Created] (IMPALA-7037) Create table if not exists "foo" loads metadata for the table

2018-05-15 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-7037: --- Summary: Create table if not exists "foo" loads metadata for the table Key: IMPALA-7037 URL: https://issues.apache.org/jira/browse/IMPALA-7037 Project: IMPALA

[jira] [Created] (IMPALA-7172) Statestore should verify that all subscribers are running the same version of Impala

2018-06-14 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-7172: --- Summary: Statestore should verify that all subscribers are running the same version of Impala Key: IMPALA-7172 URL: https://issues.apache.org/jira/browse/IMPALA-7172

[jira] [Resolved] (IMPALA-2937) Introduce sampled statistics

2018-06-15 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar resolved IMPALA-2937. - Resolution: Fixed Fix Version/s: Impala 2.12.0 > Introduce sampled statistics >

[jira] [Created] (IMPALA-6354) Consider using Guava LoadingCache to cache metadata objects opposed to a ConcurrentHashMap

2017-12-27 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6354: --- Summary: Consider using Guava LoadingCache to cache metadata objects opposed to a ConcurrentHashMap Key: IMPALA-6354 URL: https://issues.apache.org/jira/browse/IMPALA-6354

[jira] [Created] (IMPALA-6425) Change Mempool memory allocation size to be <1MB to avoid allocating from CentralFreeList

2018-01-19 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6425: --- Summary: Change Mempool memory allocation size to be <1MB to avoid allocating from CentralFreeList Key: IMPALA-6425 URL: https://issues.apache.org/jira/browse/IMPALA-6425

[jira] [Created] (IMPALA-6406) Consider partitioning metastoreDdlLock_ by database

2018-01-16 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6406: --- Summary: Consider partitioning metastoreDdlLock_ by database Key: IMPALA-6406 URL: https://issues.apache.org/jira/browse/IMPALA-6406 Project: IMPALA

[jira] [Created] (IMPALA-6396) KrpcDataStreamRecvr should correctly report peak memory in query profile and summary

2018-01-13 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6396: --- Summary: KrpcDataStreamRecvr should correctly report peak memory in query profile and summary Key: IMPALA-6396 URL: https://issues.apache.org/jira/browse/IMPALA-6396

[jira] [Created] (IMPALA-6359) Blocking in Service threads due to tcmalloc contention due to memory allocations for RowBatch::Deserialize

2018-01-02 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6359: --- Summary: Blocking in Service threads due to tcmalloc contention due to memory allocations for RowBatch::Deserialize Key: IMPALA-6359 URL:

[jira] [Created] (IMPALA-6364) Lock contention in FileHandleCache results in >2x slowdown for remote HDFS reads

2018-01-03 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6364: --- Summary: Lock contention in FileHandleCache results in >2x slowdown for remote HDFS reads Key: IMPALA-6364 URL: https://issues.apache.org/jira/browse/IMPALA-6364

[jira] [Created] (IMPALA-6652) KRPC : Data Stream Manager Deferred RPCs in memz page isn't correct

2018-03-13 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6652: --- Summary: KRPC : Data Stream Manager Deferred RPCs in memz page isn't correct Key: IMPALA-6652 URL: https://issues.apache.org/jira/browse/IMPALA-6652 Project:

[jira] [Created] (IMPALA-6683) Restarting the Catalog without restarting Impalad and SS can block topic updates

2018-03-15 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6683: --- Summary: Restarting the Catalog without restarting Impalad and SS can block topic updates Key: IMPALA-6683 URL: https://issues.apache.org/jira/browse/IMPALA-6683

[jira] [Created] (IMPALA-6684) Large payload going through exchange results in lots of untracked memory in the form of a string

2018-03-15 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6684: --- Summary: Large payload going through exchange results in lots of untracked memory in the form of a string Key: IMPALA-6684 URL:

[jira] [Created] (IMPALA-6671) Metadata operations that modify a table blocks topic updates for other unrelated operations

2018-03-14 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6671: --- Summary: Metadata operations that modify a table blocks topic updates for other unrelated operations Key: IMPALA-6671 URL: https://issues.apache.org/jira/browse/IMPALA-6671

[jira] [Created] (IMPALA-6657) Investigate why memory allocation in Exchange receiver node takes a long time

2018-03-13 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6657: --- Summary: Investigate why memory allocation in Exchange receiver node takes a long time Key: IMPALA-6657 URL: https://issues.apache.org/jira/browse/IMPALA-6657

[jira] [Created] (IMPALA-6892) CheckHashAndDecrypt doesn't report disk and host where the verification failed

2018-04-19 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6892: --- Summary: CheckHashAndDecrypt doesn't report disk and host where the verification failed Key: IMPALA-6892 URL: https://issues.apache.org/jira/browse/IMPALA-6892

[jira] [Created] (IMPALA-6746) Reduce the number of comparison for analytical functions with partitioning when incoming data is clustered

2018-03-26 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6746: --- Summary: Reduce the number of comparison for analytical functions with partitioning when incoming data is clustered Key: IMPALA-6746 URL:

[jira] [Created] (IMPALA-6785) Staring an Impalad on an already running cluster may result in inconsistent cluster subscription

2018-03-31 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6785: --- Summary: Staring an Impalad on an already running cluster may result in inconsistent cluster subscription Key: IMPALA-6785 URL:

[jira] [Created] (IMPALA-6786) List of backends doesn't get updated for dedicated coordinators

2018-04-01 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6786: --- Summary: List of backends doesn't get updated for dedicated coordinators Key: IMPALA-6786 URL: https://issues.apache.org/jira/browse/IMPALA-6786 Project:

[jira] [Created] (IMPALA-6770) Catalog service should retry connecting to Sentry similar to HMS

2018-03-29 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6770: --- Summary: Catalog service should retry connecting to Sentry similar to HMS Key: IMPALA-6770 URL: https://issues.apache.org/jira/browse/IMPALA-6770 Project:

[jira] [Created] (IMPALA-6792) Appears to be a memory leak in orphaned fragments

2018-04-02 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6792: --- Summary: Appears to be a memory leak in orphaned fragments Key: IMPALA-6792 URL: https://issues.apache.org/jira/browse/IMPALA-6792 Project: IMPALA

[jira] [Created] (IMPALA-6744) Inline codegend_compare_fn_ to avoid per row memory loads and function call

2018-03-26 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6744: --- Summary: Inline codegend_compare_fn_ to avoid per row memory loads and function call Key: IMPALA-6744 URL: https://issues.apache.org/jira/browse/IMPALA-6744

[jira] [Created] (IMPALA-6692) When partition exchange is followed by sort each sort node becomes a synchronization point across the cluster

2018-03-16 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-6692: --- Summary: When partition exchange is followed by sort each sort node becomes a synchronization point across the cluster Key: IMPALA-6692 URL:

[jira] [Created] (IMPALA-7221) While reading from object store S3/ADLS at fast rates +500MB/sec TypeArrayKlass::allocate_common becomes a CPU bottleneck

2018-06-28 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-7221: --- Summary: While reading from object store S3/ADLS at fast rates +500MB/sec TypeArrayKlass::allocate_common becomes a CPU bottleneck Key: IMPALA-7221 URL:

[jira] [Resolved] (IMPALA-2712) Investigate increasing batch size

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar resolved IMPALA-2712. - Resolution: Won't Fix > Investigate increasing batch size >

[jira] [Resolved] (IMPALA-6657) Investigate why memory allocation in Exchange receiver node takes a long time

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar resolved IMPALA-6657. - Resolution: Fixed Turns out tcmalloc is reason, issue fixe in 2.12 > Investigate why

[jira] [Resolved] (IMPALA-2682) Address exchange operator CPU bottlenecks in Thrift

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar resolved IMPALA-2682. - Resolution: Fixed Fixed in 2.13 > Address exchange operator CPU bottlenecks in Thrift

[jira] [Commented] (IMPALA-6897) Catalog server should flag tables with large number of small files

2018-05-10 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-6897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16470688#comment-16470688 ] Mostafa Mokhtar commented on IMPALA-6897: - The Catalog has a section for Top tables in terms of

[jira] [Assigned] (IMPALA-7020) Order by expressions in Analytical functions are not materialized causing slowdown

2018-05-13 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar reassigned IMPALA-7020: --- Assignee: Alexander Behm > Order by expressions in Analytical functions are not

[jira] [Created] (IMPALA-7021) Impala query fails with BlockMissingException while HDFS throws SSLHandshakeException

2018-05-13 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-7021: --- Summary: Impala query fails with BlockMissingException while HDFS throws SSLHandshakeException Key: IMPALA-7021 URL: https://issues.apache.org/jira/browse/IMPALA-7021

[jira] [Commented] (IMPALA-7015) Insert into Kudu table returns with Status OK even if there are Kudu errors

2018-05-11 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16472302#comment-16472302 ] Mostafa Mokhtar commented on IMPALA-7015: - [~tmarsh] FYI > Insert into Kudu table returns with

[jira] [Commented] (IMPALA-6311) Evaluate smaller FPP for Bloom filters

2018-05-22 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486665#comment-16486665 ] Mostafa Mokhtar commented on IMPALA-6311: - [~jbapple] Yes, 75% FPP for a bloom filter is way

[jira] [Commented] (IMPALA-6311) Evaluate smaller FPP for Bloom filters

2018-05-22 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-6311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16485741#comment-16485741 ] Mostafa Mokhtar commented on IMPALA-6311: - Actually I was thinking of reducing the default

[jira] [Commented] (IMPALA-7096) Confirm that IMPALA-4835 does not increase chance of OOM for scans of wide tables

2018-06-12 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16510283#comment-16510283 ] Mostafa Mokhtar commented on IMPALA-7096: - [~tarmstrong] Attached OOM profile where most memory

[jira] [Created] (IMPALA-7172) Statestore should verify that all subscribers are running the same version of Impala

2018-06-14 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-7172: --- Summary: Statestore should verify that all subscribers are running the same version of Impala Key: IMPALA-7172 URL: https://issues.apache.org/jira/browse/IMPALA-7172

[jira] [Resolved] (IMPALA-2937) Introduce sampled statistics

2018-06-15 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar resolved IMPALA-2937. - Resolution: Fixed Fix Version/s: Impala 2.12.0 > Introduce sampled statistics >

[jira] [Commented] (IMPALA-2682) Address exchange operator CPU bottlenecks in Thrift

2018-06-11 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16508701#comment-16508701 ] Mostafa Mokhtar commented on IMPALA-2682: - To get more speedup in exchange we should focus on: *

[jira] [Assigned] (IMPALA-6746) Reduce the number of comparison for analytical functions with partitioning when incoming data is clustered

2018-06-26 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar reassigned IMPALA-6746: --- Assignee: Adrian Ng (was: Mostafa Mokhtar) > Reduce the number of comparison for

[jira] [Assigned] (IMPALA-7020) Order by expressions in Analytical functions are not materialized causing slowdown

2018-06-26 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar reassigned IMPALA-7020: --- Assignee: Adrian Ng (was: Alexander Behm) > Order by expressions in Analytical

[jira] [Assigned] (IMPALA-6746) Reduce the number of comparison for analytical functions with partitioning when incoming data is clustered

2018-06-26 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-6746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar reassigned IMPALA-6746: --- Assignee: Mostafa Mokhtar (was: Tianyi Wang) > Reduce the number of comparison

[jira] [Commented] (IMPALA-2945) Low memory estimation in local aggregation

2018-05-02 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-2945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16461594#comment-16461594 ] Mostafa Mokhtar commented on IMPALA-2945: - I disagree, the planner and admission control should

[jira] [Created] (IMPALA-7221) While reading from object store S3/ADLS at fast rates +500MB/sec TypeArrayKlass::allocate_common becomes a CPU bottleneck

2018-06-28 Thread Mostafa Mokhtar (JIRA)
Mostafa Mokhtar created IMPALA-7221: --- Summary: While reading from object store S3/ADLS at fast rates +500MB/sec TypeArrayKlass::allocate_common becomes a CPU bottleneck Key: IMPALA-7221 URL:

[jira] [Updated] (IMPALA-7221) While reading from object store S3/ADLS at fast rates +500MB/sec TypeArrayKlass::allocate_common becomes a CPU bottleneck

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar updated IMPALA-7221: Attachment: s3_alloc_expensive_2_ps.txt s3_alloc_expensive_1_js.txt >

[jira] [Updated] (IMPALA-7221) While reading from object store S3/ADLS at +500MB/sec TypeArrayKlass::allocate_common becomes a CPU bottleneck

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar updated IMPALA-7221: Summary: While reading from object store S3/ADLS at +500MB/sec

[jira] [Assigned] (IMPALA-7221) While reading from object store S3/ADLS at fast rates +500MB/sec TypeArrayKlass::allocate_common becomes a CPU bottleneck

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-7221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar reassigned IMPALA-7221: --- Assignee: Sailesh Mukil > While reading from object store S3/ADLS at fast rates

[jira] [Resolved] (IMPALA-2712) Investigate increasing batch size

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-2712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar resolved IMPALA-2712. - Resolution: Won't Fix > Investigate increasing batch size >

[jira] [Assigned] (IMPALA-2564) Introduce mechanism to limit query fan-out

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-2564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar reassigned IMPALA-2564: --- Assignee: (was: Mostafa Mokhtar) > Introduce mechanism to limit query fan-out

[jira] [Resolved] (IMPALA-6657) Investigate why memory allocation in Exchange receiver node takes a long time

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar resolved IMPALA-6657. - Resolution: Fixed Turns out tcmalloc is reason, issue fixe in 2.12 > Investigate why

[jira] [Resolved] (IMPALA-2682) Address exchange operator CPU bottlenecks in Thrift

2018-06-28 Thread Mostafa Mokhtar (JIRA)
[ https://issues.apache.org/jira/browse/IMPALA-2682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mostafa Mokhtar resolved IMPALA-2682. - Resolution: Fixed Fixed in 2.13 > Address exchange operator CPU bottlenecks in Thrift