[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "

[Alexander Behm (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Behm updated IMPALA-7027:
---
Description: 
If we have multiple cast of '' to varchar statements in a impala query which 
has a distinct like below, the query breaks for scenario when the cast to 
varchar limit in the SQL is lower than the previous cast.

 

Query 1> Fails with " AnalysisException: null CAUSED BY: 
IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"

SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as 
VARCHAR(100))  as CL_USER_ID FROM tablename limit 1

Where as the below query succeeds

Query 2> Success

 SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as 
VARCHAR(101))  as CL_USER_ID FROM  tablename limit 1

*Workaround*
SET ENABLE_EXPR_REWRITES=false;

  was:
If we have multiple cast of '' to varchar statements in a impala query which 
has a distinct like below, the query breaks for scenario when the cast to 
varchar limit in the SQL is lower than the previous cast.

 

Query 1> Fails with " AnalysisException: null CAUSED BY: 
IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"

SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as 
VARCHAR(100))  as CL_USER_ID FROM tablename limit 1

Where as the below query succeeds

Query 2> Success

 SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as 
VARCHAR(101))  as CL_USER_ID FROM  tablename limit 1

The Query succeeds is we remove the distinct clause or add set 
enable_expr_rewrites=false;


> Multiple Cast to Varchar with different limit fails with "AnalysisException: 
> null CAUSED BY: IllegalArgumentException: "
> 
>
> Key: IMPALA-7027
> URL: https://issues.apache.org/jira/browse/IMPALA-7027
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Meenakshi
>Priority: Critical
>  Labels: planner, regression
>
> If we have multiple cast of '' to varchar statements in a impala query which 
> has a distinct like below, the query breaks for scenario when the cast to 
> varchar limit in the SQL is lower than the previous cast.
>  
> Query 1> Fails with " AnalysisException: null CAUSED BY: 
> IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"
> SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as 
> VARCHAR(100))  as CL_USER_ID FROM tablename limit 1
> Where as the below query succeeds
> Query 2> Success
>  SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as 
> VARCHAR(101))  as CL_USER_ID FROM  tablename limit 1
> *Workaround*
> SET ENABLE_EXPR_REWRITES=false;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "

[Alexander Behm (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Behm updated IMPALA-7027:
---
Affects Version/s: (was: Impala 2.9.0)
   Impala 3.0
   Impala 2.12.0
   Labels: planner regression  (was: )

> Multiple Cast to Varchar with different limit fails with "AnalysisException: 
> null CAUSED BY: IllegalArgumentException: "
> 
>
> Key: IMPALA-7027
> URL: https://issues.apache.org/jira/browse/IMPALA-7027
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.0, Impala 2.12.0
>Reporter: Meenakshi
>Priority: Critical
>  Labels: planner, regression
>
> If we have multiple cast of '' to varchar statements in a impala query which 
> has a distinct like below, the query breaks for scenario when the cast to 
> varchar limit in the SQL is lower than the previous cast.
>  
> Query 1> Fails with " AnalysisException: null CAUSED BY: 
> IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"
> SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as 
> VARCHAR(100))  as CL_USER_ID FROM tablename limit 1
> Where as the below query succeeds
> Query 2> Success
>  SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as 
> VARCHAR(101))  as CL_USER_ID FROM  tablename limit 1
> The Query succeeds is we remove the distinct clause or add set 
> enable_expr_rewrites=false;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Created] (IMPALA-7030) crash in impala::PartitionedAggregationNode::ProcessBatchNoGrouping

[Michael Brown (JIRA)]

Michael Brown created IMPALA-7030:
-

 Summary:  crash in 
impala::PartitionedAggregationNode::ProcessBatchNoGrouping
 Key: IMPALA-7030
 URL: https://issues.apache.org/jira/browse/IMPALA-7030
 Project: IMPALA
  Issue Type: Bug
  Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Michael Brown
 Attachments: crash.dump.gz, gdb.out.gz, hs_err_pid1621.log.gz

https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2176/

{noformat}
#0  0x7fc896430428 in __GI_raise (sig=sig@entry=6) at 
../sysdeps/unix/sysv/linux/raise.c:54
#1  0x7fc89643202a in __GI_abort () at abort.c:89
#2  0x7fc899379c59 in os::abort(bool) (dump_core=) at 
/build/openjdk-8-wnL82d/openjdk-8-8u171-b11/src/hotspot/src/os/linux/vm/os_linux.cpp:1509
#3  0x7fc89952f047 in VMError::report_and_die() 
(this=this@entry=0x7fc7e90287d0) at 
/build/openjdk-8-wnL82d/openjdk-8-8u171-b11/src/hotspot/src/share/vm/utilities/vmError.cpp:1060
#4  0x7fc8993836ef in JVM_handle_linux_signal(int, siginfo_t*, void*, int) 
(sig=sig@entry=11, info=info@entry=0x7fc7e9028a70, 
ucVoid=ucVoid@entry=0x7fc7e9028940, 
abort_if_unrecognized=abort_if_unrecognized@entry=1)
at 
/build/openjdk-8-wnL82d/openjdk-8-8u171-b11/src/hotspot/src/os_cpu/linux_x86/vm/os_linux_x86.cpp:541
#5  0x7fc899376d88 in signalHandler(int, siginfo_t*, void*) (sig=11, 
info=0x7fc7e9028a70, uc=0x7fc7e9028940) at 
/build/openjdk-8-wnL82d/openjdk-8-8u171-b11/src/hotspot/src/os/linux/vm/os_linux.cpp:4432
#6  0x7fc8967d6390 in  () at 
/lib/x86_64-linux-gnu/libpthread.so.0
#7  0x7fc8584ca000 in 
impala::PartitionedAggregationNode::ProcessBatchNoGrouping(impala::RowBatch*) 
[clone .1] ()
#8  0x02cd5bcf in 
impala::PartitionedAggregationNode::Open(impala::RuntimeState*) 
(this=0x15795200, state=0x14e95d40) at 
/home/ubuntu/Impala/be/src/exec/partitioned-aggregation-node.cc:314
#9  0x01c94775 in impala::FragmentInstanceState::Open() 
(this=0x1cc19e00) at 
/home/ubuntu/Impala/be/src/runtime/fragment-instance-state.cc:268
#10 0x01c91faf in impala::FragmentInstanceState::Exec() 
(this=0x1cc19e00) at 
/home/ubuntu/Impala/be/src/runtime/fragment-instance-state.cc:81
#11 0x01ca175b in 
impala::QueryState::ExecFInstance(impala::FragmentInstanceState*) 
(this=0x3a1f6000, fis=0x1cc19e00) at 
/home/ubuntu/Impala/be/src/runtime/query-state.cc:401
#12 0x01c9ffce in impala::QueryState::::operator()(void) 
const (__closure=0x7fc7e9029ce8) at 
/home/ubuntu/Impala/be/src/runtime/query-state.cc:341
#13 0x01ca2479 in 
boost::detail::function::void_function_obj_invoker0,
 void>::invoke(boost::detail::function::function_buffer &) 
(function_obj_ptr=...)
at 
/home/ubuntu/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:153
#14 0x01bd9e58 in boost::function0::operator()() const 
(this=0x7fc7e9029ce0) at 
/home/ubuntu/Impala/toolchain/boost-1.57.0-p3/include/boost/function/function_template.hpp:767
#15 0x01ec50a9 in impala::Thread::SuperviseThread(std::string const&, 
std::string const&, boost::function, impala::ThreadDebugInfo const*, 
impala::Promise*) (name="exec-finstance 
(finst:844ac2a49a9a6486:396846db0005)", category="fragment-execution", 
functor=..., parent_thread_info=0x7fc7ea82c990, thread_started=0x7fc7ea82b940) 
at /home/ubuntu/Impala/be/src/util/thread.cc:356
#16 0x01ecd245 in boost::_bi::list5, 
boost::_bi::value, 
boost::_bi::value >::operator(), impala::ThreadDebugInfo 
const*, impala::Promise*), boost::_bi::list0>(boost::_bi::type, 
void (*&)(std::string const&, std::string const&, boost::function, 
impala::ThreadDebugInfo const*, impala::Promise*), boost::_bi::list0&, 
int) (this=0xb84afc0, f=@0xb84afb8: 0x1ec4d42 
, a=...)
at 
/home/ubuntu/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind.hpp:525
#17 0x01ecd169 in boost::_bi::bind_t, 
boost::_bi::value, 
boost::_bi::value > >::operator()() (this=0xb84afb8) at 
/home/ubuntu/Impala/toolchain/boost-1.57.0-p3/include/boost/bind/bind_template.hpp:20
#18 0x01ecd12c in boost::detail::thread_data

[jira] [Closed] (IMPALA-3813) How to handle replication factor while creating KUDU table through impala

[Alex Rodoni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Rodoni closed IMPALA-3813.
---
   Resolution: Fixed
Fix Version/s: Impala 3.1.0
   Impala 2.13.0

> How to handle replication factor while creating KUDU table through impala
> -
>
> Key: IMPALA-3813
> URL: https://issues.apache.org/jira/browse/IMPALA-3813
> Project: IMPALA
>  Issue Type: Bug
>  Components: Docs, Frontend
>Affects Versions: Kudu_Impala
> Environment: CDH 5.4.7
>Reporter: Ravi sharma
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: performance
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> while to create kudu table from impala shell
> facing below error
> ERROR:
> ImpalaRuntimeException: Error creating Kudu table
> CAUSED BY: MasterErrorException: Server[Kudu Master - 
> cdhnode1.infocepts.com:7051] INVALID_ARGUMENT[code 4]: Not enough live tablet 
> servers to create a table with the requested replication factor 3. 2 tablet 
> servers are alive.
> [cdhnode1.infocepts.com:21000] >
> Please tell me how can i limit the replication factor manually, since  i have 
> only 2 tablet servers.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Closed] (IMPALA-3813) How to handle replication factor while creating KUDU table through impala

[Alex Rodoni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-3813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Rodoni closed IMPALA-3813.
---
   Resolution: Fixed
Fix Version/s: Impala 3.1.0
   Impala 2.13.0

> How to handle replication factor while creating KUDU table through impala
> -
>
> Key: IMPALA-3813
> URL: https://issues.apache.org/jira/browse/IMPALA-3813
> Project: IMPALA
>  Issue Type: Bug
>  Components: Docs, Frontend
>Affects Versions: Kudu_Impala
> Environment: CDH 5.4.7
>Reporter: Ravi sharma
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: performance
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> while to create kudu table from impala shell
> facing below error
> ERROR:
> ImpalaRuntimeException: Error creating Kudu table
> CAUSED BY: MasterErrorException: Server[Kudu Master - 
> cdhnode1.infocepts.com:7051] INVALID_ARGUMENT[code 4]: Not enough live tablet 
> servers to create a table with the requested replication factor 3. 2 tablet 
> servers are alive.
> [cdhnode1.infocepts.com:21000] >
> Please tell me how can i limit the replication factor manually, since  i have 
> only 2 tablet servers.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation

[Tim Armstrong (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475099#comment-16475099
 ] 

Tim Armstrong commented on IMPALA-7025:
---

It could be because a different set of files was selected or because the files 
were slightly different sizes. Need to investigate to determine which, although 
maybe the ultimate fix is to allow ignoring the resource estimates in these 
tests

> PlannerTest.testTableSample has wrong mem-reservation
> -
>
> Key: IMPALA-7025
> URL: https://issues.apache.org/jira/browse/IMPALA-7025
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Joe McDonnell
>Assignee: Tim Armstrong
>Priority: Critical
>  Labels: broken-build, flaky
>
> Seen once on master exhaustive:
> {noformat}
> Error Message
> Section PLAN of query:
> select id from functional_parquet.alltypes tablesample system(10) 
> repeatable(1234)
> Actual does not match expected result:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB 
> thread-reservation=2
> ^^^
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=23.70KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable
> Expected:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB 
> thread-reservation=2
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=24.23KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable{noformat}
> This succeeded on the next build, so it is flaky and might not recur.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Created] (IMPALA-7029) LHS of a between expr is not cloned when rewritten into LHS >= x and LHS <= y

[Tianyi Wang (JIRA)]

Tianyi Wang created IMPALA-7029:
---

 Summary: LHS of a between expr is not cloned when rewritten into 
LHS >= x and LHS <= y
 Key: IMPALA-7029
 URL: https://issues.apache.org/jira/browse/IMPALA-7029
 Project: IMPALA
  Issue Type: Bug
  Components: Frontend
Affects Versions: Impala 3.1.0
Reporter: Tianyi Wang
 Fix For: Impala 2.13.0


When a between predicates is rewritten into two comparing exprs, the LHS 
obviously should be cloned but it's not:

https://github.com/apache/impala/blob/19bcc3099ef5e244fe74a0466b0b0eeb673acc8e/fe/src/main/java/org/apache/impala/rewrite/BetweenToCompoundRule.java#L46



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (IMPALA-7026) buildall.sh make shell fails due to sqlparse problem

[Fredy Wijaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fredy Wijaya resolved IMPALA-7026.
--
Resolution: Not A Bug

> buildall.sh make shell fails due to sqlparse problem
> 
>
> Key: IMPALA-7026
> URL: https://issues.apache.org/jira/browse/IMPALA-7026
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: Fredy Wijaya
>Priority: Blocker
>  Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2159/
> {noformat}
> 22:43:05 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.14
> 22:43:05 [ 47%] Building CXX object 
> be/src/udf_samples/CMakeFiles/udf-sample-test.dir/udf-sample-test.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/hdr-histogram.cc.o
> 22:43:05 [ 47%] Built target uda-sample-test
> 22:43:05 [ 47%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/impalad-metrics.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/exprs/CMakeFiles/Exprs.dir/math-functions-ir.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/testutil/CMakeFiles/TestUtil.dir/desc-tbl-builder.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/testutil/CMakeFiles/TestUtil.dir/fault-injection-util.cc.o
> 22:43:05 [ 47%] Linking CXX executable 
> ../../build/debug/udf_samples/udf-sample-test
> 22:43:05 [ 47%] Built target udf-sample-test
> 22:43:05 Scanning dependencies of target krpc
> 22:43:05 [ 47%] Building CXX object 
> be/src/exprs/CMakeFiles/Exprs.dir/null-literal.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/jni-util.cc.o
> 22:43:05 [ 48%] Building CXX object 
> be/src/testutil/CMakeFiles/TestUtil.dir/impalad-query-executor.cc.o
> 22:43:05 warning: no files found matching 'docs/Makefile'
> 22:43:05 warning: no files found matching 'Makefile'
> 22:43:05 [ 48%] Building CXX object 
> be/src/runtime/CMakeFiles/Runtime.dir/hbase-table.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/kudu/rpc/CMakeFiles/krpc.dir/acceptor_pool.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/testutil/CMakeFiles/TestUtil.dir/in-process-servers.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/kudu/rpc/CMakeFiles/krpc.dir/blocking_ops.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/kudu/rpc/CMakeFiles/krpc.dir/client_negotiation.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/runtime/CMakeFiles/Runtime.dir/hbase-table-factory.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/logging-support.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/mem-info.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/memory-metrics.cc.o
> 22:43:05 zip_safe flag not set; analyzing archive contents...
> 22:43:05 Cleaning up old build artifacts.
> 22:43:06 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.19
> 22:43:06 [ 51%] Building CXX object 
> be/src/runtime/CMakeFiles/Runtime.dir/hdfs-fs-cache.cc.o
> 22:43:06 [ 51%] Building CXX object 
> be/src/runtime/CMakeFiles/Runtime.dir/initial-reservations.cc.o
> 22:43:06 python: can't open file 'setup.py': [Errno 2] No such file or 
> directory
> 22:43:06 Error in shell/make_shell_tarball.sh at line 96: python setup.py -q 
> bdist_egg clean
> 22:43:06 CMakeFiles/shell_tarball.dir/build.make:57: recipe for target 
> 'CMakeFiles/shell_tarball' failed
> 22:43:06 make[2]: *** [CMakeFiles/shell_tarball] Error 2
> 22:43:06 CMakeFiles/Makefile2:141: recipe for target 
> 'CMakeFiles/shell_tarball.dir/all' failed
> 22:43:06 make[1]: *** [CMakeFiles/shell_tarball.dir/all] Error 2
> 22:43:06 make[1]: *** Waiting for unfinished jobs
> {noformat}
> I just reproduced this locally using {{./buildall.sh -notests 
> -start_minicluster -start_impala_cluster}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (IMPALA-4464) Remove remote_data_load.py

[David Knupp (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Knupp resolved IMPALA-4464.
-
Resolution: Fixed

> Remove remote_data_load.py
> --
>
> Key: IMPALA-4464
> URL: https://issues.apache.org/jira/browse/IMPALA-4464
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Infrastructure
>Affects Versions: Impala 2.8.0
>Reporter: David Knupp
>Assignee: David Knupp
>Priority: Major
>  Labels: remote_cluster_test
>
> A patch was recently submitted that allows data load and end-end tests to run 
> against a remote cluster. At its core was this file:
> https://github.com/apache/incubator-impala/blob/master/bin/remote_data_load.py
> However, while this script relies on several changes to existing build and 
> test scripts, nothing else in turns relies on it. In retrospect, it does not 
> make sense to have this script in the Impala repo if nothing can use it 
> externally.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Comment Edited] (IMPALA-7015) Insert into Kudu table returns with Status OK even if there are Kudu errors

[Thomas Tauber-Marshall (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474673#comment-16474673
 ] 

Thomas Tauber-Marshall edited comment on IMPALA-7015 at 5/14/18 7:42 PM:
-

So I agree with the point from IMPALA-3710 that we don't want these types of 
errors to prematurely terminate the query, which would happen if we just 
returned them immediately. -I think that it would be reasonable to just save 
the errors and return an error status once all of the rows have been sent to 
Kudu, eg in FlushFinal().-

edit: realized the above is wrong - even if we return the error from 
FlushFinal(), it could still cause other fragment instances to be cancelled. In 
theory, it would be possible to achieve the behavior of "run query to 
completion but then return an error", but it would require more significant 
changes.

Agreed that this whole error propagation issue is one of the biggest problems 
with Impala/Kudu integration at the moment, but not sure anyone has the time at 
the moment to fix it the right way.


was (Author: twmarshall):
So I agree with the point from IMPALA-3710 that we don't want these types of 
errors to prematurely terminate the query, which would happen if we just 
returned them immediately. I think that it would be reasonable to just save the 
errors and return an error status once all of the rows have been sent to Kudu, 
eg in FlushFinal().

We could then return an error message with the counts of rows that hit various 
errors. That wouldn't be as good as the more structured approach suggested in 
IMPALA-4416 and IMPALA-1789, since clients would have to parse the message, but 
it would at least be something, rather than just returning no info about what 
happened.

My main hesitation is that this would be a somewhat breaking change, and if 
we're going to do that it might be better to wait until everything is in place 
to do it the right way, rather than doing breaking changes around this twice.

> Insert into Kudu table returns with Status OK even if there are Kudu errors
> ---
>
> Key: IMPALA-7015
> URL: https://issues.apache.org/jira/browse/IMPALA-7015
> Project: IMPALA
>  Issue Type: Bug
>  Components: Backend
>Affects Versions: Impala 2.12.0
>Reporter: Mostafa Mokhtar
>Priority: Major
> Attachments: Insert into kudu profile with errors.txt
>
>
> DML statements against Kudu tables return status OK even if there are Kudu 
> errors.
> This behavior is misleading. 
> {code}
>   Summary:
> Session ID: 18430b000e5dd8dc:e3e5dadb4a15d4b4
> Session Type: BEESWAX
> Start Time: 2018-05-11 10:10:07.314218000
> End Time: 2018-05-11 10:10:07.434017000
> Query Type: DML
> Query State: FINISHED
> Query Status: OK
> Impala Version: impalad version 2.12.0-cdh5.15.0 RELEASE (build 
> 2f9498d5c2f980aa7ff9505c56654c8e59e026ca)
> User: mmokhtar
> Connected User: mmokhtar
> Delegated User: 
> Network Address: :::10.17.234.27:60760
> Default Db: tpcds_1000_kudu
> Sql Statement: insert into store_2 select * from store
> Coordinator: vd1317.foo:22000
> Query Options (set by configuration): 
> Query Options (set by configuration and planner): MT_DOP=0
> Plan: 
> {code}
> {code}
> Operator  #Hosts   Avg Time  Max Time  #Rows  Est. #Rows  Peak Mem  
> Est. Peak Mem  Detail
> -
> 02:PARTIAL SORT5  909.030us   1.025ms  1.00K   1.00K   6.14 MB
> 4.00 MB
> 01:EXCHANGE56.262ms   7.232ms  1.00K   1.00K  75.50 KB
>   0  KUDU(KuduPartition(tpcds_1000_kudu.store.s_store_sk)) 
> 00:SCAN KUDU   53.694ms   4.137ms  1.00K   1.00K   4.34 MB
>   0  tpcds_1000_kudu.store 
> Errors: Key already present in Kudu table 
> 'impala::tpcds_1000_kudu.store_2'. (1 of 1002 similar)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-7005) The 3.0 changelog page should mention the previous version

[Sailesh Mukil (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474698#comment-16474698
 ] 

Sailesh Mukil commented on IMPALA-7005:
---

[~arodoni_cloudera] 
https://cwiki.apache.org/confluence/display/IMPALA/How+to+Release

Step 15 in the above link should have what you need.

> The 3.0 changelog page should mention the previous version
> --
>
> Key: IMPALA-7005
> URL: https://issues.apache.org/jira/browse/IMPALA-7005
> Project: IMPALA
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: Impala 3.0
>Reporter: Lars Volker
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: website
>
> It's not clear to me from looking at the [3.0 
> changlog|http://impala.apache.org/docs/changelog-3.0.html] what base version 
> it compares to. Can we add a sentence to point out that these changes are in 
> comparison to 2.11 (are they?)?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-7028) TestKuduOperations.test_kudu_update fails due to wrong NumModifiedRows

[Joe McDonnell (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474677#comment-16474677
 ] 

Joe McDonnell commented on IMPALA-7028:
---

{noformat}
query_test/test_kudu.py:92: in test_kudu_update
self.run_test_case('QueryTest/kudu_update', vector, use_db=unique_database)
common/impala_test_suite.py:451: in run_test_case
verify_runtime_profile(test_section['RUNTIME_PROFILE'], 
result.runtime_profile)
common/test_result_verifier.py:590: in verify_runtime_profile
actual))
E   AssertionError: Did not find matches for lines in runtime profile:
E   EXPECTED LINES:
E   NumModifiedRows: 7300
E   
E   ACTUAL PROFILE:
E   Query (id=de4b7a0e99753cd5:6ffa44fd):
E DEBUG MODE WARNING: Query profile created while running a DEBUG build of 
Impala. Use RELEASE builds to measure query performance.
E Summary:
E   Session ID: 7f4a67a73c9ecf4e:5d51ba77392831b4
E   Session Type: BEESWAX
E   Start Time: 2018-05-12 15:08:10.372271000
E   End Time: 
E   Query Type: DML
E   Query State: FINISHED
E   Query Status: OK
E   Impala Version: impalad version 3.1.0-SNAPSHOT DEBUG (build 
e12ee485cf4c77203b144c053ee167509cc39374)
E   User: jenkins
E   Connected User: jenkins
E   Delegated User: 
E   Network Address: 127.0.0.1:55173
E   Default Db: test_kudu_update_628738b5
E   Sql Statement: update tdata set vali = -1
E   Coordinator: ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22000
E   Query Options (set by configuration): 
ABORT_ON_ERROR=1,DISABLE_CODEGEN=1,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0
E   Query Options (set by configuration and planner): 
ABORT_ON_ERROR=1,DISABLE_CODEGEN=1,EXEC_SINGLE_NODE_ROWS_THRESHOLD=0,MT_DOP=0,DISABLE_CODEGEN_ROWS_THRESHOLD=0
E   Plan: 
E   
E   Max Per-Host Resource Reservation: Memory=0B Threads=2
E   Per-Host Resource Estimates: Memory=10.00MB
E   WARNING: The following tables are missing relevant table and/or column 
statistics.
E   test_kudu_update_628738b5.tdata
E   
E   F00:PLAN FRAGMENT [RANDOM] hosts=3 instances=3
E   |  Per-Host Resources: mem-estimate=0B mem-reservation=0B 
thread-reservation=2
E   UPDATE KUDU [test_kudu_update_628738b5.tdata]
E   |  mem-estimate=0B mem-reservation=0B thread-reservation=0
E   |
E   00:SCAN KUDU [test_kudu_update_628738b5.tdata]
E  mem-estimate=0B mem-reservation=0B thread-reservation=1
E  tuple-ids=0 row-size=4B cardinality=unavailable
E   
E   Estimated Per-Host Mem: 10485760
E   Tables Missing Stats: test_kudu_update_628738b5.tdata
E   Per Host Min Memory Reservation: 
ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22000(0) 
ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22001(0) 
ec2-m2-4xlarge-centos-6-4-092a.vpc.cloudera.com:22002(0) 
E   Request Pool: default-pool
E   Admission result: Admitted immediately
E   Query Compilation: 6.606ms
E  - Metadata of all 1 tables cached: 1.087ms (1.087ms)
E  - Analysis finished: 1.457ms (369.234us)
E  - Value transfer graph computed: 1.508ms (51.577us)
E  - Single node plan created: 4.839ms (3.330ms)
E  - Runtime filters computed: 4.876ms (37.680us)
E  - Distributed plan created: 4.934ms (57.370us)
E  - Planning finished: 6.606ms (1.672ms)
E   Query Timeline: 84.999ms
E  - Query submitted: 999.994us (999.994us)
E  - Planning finished: 8.999ms (7.999ms)
E  - Submit for admission: 11.999ms (2.999ms)
E  - Completed admission: 11.999ms (0.000ns)
E  - Ready to start on 3 backends: 12.999ms (999.994us)
E  - All 3 execution backends (3 fragment instances) started: 24.999ms 
(11.999ms)
E  - Last row fetched: 39.999ms (14.999ms)
E  - Released admission control resources: 39.999ms (0.000ns)
E  - DML data written: 40.999ms (999.993us)
E  - DML Metastore update finished: 40.999ms (0.000ns)
E  - Request finished: 40.999ms (0.000ns)
E  - First row fetched: 82.999ms (41.999ms)
E- ComputeScanRangeAssignmentTimer: 0.000ns
E ImpalaServer:
E- ClientFetchWaitTimer: 0.000ns
E- MetastoreUpdateTimer: 0.000ns
E- RowMaterializationTimer: 0.000ns
E Execution Profile de4b7a0e99753cd5:6ffa44fd:(Total: 27.999ms, 
non-child: 0.000ns, % non-child: 0.00%)
E   Number of filters: 0
E   Filter routing table: 
EID  Src. Node  Tgt. Node(s)  Target type  Partition filter  Pending 
(Expected)  First arrived  Completed   Enabled
E   
---
E   
E   Backend startup latencies: Count: 3, min / max: 1ms / 11ms, 25th %-ile: 
1ms, 50th %-ile: 2ms, 75th %-ile: 2ms, 90th %-ile: 11ms, 95th %-ile: 11ms, 
99.9th %-ile: 11ms
E   Per Node Peak 

[jira] [Created] (IMPALA-7028) TestKuduOperations.test_kudu_update fails due to wrong NumModifiedRows

[Joe McDonnell (JIRA)]

Joe McDonnell created IMPALA-7028:
-

 Summary: TestKuduOperations.test_kudu_update fails due to wrong 
NumModifiedRows
 Key: IMPALA-7028
 URL: https://issues.apache.org/jira/browse/IMPALA-7028
 Project: IMPALA
  Issue Type: Bug
  Components: Backend
Affects Versions: Impala 3.1.0
Reporter: Joe McDonnell


Seen once on master exhaustive:
{noformat}
query_test/test_kudu.py:92: in test_kudu_update
self.run_test_case('QueryTest/kudu_update', vector, use_db=unique_database)
common/impala_test_suite.py:451: in run_test_case
verify_runtime_profile(test_section['RUNTIME_PROFILE'], 
result.runtime_profile)
common/test_result_verifier.py:590: in verify_runtime_profile
actual))
E   AssertionError: Did not find matches for lines in runtime profile:
E   EXPECTED LINES:
E   NumModifiedRows: 7300
...
# From the actual profile:
E   Partition: Default
E   NumModifiedRows: 32
E   NumRowErrors: 0{noformat}
Will add a comment with the profile.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation

[Joe McDonnell (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474672#comment-16474672
 ] 

Joe McDonnell commented on IMPALA-7025:
---

[~tarmstrong] This is on the 3.x branch

> PlannerTest.testTableSample has wrong mem-reservation
> -
>
> Key: IMPALA-7025
> URL: https://issues.apache.org/jira/browse/IMPALA-7025
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Joe McDonnell
>Assignee: Tim Armstrong
>Priority: Critical
>  Labels: broken-build, flaky
>
> Seen once on master exhaustive:
> {noformat}
> Error Message
> Section PLAN of query:
> select id from functional_parquet.alltypes tablesample system(10) 
> repeatable(1234)
> Actual does not match expected result:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB 
> thread-reservation=2
> ^^^
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=23.70KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable
> Expected:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB 
> thread-reservation=2
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=24.23KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable{noformat}
> This succeeded on the next build, so it is flaky and might not recur.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "

[Meenakshi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Meenakshi updated IMPALA-7027:
--
Description: 
If we have multiple cast of '' to varchar statements in a impala query which 
has a distinct like below, the query breaks for scenario when the cast to 
varchar limit in the SQL is lower than the previous cast.

 

Query 1> Fails with " AnalysisException: null CAUSED BY: 
IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"

SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as 
VARCHAR(100))  as CA_USER_ID FROM tablename limit 1

Where as the below query succeeds

Query 2> Success

 SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as 
VARCHAR(101))  as CA_USER_ID FROM  tablename limit 1

The Query succeeds is we remove the distinct clause or add set 
enable_expr_rewrites=false;

  was:
If we have multiple cast of '' to varchar statements in a impala query which 
has a distinct like below, the query breaks for scenario when the cast to 
varchar limit in the SQL is lower than the previous cast.

 

 Query 1> Fails with " AnalysisException: null CAUSED BY: 
IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"

SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as 
VARCHAR(100))  as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
limit 1

Where as the below query succeeds

Query 2> Success

 SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as 
VARCHAR(101))  as CA_USER_ID FROM  pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
limit 1

The Query succeeds is we remove the distinct clause or add set 
enable_expr_rewrites=false;


> Multiple Cast to Varchar with different limit fails with "AnalysisException: 
> null CAUSED BY: IllegalArgumentException: "
> 
>
> Key: IMPALA-7027
> URL: https://issues.apache.org/jira/browse/IMPALA-7027
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 2.9.0
>Reporter: Meenakshi
>Priority: Critical
>
> If we have multiple cast of '' to varchar statements in a impala query which 
> has a distinct like below, the query breaks for scenario when the cast to 
> varchar limit in the SQL is lower than the previous cast.
>  
> Query 1> Fails with " AnalysisException: null CAUSED BY: 
> IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"
> SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as 
> VARCHAR(100))  as CA_USER_ID FROM tablename limit 1
> Where as the below query succeeds
> Query 2> Success
>  SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as 
> VARCHAR(101))  as CA_USER_ID FROM  tablename limit 1
> The Query succeeds is we remove the distinct clause or add set 
> enable_expr_rewrites=false;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "

[Meenakshi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Meenakshi updated IMPALA-7027:
--
Description: 
If we have multiple cast of '' to varchar statements in a impala query which 
has a distinct like below, the query breaks for scenario when the cast to 
varchar limit in the SQL is lower than the previous cast.

 

Query 1> Fails with " AnalysisException: null CAUSED BY: 
IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"

SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as 
VARCHAR(100))  as CL_USER_ID FROM tablename limit 1

Where as the below query succeeds

Query 2> Success

 SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as 
VARCHAR(101))  as CL_USER_ID FROM  tablename limit 1

The Query succeeds is we remove the distinct clause or add set 
enable_expr_rewrites=false;

  was:
If we have multiple cast of '' to varchar statements in a impala query which 
has a distinct like below, the query breaks for scenario when the cast to 
varchar limit in the SQL is lower than the previous cast.

 

Query 1> Fails with " AnalysisException: null CAUSED BY: 
IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"

SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as 
VARCHAR(100))  as CA_USER_ID FROM tablename limit 1

Where as the below query succeeds

Query 2> Success

 SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as 
VARCHAR(101))  as CA_USER_ID FROM  tablename limit 1

The Query succeeds is we remove the distinct clause or add set 
enable_expr_rewrites=false;


> Multiple Cast to Varchar with different limit fails with "AnalysisException: 
> null CAUSED BY: IllegalArgumentException: "
> 
>
> Key: IMPALA-7027
> URL: https://issues.apache.org/jira/browse/IMPALA-7027
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 2.9.0
>Reporter: Meenakshi
>Priority: Critical
>
> If we have multiple cast of '' to varchar statements in a impala query which 
> has a distinct like below, the query breaks for scenario when the cast to 
> varchar limit in the SQL is lower than the previous cast.
>  
> Query 1> Fails with " AnalysisException: null CAUSED BY: 
> IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"
> SELECT DISTINCT CAST('' as VARCHAR(101)) as CL_COMMENTS,CAST('' as 
> VARCHAR(100))  as CL_USER_ID FROM tablename limit 1
> Where as the below query succeeds
> Query 2> Success
>  SELECT DISTINCT CAST('' as VARCHAR(100)) as CL_COMMENTS,CAST('' as 
> VARCHAR(101))  as CL_USER_ID FROM  tablename limit 1
> The Query succeeds is we remove the distinct clause or add set 
> enable_expr_rewrites=false;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-4268) Allow PlanRootSink to buffer more than a batch of rows

[Tim Armstrong (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-4268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474669#comment-16474669
 ] 

Tim Armstrong commented on IMPALA-4268:
---

I think it would solve that as a side-effect.

> Allow PlanRootSink to buffer more than a batch of rows
> --
>
> Key: IMPALA-4268
> URL: https://issues.apache.org/jira/browse/IMPALA-4268
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Backend
>Affects Versions: Impala 2.8.0
>Reporter: Henry Robinson
>Priority: Major
>  Labels: resource-management
>
> In IMPALA-2905, we are introducing a {{PlanRootSink}} that handles the 
> production of output rows at the root of a plan.
> The implementation in IMPALA-2905 has the plan execute in a separate thread 
> to the consumer, which calls {{GetNext()}} to retrieve the rows. However, the 
> sender thread will block until {{GetNext()}} is called, so that there are no 
> complications about memory usage and ownership due to having several batches 
> in flight at one time.
> However, this also leads to many context switches, as each {{GetNext()}} call 
> yields to the sender to produce the rows. If the sender was to fill a buffer 
> asynchronously, the consumer could pull out of that buffer without taking a 
> context switch in many cases (and the extra buffering might smooth out any 
> performance spikes due to client delays, which currently directly affect plan 
> execution).
> The tricky part is managing the mismatch between the size of the row batches 
> processed in {{Send()}} and the size of the fetch result asked for by the 
> client. The sender materializes output rows in a {{QueryResultSet}} that is 
> owned by the coordinator. That is not, currently, a splittable object - 
> instead it contains the actual RPC response struct that will hit the wire 
> when the RPC completes. As asynchronous sender cannot know the batch size, 
> which may change on every fetch call. So the {{GetNext()}} implementation 
> would need to be able to split out the {{QueryResultSet}} to match the 
> correct fetch size, and handle stitching together other {{QueryResultSets}} - 
> without doing extra copies.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation

[Tim Armstrong (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474666#comment-16474666
 ] 

Tim Armstrong commented on IMPALA-7025:
---

Is this on a 2.x or 3.x branch?

> PlannerTest.testTableSample has wrong mem-reservation
> -
>
> Key: IMPALA-7025
> URL: https://issues.apache.org/jira/browse/IMPALA-7025
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Joe McDonnell
>Assignee: Tim Armstrong
>Priority: Critical
>  Labels: broken-build, flaky
>
> Seen once on master exhaustive:
> {noformat}
> Error Message
> Section PLAN of query:
> select id from functional_parquet.alltypes tablesample system(10) 
> repeatable(1234)
> Actual does not match expected result:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB 
> thread-reservation=2
> ^^^
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=23.70KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable
> Expected:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB 
> thread-reservation=2
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=24.23KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable{noformat}
> This succeeded on the next build, so it is flaky and might not recur.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-7026) buildall.sh make shell fails due to sqlparse problem

[Fredy Wijaya (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474667#comment-16474667
 ] 

Fredy Wijaya commented on IMPALA-7026:
--

Can you rebase the CR against master?

> buildall.sh make shell fails due to sqlparse problem
> 
>
> Key: IMPALA-7026
> URL: https://issues.apache.org/jira/browse/IMPALA-7026
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.1.0
>Reporter: Michael Brown
>Assignee: Fredy Wijaya
>Priority: Blocker
>  Labels: broken-build
> Fix For: Impala 3.1.0
>
>
> https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2159/
> {noformat}
> 22:43:05 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.14
> 22:43:05 [ 47%] Building CXX object 
> be/src/udf_samples/CMakeFiles/udf-sample-test.dir/udf-sample-test.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/hdr-histogram.cc.o
> 22:43:05 [ 47%] Built target uda-sample-test
> 22:43:05 [ 47%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/impalad-metrics.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/exprs/CMakeFiles/Exprs.dir/math-functions-ir.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/testutil/CMakeFiles/TestUtil.dir/desc-tbl-builder.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/testutil/CMakeFiles/TestUtil.dir/fault-injection-util.cc.o
> 22:43:05 [ 47%] Linking CXX executable 
> ../../build/debug/udf_samples/udf-sample-test
> 22:43:05 [ 47%] Built target udf-sample-test
> 22:43:05 Scanning dependencies of target krpc
> 22:43:05 [ 47%] Building CXX object 
> be/src/exprs/CMakeFiles/Exprs.dir/null-literal.cc.o
> 22:43:05 [ 47%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/jni-util.cc.o
> 22:43:05 [ 48%] Building CXX object 
> be/src/testutil/CMakeFiles/TestUtil.dir/impalad-query-executor.cc.o
> 22:43:05 warning: no files found matching 'docs/Makefile'
> 22:43:05 warning: no files found matching 'Makefile'
> 22:43:05 [ 48%] Building CXX object 
> be/src/runtime/CMakeFiles/Runtime.dir/hbase-table.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/kudu/rpc/CMakeFiles/krpc.dir/acceptor_pool.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/testutil/CMakeFiles/TestUtil.dir/in-process-servers.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/kudu/rpc/CMakeFiles/krpc.dir/blocking_ops.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/kudu/rpc/CMakeFiles/krpc.dir/client_negotiation.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/runtime/CMakeFiles/Runtime.dir/hbase-table-factory.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/logging-support.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/mem-info.cc.o
> 22:43:05 [ 50%] Building CXX object 
> be/src/util/CMakeFiles/Util.dir/memory-metrics.cc.o
> 22:43:05 zip_safe flag not set; analyzing archive contents...
> 22:43:05 Cleaning up old build artifacts.
> 22:43:06 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.19
> 22:43:06 [ 51%] Building CXX object 
> be/src/runtime/CMakeFiles/Runtime.dir/hdfs-fs-cache.cc.o
> 22:43:06 [ 51%] Building CXX object 
> be/src/runtime/CMakeFiles/Runtime.dir/initial-reservations.cc.o
> 22:43:06 python: can't open file 'setup.py': [Errno 2] No such file or 
> directory
> 22:43:06 Error in shell/make_shell_tarball.sh at line 96: python setup.py -q 
> bdist_egg clean
> 22:43:06 CMakeFiles/shell_tarball.dir/build.make:57: recipe for target 
> 'CMakeFiles/shell_tarball' failed
> 22:43:06 make[2]: *** [CMakeFiles/shell_tarball] Error 2
> 22:43:06 CMakeFiles/Makefile2:141: recipe for target 
> 'CMakeFiles/shell_tarball.dir/all' failed
> 22:43:06 make[1]: *** [CMakeFiles/shell_tarball.dir/all] Error 2
> 22:43:06 make[1]: *** Waiting for unfinished jobs
> {noformat}
> I just reproduced this locally using {{./buildall.sh -notests 
> -start_minicluster -start_impala_cluster}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Assigned] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation

[Tim Armstrong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Armstrong reassigned IMPALA-7025:
-

Assignee: Tim Armstrong

> PlannerTest.testTableSample has wrong mem-reservation
> -
>
> Key: IMPALA-7025
> URL: https://issues.apache.org/jira/browse/IMPALA-7025
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Joe McDonnell
>Assignee: Tim Armstrong
>Priority: Critical
>  Labels: broken-build, flaky
>
> Seen once on master exhaustive:
> {noformat}
> Error Message
> Section PLAN of query:
> select id from functional_parquet.alltypes tablesample system(10) 
> repeatable(1234)
> Actual does not match expected result:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB 
> thread-reservation=2
> ^^^
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=23.70KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable
> Expected:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB 
> thread-reservation=2
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=24.23KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable{noformat}
> This succeeded on the next build, so it is flaky and might not recur.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "

[Tim Armstrong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Armstrong updated IMPALA-7027:
--
Target Version: Impala 2.13.0, Impala 3.1.0

> Multiple Cast to Varchar with different limit fails with "AnalysisException: 
> null CAUSED BY: IllegalArgumentException: "
> 
>
> Key: IMPALA-7027
> URL: https://issues.apache.org/jira/browse/IMPALA-7027
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 2.9.0
>Reporter: Meenakshi
>Priority: Critical
>
> If we have multiple cast of '' to varchar statements in a impala query which 
> has a distinct like below, the query breaks for scenario when the cast to 
> varchar limit in the SQL is lower than the previous cast.
>  
>  Query 1> Fails with " AnalysisException: null CAUSED BY: 
> IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"
> SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as 
> VARCHAR(100))  as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
> limit 1
> Where as the below query succeeds
> Query 2> Success
>  SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as 
> VARCHAR(101))  as CA_USER_ID FROM  pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
> limit 1
> The Query succeeds is we remove the distinct clause or add set 
> enable_expr_rewrites=false;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "

[Tim Armstrong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Armstrong updated IMPALA-7027:
--
Priority: Critical  (was: Major)

> Multiple Cast to Varchar with different limit fails with "AnalysisException: 
> null CAUSED BY: IllegalArgumentException: "
> 
>
> Key: IMPALA-7027
> URL: https://issues.apache.org/jira/browse/IMPALA-7027
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 2.9.0
>Reporter: Meenakshi
>Priority: Critical
>
> If we have multiple cast of '' to varchar statements in a impala query which 
> has a distinct like below, the query breaks for scenario when the cast to 
> varchar limit in the SQL is lower than the previous cast.
>  
>  Query 1> Fails with " AnalysisException: null CAUSED BY: 
> IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"
> SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as 
> VARCHAR(100))  as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
> limit 1
> Where as the below query succeeds
> Query 2> Success
>  SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as 
> VARCHAR(101))  as CA_USER_ID FROM  pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
> limit 1
> The Query succeeds is we remove the distinct clause or add set 
> enable_expr_rewrites=false;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Updated] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "

[Tim Armstrong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Armstrong updated IMPALA-7027:
--
Component/s: (was: Clients)
 Frontend

> Multiple Cast to Varchar with different limit fails with "AnalysisException: 
> null CAUSED BY: IllegalArgumentException: "
> 
>
> Key: IMPALA-7027
> URL: https://issues.apache.org/jira/browse/IMPALA-7027
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 2.9.0
>Reporter: Meenakshi
>Priority: Critical
>
> If we have multiple cast of '' to varchar statements in a impala query which 
> has a distinct like below, the query breaks for scenario when the cast to 
> varchar limit in the SQL is lower than the previous cast.
>  
>  Query 1> Fails with " AnalysisException: null CAUSED BY: 
> IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"
> SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as 
> VARCHAR(100))  as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
> limit 1
> Where as the below query succeeds
> Query 2> Success
>  SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as 
> VARCHAR(101))  as CA_USER_ID FROM  pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
> limit 1
> The Query succeeds is we remove the distinct clause or add set 
> enable_expr_rewrites=false;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Updated] (IMPALA-7025) PlannerTest.testTableSample has wrong mem-reservation

[Joe McDonnell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe McDonnell updated IMPALA-7025:
--
Summary: PlannerTest.testTableSample has wrong mem-reservation  (was: 
PlannerTest.testTableSample)

> PlannerTest.testTableSample has wrong mem-reservation
> -
>
> Key: IMPALA-7025
> URL: https://issues.apache.org/jira/browse/IMPALA-7025
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 3.1.0
>Reporter: Joe McDonnell
>Priority: Critical
>  Labels: broken-build, flaky
>
> Seen once on master exhaustive:
> {noformat}
> Error Message
> Section PLAN of query:
> select id from functional_parquet.alltypes tablesample system(10) 
> repeatable(1234)
> Actual does not match expected result:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB 
> thread-reservation=2
> ^^^
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=23.70KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable
> Expected:
> F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
> |  Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB 
> thread-reservation=2
> PLAN-ROOT SINK
> |  mem-estimate=0B mem-reservation=0B thread-reservation=0
> |
> 00:SCAN HDFS [functional_parquet.alltypes]
>partitions=3/24 files=3 size=24.23KB
>stored statistics:
>  table: rows=unavailable size=unavailable
>  partitions: 0/24 rows=unavailable
>  columns: unavailable
>extrapolated-rows=disabled max-scan-range-rows=unavailable
>mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
>tuple-ids=0 row-size=4B cardinality=unavailable{noformat}
> This succeeded on the next build, so it is flaky and might not recur.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Created] (IMPALA-7027) Multiple Cast to Varchar with different limit fails with "AnalysisException: null CAUSED BY: IllegalArgumentException: "

[Meenakshi (JIRA)]

Meenakshi created IMPALA-7027:
-

 Summary: Multiple Cast to Varchar with different limit fails with 
"AnalysisException: null CAUSED BY: IllegalArgumentException: "
 Key: IMPALA-7027
 URL: https://issues.apache.org/jira/browse/IMPALA-7027
 Project: IMPALA
  Issue Type: Bug
  Components: Clients
Affects Versions: Impala 2.9.0
Reporter: Meenakshi


If we have multiple cast of '' to varchar statements in a impala query which 
has a distinct like below, the query breaks for scenario when the cast to 
varchar limit in the SQL is lower than the previous cast.

 

 Query 1> Fails with " AnalysisException: null CAUSED BY: 
IllegalArgumentException: targetType=VARCHAR(100) type=VARCHAR(101)"

SELECT DISTINCT CAST('' as VARCHAR(101)) as CLAIM_COMMENTS,CAST('' as 
VARCHAR(100))  as CA_USER_ID FROM pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
limit 1

Where as the below query succeeds

Query 2> Success

 SELECT DISTINCT CAST('' as VARCHAR(100)) as CLAIM_COMMENTS,CAST('' as 
VARCHAR(101))  as CA_USER_ID FROM  pinnga1ph_ds_prd_wh_gbd.concept_flat_dtl 
limit 1

The Query succeeds is we remove the distinct clause or add set 
enable_expr_rewrites=false;



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Created] (IMPALA-7026) buildall.sh make shell fails due to sqlparse problem

[Michael Brown (JIRA)]

Michael Brown created IMPALA-7026:
-

 Summary: buildall.sh make shell fails due to sqlparse problem
 Key: IMPALA-7026
 URL: https://issues.apache.org/jira/browse/IMPALA-7026
 Project: IMPALA
  Issue Type: Bug
  Components: Infrastructure
Affects Versions: Impala 3.1.0
Reporter: Michael Brown
Assignee: Fredy Wijaya
 Fix For: Impala 3.1.0


https://jenkins.impala.io/job/ubuntu-16.04-from-scratch/2159/

{noformat}
22:43:05 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.14
22:43:05 [ 47%] Building CXX object 
be/src/udf_samples/CMakeFiles/udf-sample-test.dir/udf-sample-test.cc.o
22:43:05 [ 47%] Building CXX object 
be/src/util/CMakeFiles/Util.dir/hdr-histogram.cc.o
22:43:05 [ 47%] Built target uda-sample-test
22:43:05 [ 47%] Building CXX object 
be/src/util/CMakeFiles/Util.dir/impalad-metrics.cc.o
22:43:05 [ 47%] Building CXX object 
be/src/exprs/CMakeFiles/Exprs.dir/math-functions-ir.cc.o
22:43:05 [ 47%] Building CXX object 
be/src/testutil/CMakeFiles/TestUtil.dir/desc-tbl-builder.cc.o
22:43:05 [ 47%] Building CXX object 
be/src/testutil/CMakeFiles/TestUtil.dir/fault-injection-util.cc.o
22:43:05 [ 47%] Linking CXX executable 
../../build/debug/udf_samples/udf-sample-test
22:43:05 [ 47%] Built target udf-sample-test
22:43:05 Scanning dependencies of target krpc
22:43:05 [ 47%] Building CXX object 
be/src/exprs/CMakeFiles/Exprs.dir/null-literal.cc.o
22:43:05 [ 47%] Building CXX object 
be/src/util/CMakeFiles/Util.dir/jni-util.cc.o
22:43:05 [ 48%] Building CXX object 
be/src/testutil/CMakeFiles/TestUtil.dir/impalad-query-executor.cc.o
22:43:05 warning: no files found matching 'docs/Makefile'
22:43:05 warning: no files found matching 'Makefile'
22:43:05 [ 48%] Building CXX object 
be/src/runtime/CMakeFiles/Runtime.dir/hbase-table.cc.o
22:43:05 [ 50%] Building CXX object 
be/src/kudu/rpc/CMakeFiles/krpc.dir/acceptor_pool.cc.o
22:43:05 [ 50%] Building CXX object 
be/src/testutil/CMakeFiles/TestUtil.dir/in-process-servers.cc.o
22:43:05 [ 50%] Building CXX object 
be/src/kudu/rpc/CMakeFiles/krpc.dir/blocking_ops.cc.o
22:43:05 [ 50%] Building CXX object 
be/src/kudu/rpc/CMakeFiles/krpc.dir/client_negotiation.cc.o
22:43:05 [ 50%] Building CXX object 
be/src/runtime/CMakeFiles/Runtime.dir/hbase-table-factory.cc.o
22:43:05 [ 50%] Building CXX object 
be/src/util/CMakeFiles/Util.dir/logging-support.cc.o
22:43:05 [ 50%] Building CXX object 
be/src/util/CMakeFiles/Util.dir/mem-info.cc.o
22:43:05 [ 50%] Building CXX object 
be/src/util/CMakeFiles/Util.dir/memory-metrics.cc.o
22:43:05 zip_safe flag not set; analyzing archive contents...
22:43:05 Cleaning up old build artifacts.
22:43:06 Creating an egg for /home/ubuntu/Impala/shell/ext-py/sqlparse-0.1.19
22:43:06 [ 51%] Building CXX object 
be/src/runtime/CMakeFiles/Runtime.dir/hdfs-fs-cache.cc.o
22:43:06 [ 51%] Building CXX object 
be/src/runtime/CMakeFiles/Runtime.dir/initial-reservations.cc.o
22:43:06 python: can't open file 'setup.py': [Errno 2] No such file or directory
22:43:06 Error in shell/make_shell_tarball.sh at line 96: python setup.py -q 
bdist_egg clean
22:43:06 CMakeFiles/shell_tarball.dir/build.make:57: recipe for target 
'CMakeFiles/shell_tarball' failed
22:43:06 make[2]: *** [CMakeFiles/shell_tarball] Error 2
22:43:06 CMakeFiles/Makefile2:141: recipe for target 
'CMakeFiles/shell_tarball.dir/all' failed
22:43:06 make[1]: *** [CMakeFiles/shell_tarball.dir/all] Error 2
22:43:06 make[1]: *** Waiting for unfinished jobs
{noformat}

I just reproduced this locally using {{./buildall.sh -notests 
-start_minicluster -start_impala_cluster}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Created] (IMPALA-7025) PlannerTest.testTableSample

[Joe McDonnell (JIRA)]

Joe McDonnell created IMPALA-7025:
-

 Summary: PlannerTest.testTableSample
 Key: IMPALA-7025
 URL: https://issues.apache.org/jira/browse/IMPALA-7025
 Project: IMPALA
  Issue Type: Bug
  Components: Frontend
Affects Versions: Impala 3.1.0
Reporter: Joe McDonnell


Seen once on master exhaustive:
{noformat}
Error Message
Section PLAN of query:
select id from functional_parquet.alltypes tablesample system(10) 
repeatable(1234)

Actual does not match expected result:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
|  Per-Host Resources: mem-estimate=16.00MB mem-reservation=8.00KB 
thread-reservation=2
^^^
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B thread-reservation=0
|
00:SCAN HDFS [functional_parquet.alltypes]
   partitions=3/24 files=3 size=23.70KB
   stored statistics:
 table: rows=unavailable size=unavailable
 partitions: 0/24 rows=unavailable
 columns: unavailable
   extrapolated-rows=disabled max-scan-range-rows=unavailable
   mem-estimate=16.00MB mem-reservation=8.00KB thread-reservation=1
   tuple-ids=0 row-size=4B cardinality=unavailable

Expected:
F00:PLAN FRAGMENT [UNPARTITIONED] hosts=1 instances=1
|  Per-Host Resources: mem-estimate=16.00MB mem-reservation=16.00KB 
thread-reservation=2
PLAN-ROOT SINK
|  mem-estimate=0B mem-reservation=0B thread-reservation=0
|
00:SCAN HDFS [functional_parquet.alltypes]
   partitions=3/24 files=3 size=24.23KB
   stored statistics:
 table: rows=unavailable size=unavailable
 partitions: 0/24 rows=unavailable
 columns: unavailable
   extrapolated-rows=disabled max-scan-range-rows=unavailable
   mem-estimate=16.00MB mem-reservation=16.00KB thread-reservation=1
   tuple-ids=0 row-size=4B cardinality=unavailable{noformat}
This succeeded on the next build, so it is flaky and might not recur.

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-6987) Impala Doc: Clear up Impala' invalidate metadata page

[Alex Rodoni (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-6987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474638#comment-16474638
 ] 

Alex Rodoni commented on IMPALA-6987:
-

[~balazsj_impala_220b]

> Impala Doc: Clear up Impala' invalidate metadata page
> -
>
> Key: IMPALA-6987
> URL: https://issues.apache.org/jira/browse/IMPALA-6987
> Project: IMPALA
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: Impala 2.12.0
>Reporter: Alex Rodoni
>Assignee: Alex Rodoni
>Priority: Major
>
> The usage notes section on this page has information that is from pre-catalog 
> service days:
> {code:java}
> A metadata update for an impalad instance is required if:
> A metadata change occurs
> and the change is made from another impalad instance in your cluster, or 
> through Hive.
> and the change is made to a metastore database to which clients such as the 
> Impala shell or ODBC directly connect.
> A metadata update for an Impala node is not required when you issue queries 
> from the same Impala node where you ran ALTER TABLE, INSERT, or other 
> table-modifying statement.
> {code}
> This should be cleared up.
> The first point is incorrect, catalogd distributes metadata from other 
> impalads.
> The last point is unnecessary and should be removed.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-4268) Allow PlanRootSink to buffer more than a batch of rows

[Dan Hecht (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-4268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474634#comment-16474634
 ] 

Dan Hecht commented on IMPALA-4268:
---

[~tarmstrong], do you also expect to solve IMPALA-1618 with that?

> Allow PlanRootSink to buffer more than a batch of rows
> --
>
> Key: IMPALA-4268
> URL: https://issues.apache.org/jira/browse/IMPALA-4268
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Backend
>Affects Versions: Impala 2.8.0
>Reporter: Henry Robinson
>Priority: Major
>  Labels: resource-management
>
> In IMPALA-2905, we are introducing a {{PlanRootSink}} that handles the 
> production of output rows at the root of a plan.
> The implementation in IMPALA-2905 has the plan execute in a separate thread 
> to the consumer, which calls {{GetNext()}} to retrieve the rows. However, the 
> sender thread will block until {{GetNext()}} is called, so that there are no 
> complications about memory usage and ownership due to having several batches 
> in flight at one time.
> However, this also leads to many context switches, as each {{GetNext()}} call 
> yields to the sender to produce the rows. If the sender was to fill a buffer 
> asynchronously, the consumer could pull out of that buffer without taking a 
> context switch in many cases (and the extra buffering might smooth out any 
> performance spikes due to client delays, which currently directly affect plan 
> execution).
> The tricky part is managing the mismatch between the size of the row batches 
> processed in {{Send()}} and the size of the fetch result asked for by the 
> client. The sender materializes output rows in a {{QueryResultSet}} that is 
> owned by the coordinator. That is not, currently, a splittable object - 
> instead it contains the actual RPC response struct that will hit the wire 
> when the RPC completes. As asynchronous sender cannot know the batch size, 
> which may change on every fetch call. So the {{GetNext()}} implementation 
> would need to be able to split out the {{QueryResultSet}} to match the 
> correct fetch size, and handle stitching together other {{QueryResultSets}} - 
> without doing extra copies.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-7000) Wrong info about Impala dedicated executore

[Alex Rodoni (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474632#comment-16474632
 ] 

Alex Rodoni commented on IMPALA-7000:
-

https://gerrit.cloudera.org/#/c/10357/

> Wrong info about Impala dedicated executore
> ---
>
> Key: IMPALA-7000
> URL: https://issues.apache.org/jira/browse/IMPALA-7000
> Project: IMPALA
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: Impala 2.12.0
>Reporter: Alex Rodoni
>Assignee: Alex Rodoni
>Priority: Major
>
> The following is not correct.
> "Then, you specify that the other hosts act as executors but not 
> coordinators. These hosts do not communicate with the statestored daemon or 
> process the final result sets from queries. You cannot connect to these hosts 
> through clients such as impala-shell or business intelligence tools."
> executor still communicates with statestore for other topics (membership, 
> admission control, etc.) The only part it doesn't get from statestore is the 
> metadata topic.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-7005) The 3.0 changelog page should mention the previous version

[Alex Rodoni (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474630#comment-16474630
 ] 

Alex Rodoni commented on IMPALA-7005:
-

[~lv] [~sailesh] Could you tell me where the script to generate change logs? I 
would like to take a look to see if this is something I can take care of. 
Thanks!

> The 3.0 changelog page should mention the previous version
> --
>
> Key: IMPALA-7005
> URL: https://issues.apache.org/jira/browse/IMPALA-7005
> Project: IMPALA
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: Impala 3.0
>Reporter: Lars Volker
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: website
>
> It's not clear to me from looking at the [3.0 
> changlog|http://impala.apache.org/docs/changelog-3.0.html] what base version 
> it compares to. Can we add a sentence to point out that these changes are in 
> comparison to 2.11 (are they?)?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Resolved] (IMPALA-1790) FetchResults() sometimes returns very few resuts

[Dan Hecht (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-1790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dan Hecht resolved IMPALA-1790.
---
Resolution: Duplicate

> FetchResults() sometimes returns very few resuts
> 
>
> Key: IMPALA-1790
> URL: https://issues.apache.org/jira/browse/IMPALA-1790
> Project: IMPALA
>  Issue Type: Bug
>  Components: Backend
>Affects Versions: Impala 2.1.1
>Reporter: Henry Robinson
>Assignee: Dan Hecht
>Priority: Major
>
> Hue has found that the {{FetchResults()}} RPC sometimes returns as few as 2 
> results, when it really should be waiting for a full batch if possible.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Created] (IMPALA-7024) Convert Coordinator::wait_lock_ from boost::mutex to SpinLock

[Dan Hecht (JIRA)]

Dan Hecht created IMPALA-7024:
-

 Summary: Convert Coordinator::wait_lock_ from boost::mutex to 
SpinLock
 Key: IMPALA-7024
 URL: https://issues.apache.org/jira/browse/IMPALA-7024
 Project: IMPALA
  Issue Type: Improvement
Affects Versions: Impala 3.1.0
Reporter: Dan Hecht
Assignee: Dan Hecht


For consistency with other locks in this class, convert 
Coordinator::wait_lock_. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-7015) Insert into Kudu table returns with Status OK even if there are Kudu errors

[Lars Volker (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474595#comment-16474595
 ] 

Lars Volker commented on IMPALA-7015:
-

[~mmokhtar] - I suspect you meant [~twmarshall] ?

> Insert into Kudu table returns with Status OK even if there are Kudu errors
> ---
>
> Key: IMPALA-7015
> URL: https://issues.apache.org/jira/browse/IMPALA-7015
> Project: IMPALA
>  Issue Type: Bug
>  Components: Backend
>Affects Versions: Impala 2.12.0
>Reporter: Mostafa Mokhtar
>Priority: Major
> Attachments: Insert into kudu profile with errors.txt
>
>
> DML statements against Kudu tables return status OK even if there are Kudu 
> errors.
> This behavior is misleading. 
> {code}
>   Summary:
> Session ID: 18430b000e5dd8dc:e3e5dadb4a15d4b4
> Session Type: BEESWAX
> Start Time: 2018-05-11 10:10:07.314218000
> End Time: 2018-05-11 10:10:07.434017000
> Query Type: DML
> Query State: FINISHED
> Query Status: OK
> Impala Version: impalad version 2.12.0-cdh5.15.0 RELEASE (build 
> 2f9498d5c2f980aa7ff9505c56654c8e59e026ca)
> User: mmokhtar
> Connected User: mmokhtar
> Delegated User: 
> Network Address: :::10.17.234.27:60760
> Default Db: tpcds_1000_kudu
> Sql Statement: insert into store_2 select * from store
> Coordinator: vd1317.foo:22000
> Query Options (set by configuration): 
> Query Options (set by configuration and planner): MT_DOP=0
> Plan: 
> {code}
> {code}
> Operator  #Hosts   Avg Time  Max Time  #Rows  Est. #Rows  Peak Mem  
> Est. Peak Mem  Detail
> -
> 02:PARTIAL SORT5  909.030us   1.025ms  1.00K   1.00K   6.14 MB
> 4.00 MB
> 01:EXCHANGE56.262ms   7.232ms  1.00K   1.00K  75.50 KB
>   0  KUDU(KuduPartition(tpcds_1000_kudu.store.s_store_sk)) 
> 00:SCAN KUDU   53.694ms   4.137ms  1.00K   1.00K   4.34 MB
>   0  tpcds_1000_kudu.store 
> Errors: Key already present in Kudu table 
> 'impala::tpcds_1000_kudu.store_2'. (1 of 1002 similar)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Assigned] (IMPALA-7022) TestQueries.test_subquery: Subquery must not return more than one row

[Tim Armstrong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tim Armstrong reassigned IMPALA-7022:
-

Assignee: Zoltán Borók-Nagy

> TestQueries.test_subquery: Subquery must not return more than one row
> -
>
> Key: IMPALA-7022
> URL: https://issues.apache.org/jira/browse/IMPALA-7022
> Project: IMPALA
>  Issue Type: Bug
>  Components: Frontend
>Affects Versions: Impala 2.13.0, Impala 3.1.0
>Reporter: Joe McDonnell
>Assignee: Zoltán Borók-Nagy
>Priority: Blocker
>  Labels: broken-build
>
> Seen on master and on 2.x based branches on exhaustive tests:
> {noformat}
> Error Message
> query_test/test_queries.py:102: in test_subquery 
> self.run_test_case('QueryTest/subquery', vector) 
> common/impala_test_suite.py:408: in run_test_case 
> self.__verify_exceptions(test_section['CATCH'], str(e), use_db) 
> common/impala_test_suite.py:286: in __verify_exceptions (expected_str, 
> actual_str) E   AssertionError: Unexpected exception string. Expected: Query 
> aborted:Subquery must not return more than one row: SELECT bigint_col FROM 
> functional.alltypes_view E   Not found in actual: ImpalaBeeswaxException: 
> INNER EXCEPTION:  MESSAGE: Subquery 
> must not return more than one row: SELECT bigint_col FROM 
> functional.alltypes_view
> Stacktrace
> query_test/test_queries.py:102: in test_subquery
> self.run_test_case('QueryTest/subquery', vector)
> common/impala_test_suite.py:408: in run_test_case
> self.__verify_exceptions(test_section['CATCH'], str(e), use_db)
> common/impala_test_suite.py:286: in __verify_exceptions
> (expected_str, actual_str)
> E   AssertionError: Unexpected exception string. Expected: Query 
> aborted:Subquery must not return more than one row: SELECT bigint_col FROM 
> functional.alltypes_view
> E   Not found in actual: ImpalaBeeswaxException: INNER EXCEPTION:  'beeswaxd.ttypes.BeeswaxException'> MESSAGE: Subquery must not return more 
> than one row: SELECT bigint_col FROM functional.alltypes_view{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Created] (IMPALA-7023) TestInsertQueries.test_insert_overwrite fails by hitting memory limit

[Joe McDonnell (JIRA)]

Joe McDonnell created IMPALA-7023:
-

 Summary: TestInsertQueries.test_insert_overwrite fails by hitting 
memory limit
 Key: IMPALA-7023
 URL: https://issues.apache.org/jira/browse/IMPALA-7023
 Project: IMPALA
  Issue Type: Bug
  Components: Backend
Affects Versions: Impala 2.13.0, Impala 3.1.0
Reporter: Joe McDonnell


This failure is seen on exhaustive builds on both master and 2.x:
{noformat}
Error Message
ImpalaBeeswaxException: ImpalaBeeswaxException:  INNER EXCEPTION:   MESSAGE: AnalysisException: Failed to 
evaluate expr: 20 CAUSED BY: InternalException: Memory limit exceeded: Error 
occurred on backend 
impala-boost-static-burst-slave-el7-03d4.vpc.cloudera.com:22000 by fragment 0:0 
Memory left in process limit: -4.29 GB 
Query(d24ea53242b4cedc:fc8e0885): Total=0 Peak=0   : Total=0 
Peak=0Process: memory limit exceeded. Limit=12.00 GB Total=16.29 GB Peak=16.29 
GB   Buffer Pool: Free Buffers: Total=160.00 KB   Buffer Pool: Clean Pages: 
Total=0   Buffer Pool: Unused Reservation: Total=-328.00 KB   Data Stream 
Service Queue: Limit=614.40 MB Total=0 Peak=116.12 KB   Data Stream Manager 
Early RPCs: Total=0 Peak=6.76 KB   TCMalloc Overhead: Total=103.56 MB   
RequestPool=fe-eval-exprs: Total=0 Peak=52.83 KB 
Query(d24ea53242b4cedc:fc8e0885): Total=0 Peak=0   
RequestPool=default-pool: Total=5.20 GB Peak=5.20 GB 
Query(f4014f7bb49ea78:6b926b19): memory limit exceeded. Limit=64.00 MB 
Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.76 GB Total=1.76 GB 
Peak=1.96 GB Query(2c44b65fbcb4e1ce:3d73badf): memory limit 
exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB 
OtherMemory=1.04 GB Total=1.04 GB Peak=1.61 GB 
Query(214cc23c1376176f:7844977b): memory limit exceeded. Limit=64.00 MB 
Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.23 GB Total=1.23 GB 
Peak=1.23 GB Query(8949bdf792a32ad2:33a36c03): memory limit 
exceeded. Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB 
OtherMemory=642.20 MB Total=642.20 MB Peak=642.20 MB 
Query(5412ff4e6065721:519d3e61): memory limit exceeded. Limit=64.00 MB 
Reservation=0 ReservationLimit=32.00 MB OtherMemory=556.49 MB Total=556.49 MB 
Peak=556.49 MB   Untracked Memory: Total=10.98 GB
Stacktrace
query_test/test_insert.py:132: in test_insert_overwrite
multiple_impalad=vector.get_value('exec_option')['sync_ddl'] == 1)
common/impala_test_suite.py:405: in run_test_case
result = self.__execute_query(target_impalad_client, query, user=user)
common/impala_test_suite.py:620: in __execute_query
return impalad_client.execute(query, user=user)
common/impala_connection.py:160: in execute
return self.__beeswax_client.execute(sql_stmt, user=user)
beeswax/impala_beeswax.py:173: in execute
handle = self.__execute_query(query_string.strip(), user=user)
beeswax/impala_beeswax.py:339: in __execute_query
handle = self.execute_query_async(query_string, user=user)
beeswax/impala_beeswax.py:335: in execute_query_async
return self.__do_rpc(lambda: self.imp_service.query(query,))
beeswax/impala_beeswax.py:460: in __do_rpc
raise ImpalaBeeswaxException(self.__build_error_message(b), b)
E   ImpalaBeeswaxException: ImpalaBeeswaxException:
EINNER EXCEPTION: 
EMESSAGE: AnalysisException: Failed to evaluate expr: 20
E   CAUSED BY: InternalException: Memory limit exceeded: Error occurred on 
backend impala-boost-static-burst-slave-el7-03d4.vpc.cloudera.com:22000 by 
fragment 0:0
E   Memory left in process limit: -4.29 GB
E   Query(d24ea53242b4cedc:fc8e0885): Total=0 Peak=0
E : Total=0 Peak=0Process: memory limit exceeded. Limit=12.00 GB 
Total=16.29 GB Peak=16.29 GB
E Buffer Pool: Free Buffers: Total=160.00 KB
E Buffer Pool: Clean Pages: Total=0
E Buffer Pool: Unused Reservation: Total=-328.00 KB
E Data Stream Service Queue: Limit=614.40 MB Total=0 Peak=116.12 KB
E Data Stream Manager Early RPCs: Total=0 Peak=6.76 KB
E TCMalloc Overhead: Total=103.56 MB
E RequestPool=fe-eval-exprs: Total=0 Peak=52.83 KB
E   Query(d24ea53242b4cedc:fc8e0885): Total=0 Peak=0
E RequestPool=default-pool: Total=5.20 GB Peak=5.20 GB
E   Query(f4014f7bb49ea78:6b926b19): memory limit exceeded. 
Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.76 GB 
Total=1.76 GB Peak=1.96 GB
E   Query(2c44b65fbcb4e1ce:3d73badf): memory limit exceeded. 
Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.04 GB 
Total=1.04 GB Peak=1.61 GB
E   Query(214cc23c1376176f:7844977b): memory limit exceeded. 
Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=1.23 GB 
Total=1.23 GB Peak=1.23 GB
E   Query(8949bdf792a32ad2:33a36c03): memory limit exceeded. 
Limit=64.00 MB Reservation=0 ReservationLimit=32.00 MB OtherMemory=642.20 

[jira] [Commented] (IMPALA-6819) Add new performance test workloads

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-6819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474582#comment-16474582
 ] 

ASF subversion and git services commented on IMPALA-6819:
-

Commit bb9237d942abc69efcf1b26bfd10f149dd781948 in impala's branch 
refs/heads/master from [~janarthanan]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=bb9237d ]

IMPALA-6819: Add new performance test workload - tpcds-unmodified used by 
Impala Performance Tests

Description:
Impala versions prior to 2.5 didn't have Runtimefilters, which made TPC-DS 
queries run very slow,
so queries under tpcds have explicit partition filters to workaround the 
limitation.
Post Impala 2.5 adding tpcds-unmodified which has the unmodified version of the
workload to provide more coverage

Testing:
Ran the performance tests using the new workload
and all tests passed

Change-Id: I3957621d88b80fffc8fc89fd8104a58137b86e92
Reviewed-on: http://gerrit.cloudera.org:8080/9973
Reviewed-by: David Knupp 
Tested-by: Impala Public Jenkins 


> Add new performance test workloads 
> ---
>
> Key: IMPALA-6819
> URL: https://issues.apache.org/jira/browse/IMPALA-6819
> Project: IMPALA
>  Issue Type: Task
>  Components: Infrastructure
>Reporter: nithya
>Assignee: nithya
>Priority: Major
>
> Add additional workloads to impala-asf rep
> Workloads that will be added
> {code:java}
> [targeted-perf]
> [tpcds-unmodified]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-6221) Kudu partition clause doc is wrong

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474583#comment-16474583
 ] 

ASF subversion and git services commented on IMPALA-6221:
-

Commit 19bcc3099ef5e244fe74a0466b0b0eeb673acc8e in impala's branch 
refs/heads/master from [~arodoni_cloudera]
[ https://git-wip-us.apache.org/repos/asf?p=impala.git;h=19bcc30 ]

IMPALA-6221: [DOCS] Correct syntax for Kudu Partition clause

Change-Id: I8d8ccdcc8b28fcd44511040f1e05969af83b3d1e
Reviewed-on: http://gerrit.cloudera.org:8080/10353
Reviewed-by: Thomas Tauber-Marshall 
Tested-by: Impala Public Jenkins 


> Kudu partition clause doc is wrong
> --
>
> Key: IMPALA-6221
> URL: https://issues.apache.org/jira/browse/IMPALA-6221
> Project: IMPALA
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: Impala 2.11.0
>Reporter: Thomas Tauber-Marshall
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: newbie
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> The following line in the docs:
> {noformat}
> kudu_partition_clause ::= [hash_clause] [, range_clause [ , range_clause ] ]
> {noformat}
> which can be found here: 
> https://impala.apache.org/docs/build/html/topics/impala_create_table.html
> is incorrect. We support an unlimited number of hash clauses followed by 
> either 0 or 1 range clauses, whereas the doc shows the opposite.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Closed] (IMPALA-6221) Kudu partition clause doc is wrong

[Alex Rodoni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Rodoni closed IMPALA-6221.
---
   Resolution: Fixed
Fix Version/s: Impala 3.1.0
   Impala 2.13.0

> Kudu partition clause doc is wrong
> --
>
> Key: IMPALA-6221
> URL: https://issues.apache.org/jira/browse/IMPALA-6221
> Project: IMPALA
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: Impala 2.11.0
>Reporter: Thomas Tauber-Marshall
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: newbie
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> The following line in the docs:
> {noformat}
> kudu_partition_clause ::= [hash_clause] [, range_clause [ , range_clause ] ]
> {noformat}
> which can be found here: 
> https://impala.apache.org/docs/build/html/topics/impala_create_table.html
> is incorrect. We support an unlimited number of hash clauses followed by 
> either 0 or 1 range clauses, whereas the doc shows the opposite.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Closed] (IMPALA-6221) Kudu partition clause doc is wrong

[Alex Rodoni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-6221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Rodoni closed IMPALA-6221.
---
   Resolution: Fixed
Fix Version/s: Impala 3.1.0
   Impala 2.13.0

> Kudu partition clause doc is wrong
> --
>
> Key: IMPALA-6221
> URL: https://issues.apache.org/jira/browse/IMPALA-6221
> Project: IMPALA
>  Issue Type: Bug
>  Components: Docs
>Affects Versions: Impala 2.11.0
>Reporter: Thomas Tauber-Marshall
>Assignee: Alex Rodoni
>Priority: Major
>  Labels: newbie
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> The following line in the docs:
> {noformat}
> kudu_partition_clause ::= [hash_clause] [, range_clause [ , range_clause ] ]
> {noformat}
> which can be found here: 
> https://impala.apache.org/docs/build/html/topics/impala_create_table.html
> is incorrect. We support an unlimited number of hash clauses followed by 
> either 0 or 1 range clauses, whereas the doc shows the opposite.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Created] (IMPALA-7022) TestQueries.test_subquery: Subquery must not return more than one row

[Joe McDonnell (JIRA)]

Joe McDonnell created IMPALA-7022:
-

 Summary: TestQueries.test_subquery: Subquery must not return more 
than one row
 Key: IMPALA-7022
 URL: https://issues.apache.org/jira/browse/IMPALA-7022
 Project: IMPALA
  Issue Type: Bug
  Components: Frontend
Affects Versions: Impala 2.13.0, Impala 3.1.0
Reporter: Joe McDonnell


Seen on master and on 2.x based branches on exhaustive tests:
{noformat}
Error Message
query_test/test_queries.py:102: in test_subquery 
self.run_test_case('QueryTest/subquery', vector) 
common/impala_test_suite.py:408: in run_test_case 
self.__verify_exceptions(test_section['CATCH'], str(e), use_db) 
common/impala_test_suite.py:286: in __verify_exceptions (expected_str, 
actual_str) E   AssertionError: Unexpected exception string. Expected: Query 
aborted:Subquery must not return more than one row: SELECT bigint_col FROM 
functional.alltypes_view E   Not found in actual: ImpalaBeeswaxException: INNER 
EXCEPTION:  MESSAGE: Subquery must 
not return more than one row: SELECT bigint_col FROM functional.alltypes_view
Stacktrace
query_test/test_queries.py:102: in test_subquery
self.run_test_case('QueryTest/subquery', vector)
common/impala_test_suite.py:408: in run_test_case
self.__verify_exceptions(test_section['CATCH'], str(e), use_db)
common/impala_test_suite.py:286: in __verify_exceptions
(expected_str, actual_str)
E   AssertionError: Unexpected exception string. Expected: Query 
aborted:Subquery must not return more than one row: SELECT bigint_col FROM 
functional.alltypes_view
E   Not found in actual: ImpalaBeeswaxException: INNER EXCEPTION:  MESSAGE: Subquery must not return more than 
one row: SELECT bigint_col FROM functional.alltypes_view{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Resolved] (IMPALA-6907) ImpalaServer::MembershipCallback() may not remove all stale connections to disconnected Impalad nodes

[Michael Ho (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-6907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Ho resolved IMPALA-6907.

   Resolution: Fixed
Fix Version/s: Impala 3.1.0
   Impala 2.13.0

> ImpalaServer::MembershipCallback() may not remove all stale connections to 
> disconnected Impalad nodes
> -
>
> Key: IMPALA-6907
> URL: https://issues.apache.org/jira/browse/IMPALA-6907
> Project: IMPALA
>  Issue Type: Bug
>  Components: Distributed Exec
>Affects Versions: Impala 2.9.0, Impala 2.10.0, Impala 2.11.0, Impala 3.0, 
> Impala 2.12.0
>Reporter: Michael Ho
>Assignee: Michael Ho
>Priority: Major
> Fix For: Impala 2.13.0, Impala 3.1.0
>
>
> Currently, {{ImpalaServer::MembershipCallback()}} will remove stale 
> connections to hosts which were removed from the cluster membership.
> {noformat}
>   while (loc_entry != query_locations_.end()) {
> if (current_membership.find(loc_entry->first) == 
> current_membership.end()) {
>   unordered_set::const_iterator query_id = 
> loc_entry->second.begin();
>   // Add failed backend locations to all queries that ran on that 
> backend.
>   for(; query_id != loc_entry->second.end(); ++query_id) {
> vector& failed_hosts = 
> queries_to_cancel[*query_id];
> failed_hosts.push_back(loc_entry->first);
>   }
>   
> exec_env_->impalad_client_cache()->CloseConnections(loc_entry->first); 
> <<<-
> {noformat}
> However, it's relies on checking against {{query_locations_}} which is 
> populated only when the Impalad node acts as a coordinator and currently 
> running queries using the disconnected backend. So 
> {{ImpalaServer::MembershipCallback()}} will not reliably remove stale 
> connections to hosts removed from cluster. This may cause stale connections 
> to stay in connection cache for extended period of time, leading to query 
> failure after the removed hosts rejoined the cluster as the stale connections 
> are used.
> Instead, we should remove stale connections regardless of whether this node 
> happens to be currently coordinating a query using that backend.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (IMPALA-7008) TestSpillingDebugActionDimensions.test_spilling test setup fails

[Joe McDonnell (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474539#comment-16474539
 ] 

Joe McDonnell commented on IMPALA-7008:
---

OOM killer killed python:

Out of memory: Kill process 14714 (python) score 780 or sacrifice child
Killed process 14714, UID 1106, (python) total-vm:54718348kB, 
anon-rss:54115332kB, file-rss:276kB

> TestSpillingDebugActionDimensions.test_spilling test setup fails
> 
>
> Key: IMPALA-7008
> URL: https://issues.apache.org/jira/browse/IMPALA-7008
> Project: IMPALA
>  Issue Type: Bug
>  Components: Infrastructure
>Affects Versions: Impala 3.0, Impala 2.13.0
>Reporter: Sailesh Mukil
>Priority: Blocker
>  Labels: broken-build, flaky
>
> We've seen multiple instances of this test failing with the following error:
> {code:java}
> Error Message
> test setup failure
> Stacktrace
> Slave 'gw0' crashed while running 
> "query_test/test_spilling.py::TestSpillingDebugActionDimensions::()::test_spilling[exec_option:
>  {'debug_action': None, 'default_spillable_buffer_size': '256k'} | 
> table_format: parquet/none]"
> {code}
> We need to investigate why this is happening.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Commented] (IMPALA-6981) Set pthread names

[Lars Volker (JIRA)]

[ 
https://issues.apache.org/jira/browse/IMPALA-6981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16474426#comment-16474426
 ] 

Lars Volker commented on IMPALA-6981:
-

Assuming we need to keep those printable, I think it would even make sense to 
shorten the prefix, eg to LaThNa (either automatically or manually) to make 
sure we capture as much of the ID as possible. If we drop the IDs, then it 
seems reasonable to just set the thread name to something short. I don't think 
we have that many different thread names and it could be a good ramp-up task to 
show folks around the codebase.

> Set pthread names
> -
>
> Key: IMPALA-6981
> URL: https://issues.apache.org/jira/browse/IMPALA-6981
> Project: IMPALA
>  Issue Type: Improvement
>  Components: Backend
>Affects Versions: Impala 2.13.0, Impala 3.1.0
>Reporter: Tianyi Wang
>Priority: Minor
>  Labels: newbie, ramp-up, supportability
>
> Currently impala has names for some threads but pthread names are not set. 
> pthread names can be use useful for debugging/logging, especially for some 
> process hang cases, so we should set it. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org

[jira] [Assigned] (IMPALA-7016) Statement to allow setting ownership for database

[Fredy Wijaya (JIRA)]

 [ 
https://issues.apache.org/jira/browse/IMPALA-7016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Fredy Wijaya reassigned IMPALA-7016:


Assignee: Fredy Wijaya

> Statement to allow setting ownership for database
> -
>
> Key: IMPALA-7016
> URL: https://issues.apache.org/jira/browse/IMPALA-7016
> Project: IMPALA
>  Issue Type: Sub-task
>  Components: Frontend
>Affects Versions: Impala 3.0, Impala 2.13.0
>Reporter: Adam Holley
>Assignee: Fredy Wijaya
>Priority: Major
>
> Create statement to allow setting owner on database
> {{ALTER DATABASE database_name SET OWNER [USER|ROLE] user_or_role;}}
> examples:
> ALTER DATABASE  SET OWNER USER 
> ALTER DATABASE  SET OWNER ROLE 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: issues-all-unsubscr...@impala.apache.org
For additional commands, e-mail: issues-all-h...@impala.apache.org